## Are facial recognition systems so bad?

by Nikolay Donets – July 11, 2019 · 2 min read

**Forbes** posted an article *London Police Facial Recognition ‘Fails 80% Of The Time And Must Stop Now’*. The question is, are facial recognition systems so bad in reality? To estimate it quantitively let’s rephrase the question to the following one: what is the probability that an individual identified as a criminal is a real offender?

First, let’s write a simple function in *python* to estimate it:

```
def if_prob(sensitivity: float, specificity: float, prob: float) -> float:
"""
:param sensitivity: true positive results, probability
:param specificity: true negative results, probability
:param prob: probability of the event
"""
numerator = sensitivity * prob
denominator = sensitivity * prob + (1.0 - specificity) * (1.0 - prob)
return numerator / denominator
```

Now we are ready to answer the question. We can assume that the sensitivity of a given facial recognition system is 0.99 with a specificity of 0.99. Therefore we have true positive results in 99 trials out of 100, and we correctly identify false-positive 99 times out of 100. According to the **Annual Abstract of Statistics** (National Office of Statistics. 2009. Retrieved 23 January 2009), the prison population was about 149 people per 100,000 in 2007. From that, the probability of criminal might be estimated as ^{149}⁄_{100000}. With these numbers we have:

>>> if_prob(0.99, 0.99, 149/100_000)

0.1287150311512887

That is, a 12.8% probability to correctly identify a criminal. And to adjust probability to the claimed one in the Forbes article, let’s set sensitivity equal to 99.5% and specificity to 99.5%:

>>> if_prob(0.995, 0.995, 149/100_000)

0.22896171487699016

That is, a 22.9% probability to correctly identify a real criminal, almost equal to what **Forbes** reported in the article. Given this, are there any ways to improve performance? One of the options might be a sequential identification with different conditions:

| Identifications in sequence | Probability of error |

|-----------------------------|------------------------------|

| 1 | (1-0.229)**1 = 0.771 = 77.1% |

| 3 | (1-0.229)**3 = 0.458 = 45.8% |

| 5 | (1-0.229)**5 = 0.272 = 27.2% |

| 7 | (1-0.229)**5 = 0.161 = 16.1% |

To summarize this short note, remember to check business requirements to estimate the performance of systems and understand their limitations in real applications. Properties of a system might be outstanding, but it can produce miserable results in wrong conditions.