GizmoChina Explains: How Does Facial Recognition Work?

Just like 18:9 displays, facial recognition is one of the features manufacturers are adding to phones this year. While the tech has gained popularity in the past few months, it’s worth mentioning that facial recognition has been found on smartphones for years.

The iPhone X may have made it popular but it has been available on Android debuting with Android 4.0 Ice Cream Sandwich, though not as advanced and secure as that of the iPhone X.

Facial Recognition on Android

Setting up face unlock on Android back then was fairly simple. After accessing the menu under settings, you will be required to set up your face by positioning it within a dotted circle and the front facing camera would capture your facial details. To increase accuracy, you will be required to try it under different lighting conditions and under different conditions (glasses on/off, hat on etc).

There was a caveat though, Google knew it wasn’t secure enough, so you were asked to set up an alternative security lock. And now, most manufacturers don’t even bother to include the feature anymore, ever since fingerprint scanners have become mainstream.

On my OnePlus 3T, all you have is swipe, pattern, PIN, and password, in addition to fingerprint recognition. The Zoji Z6, which acts as my secondary phone, adds voice unlock along with the above mentioned security features.

 

Iris Recognition

But that wasn’t the end of face unlock. While it took a break, manufacturers branched into something else – Iris Recognition.
The first phone to come with an iris scanner was the Fujitsu ARROWS NX-F04G. It was released in 2015 and made use of an IR LED for illuminating the pattern of the iris, which according to research doesn’t change after the age of two years; and an IR camera for  capturing the pattern.

The image is then stored in the database and used to compare against that of the person who wants to unlock the phone. At the end of 2015, Microsoft released the Lumia 950 and Lumia 950 XL, joining Fujitsu, as smartphones with Iris scanners.

Nearly a year later, Samsung joined the short list with the ill-fated Galaxy Note7. This year, Samsung launched the Galaxy S8 and S8+ and both of them have iris scanners. Samsung touted them as being airtight and you could even use it for payment, just the way you used the fingerprint scanner.

However, the guys at Chaos Computer Club (CCC), a European hacker association didn’t like the way Samsung bragged about its security and decided to disprove their claim. They were able to fool the scanner.

How they did it was actually pretty simple.

All they had to do was take a picture of the owner’s eye using a 200mm lens from 15 feet away. Then they had it printed out on paper and covered it with wet contact lens to imitate the curvature of real eyes. After pointing the phone towards it, it unlocked the device.

CCC in its official article says it will be even easier to unlock with the tons of high resolution selfies on the internet, so you might not need to even take a picture of the owner of the phone. Their suggestion is to use a PIN as it is more secure than biometric scanners.

Windows Hello Facial Recognition

Back to Facial Recognition. Microsoft announced Windows Hello the same year as the company announced the Lumia 950 and 950 XL, which is a windows 10 biometric technology system that includes the iris scanner on the Lumia 950 and 950 XL, the fingerprint reader on windows 10 laptops, and facial recognition on the Surface Book laptop and the Surface Pro 4.

Both the surface book and laptop feature a near IR sensor that does the capturing and is more effective under varying light conditions.

So how does Windows Hello face recognition work?

Using an algorithm, the face is detected in the camera stream. Facial landmark points such as the nose, eyes, and mouth are located. The algorithm also ensures that the user is facing the device at an angle of ±15 degrees.

The algorithm then uses the landmarks as anchor points and takes several samples of the face to build a representation. The result is a histogram representing the light and dark differences around specific areas. So, the image of the user is actually never stored.

Microsoft says the representation of the user in front of the PC is then compared to the one enrolled. And before it accepts it as a match, it has to cross a certain machine-learned threshold. This threshold increases if there are multiple users enrolled on the device.

 

Microsoft measures the accuracy of its facial recognition in three ways:
1. False positive – also called false acception rate (FAR), this is the likelihood of a random person who gains physical access to your PC being recognized as you. This number should be as low as possible and for Windows Hello, it is less than 0.001 or 1/100,000 FAR.

2. True positive – this is the likelihood of you being correctly matched to your enrolled profile. This number should be high and Windows Hello scores 95% for a single enrolled user.

3. False negative – is the likelihood of you being denied access even though your profile is enrolled and Microsoft says it is 5% for a single enrolled user.

Windows Hello works pretty well judging from the tons of videos on the internet. It cant be fooled by a photograph, it won’t unlock if your face is partially covered and it unlocks fast in the dark thanks to the near IR sensor.

 

 

Apple Face ID

 

Apple joined the facial recognition train this year with its iPhone X. The iPhone X doesn’t have Touch ID due to its bezel-less design and Apple didn’t want to put it on the back of the phone like a number of Android manufacturers do. Instead, it came up with Face ID, a facial recognition system it claims is more secure than TouchID.

Face ID captures your face with the aid of light projector and a bunch of sensors — Apple collectively refers to it as the TrueDepth Camera System.

 

The process the TrueDepth system uses to capture a 3D map of your face is explained below:

1.The proximity sensor and ambient light sensor help to determine the amount of illumination needed for face recognition.
2.The flood illuminator produces IR light to illuminate your face. The IR light is not visible to the human eye and and works effectively in poorly illuminated or even bright areas (outdoors).
3. The dot projector produces tens of thousands dots of IR light to create a 3D map of your face.
4. The infrared camera captures images of the dot pattern and the IR light reflected back by your face.

The recognition process is similar to that of other biometric ones – compare the stored template against the presented pattern (verification image) and see if it is a match. The stored template and verification image won’t be the same since the capture conditions will always be different, so it uses a threshold to grant access.

The threshold score is said to be between 0 and 1. If the score is closer to 1 then it is the same face (or biometric) and if it is closer to 0, then it is not the same person.

 

Apple explains the step for verification:
1. IR images are sent from the camera to the phone’s neural engine to build a 3D map of the face.
2. The map is compared against the stored template.
3. The processor calculates if the verification and stored templates match based on a comparison score of the similarity between your images.
4. Phone unlocks if the comparison score is higher than a certain threshold value.

 

So what happens if your facial appearance changes, say you grow a beard, shave your moustache, or apply makeup?
Apple says that FaceID continually updates the stored model by occasionally saving images from successful login maps. So it kind of a continuous enrollment.

READ MORE: GizmoChina Explains: How Does An eSIM Work?

How secure is Face ID?

According to Apple, the probability of a random person unlocking your iPhone X is 1 in 1,000,000 which is way higher than Touch ID’s 1 in 50,000. It however differs for twins and siblings that share the same physical features or even siblings under 13 whose distinct features are not fully developed. Apple suggests using a pass-code for authentication in this case.

Bottom line is that Face ID or any form of facial recognition isn’t foolproof, but unlike the basic model of facial recognition, Windows Hello and Face ID provide a much more secure way of keeping your device protected.

As the technology develops, facial recognition could potentially be our biometric verification standard on smartphones and other gadgets. Apple took the leap skipping TouchID in favour of Face ID, and other manufacturers could follow suit.