facial recognition

What is facial recognition?

Facial recognition is a category of biometric software that maps an individual's facial features to confirm their identity. Facial recognition is commonly used in security systems to identify specific individuals or users. One of the most common uses of facial recognition is for unlocking smartphones. The technology is also used for law enforcement, video surveillance and passenger screening.

Facial recognition maps a person's facial features mathematically and stores the data as a faceprint. The software uses deep learning algorithms to compare a separate video or digital image to a database of stored faceprints to verify an individual's identity.

Facial recognition is an important biometric for security and identity verification due to its contactless process. However, the accuracy of facial recognition systems is lower than that of other biometric technologies, such as fingerprint, palm or iris recognition.

How a facial recognition application works

Facial recognition works using the following four steps:

1. Detection. The process starts with finding a face in a photo, video or real-time video. This is done using computer vision, which is a process that uses artificial intelligence (AI) to enable a computer to identify data from inputs like videos or images.
2. Analysis. Facial recognition software identifies approximately 80 nodal points on a human face. In this context, nodal points are endpoints used to measure variables of a person's face, such as the length or width of their nose, the depth of their eye sockets and the shape of their cheekbones.
3. Faceprint storage. The captured and analyzed nodal points are stored in a database of other captured faces as a faceprint. A faceprint represents the nodal data as a string of numbers -- or a mathematical formula.
4. Recognition. The faceprint is then compared against a database containing data captured from faces in an image or video.

Even though the facial recognition system only uses 80 nodal points, it can quickly and accurately identify target individuals when the conditions are favorable. However, if the subject's face is partially obscured or in profile rather than facing forward, this type of software is less reliable. According to the National Institute of Standards and Technology, the incidence of false positives in facial recognition systems has been halved every two years since 1993.

Examples of facial recognition

High-quality cameras in mobile devices have made facial recognition a viable option for authentication as well as identification. Each subsequent Apple iPhone release after the iPhone X, for example, includes Face ID technology that lets users unlock their phones with a faceprint mapped by the phone's camera. The phone's software, which is designed with three-dimensional modeling to prevent being spoofed by photos or masks, captures and compares more than 30,000 variables. Face ID can be used to authenticate purchases with Apple Pay and in the iTunes Store, App Store and iBooks Store. Apple encrypts and stores faceprint data in the cloud, but authentication takes place directly on the device.

Facebook uses facial recognition software to tag individuals in photographs. Each time an individual is tagged in a photograph, the software stores mapping information about that person's facial characteristics. Once enough data has been collected, the software can use that information to identify a specific individual's face when it appears in a new photograph. To protect peoples' privacy, a feature called Photo Review notifies the Facebook member who has been identified.

Smart advertisements in airports are now able to identify the gender, ethnicity and approximate age of a passerby and target the advertisement to the person's demographic.

Other examples of facial recognition include Amazon, MasterCard and Alibaba, which have rolled out facial recognition payment methods commonly referred to as selfie pay. The Google Arts & Culture app uses facial recognition to identify museum doppelgangers by matching a real person's faceprint with a portrait's faceprint.

Developers can use Amazon Rekognition, an image analysis service that's part of the Amazon AI suite, to add facial recognition and analysis features to an application. Google provides a similar capability with its Google Cloud Vision application programming interface (API). The technology, which uses machine learning (ML) to detect, match and identify faces, is being used in a wide variety of ways, including entertainment and marketing.

Uses of facial recognition technology

Facial recognition can be used for a multitude of applications, from security to advertisements. Some use cases include the following:

• Mobile phone manufacturers. Companies, such as Apple, use facial recognition for consumer security, requiring users to unlock their phones using facial recognition.
• Airports. Airports use facial recognition for screening verification by matching a passenger's image to a physical scan of a license or passport.
• Law enforcement agencies. They use facial recognition technology to compare mugshots against databases from local, state and federal resources.
• Social media. Facebook uses facial recognition to tag individuals in photographs.
• Business security. Businesses can use facial recognition to provide employees entry to their buildings.
• Marketing. Marketers use facial recognition to determine age, gender and ethnicity to target specific audiences.
• Healthcare. Some healthcare providers are testing facial recognition to help them gain access to patient records and detect diseases.
• Banking. Some banking apps, such as Bank of America's CashPro App, use facial recognition to authenticate users.

Benefits

With the use of facial recognition comes a host of potential benefits, including the following:

• Increased authentication. With facial recognition, there's no need to physically contact a device for authentication like with other contact-based biometric authentication techniques, such as fingerprint scanners, which might not work properly if there's dirt on a person's hand.
• Improved security in certain scenarios. Facial recognition is more secure than using an easy-to-guess password and is safer in public settings where someone can look over a person's shoulder to see their passcode. However, other biometric security methods, such as iris detection, are more secure.
• Easy integration with existing security features. Applications on phones that have a front-facing camera can use facial recognition for security.
• Improved accuracy of readings. Technology improvements in ML, mapping processes and overall processing speeds have increased the accuracy of the results.
• Automated processes. In many airports, facial recognition can automate the authentication process for passengers. They can check in for their flights using biometric passports, which lets them skip the long lines and go through an automated checkpoint instead.

Security and privacy concerns

Facial recognition systems can capture and store data without an individual even knowing. This brings up many widespread data security and privacy concerns, considering faceprints as personal information. Facial recognition data could be accessed by a hacker, and an individual's information spread without them ever knowing it. Similarly, faceprint databases might be sold by companies for profit without user knowledge or consent. This data could be used by government agencies for mass surveillance, or by advertisers to track individuals. Even worse, a false positive could implicate an individual for a crime they didn't commit. If a facial recognition system isn't well-trained, it could also indicate racial biases.

Currently, there's no federal law in the U.S. that specifically protects an individual's biometric data. Bills that have been introduced but not yet passed would place limits on the use of biometric surveillance systems by government entities. There are, however, state laws -- like the Illinois Biometric Information Privacy Act -- that regulate the collection and use of biometric data by private entities. Likewise, the General Data Protection Regulation for European Union member states, also addresses biometric data.

Some facial recognition software also comes with a rating to show the accuracy of a comparison. For example, the image and video analysis service Amazon Rekognition provides what's known as a confidence score. AWS Rekognition provides users with face recognition APIs, that when used, also give a confidence score. The score rates the similarity between faces, with a higher percentage indicating the greater likelihood of a match. For example, a 99% match is a greater indicator of a real match when compared to a 65% confidence score.

Facial recognition provides convenience and security, but at the potential cost of privacy. Learn more about the privacy and bias concerns surrounding facial recognition when used by government and third-party services.