Your face could soon help AI improve facial recognition technology

People are rightfully worried about their faces being part of the 60 billion that were captured by a company called Clearview AI. Unless you have somehow been able to dodge security cameras, family photos, video from people recording events on their phones or snapping photos for social media, then a picture of your face is likely already sitting deep inside their database. With only 333 million people living in the United States, it’s very likely that most people’s likenesses are captured inside that facial database multiple times.

The original founder and CEO of the company, Hoan Ton-That, was apparently not a fan of using their database to assist with government surveillance. But he recently resigned from his position. The new company leader, Hal Lambert, has stated that one of the new goals of Clearview is to get its facial database into the hands of the government, and hopefully land some lucrative government contracts.

In terms of the database itself, Clearview says it acquired pictures of people by scraping the internet in a variety of ways that included both manual capture techniques and also automation driven by AI. Earlier this year, company officials said the database had 60 billion faces, and that number continues to grow.

It’s actually a surprisingly easy process to identify and grab faces for a database, either manually by hand or automatically using AI. In fact, I recently came across a really interesting new game called Faceminer on the Steam platform that challenges players to do just that. The game is surprisingly accurate, asking players to purchase different kinds of media and then identify clear pictures of faces for inclusion in a database. You first do everything manually and then later are able to set up an AI to help out. Perhaps to avoid being overly political, the game is set in 1999 and comes complete with a Windows 98 desktop environment to control everything. But the techniques used today are nearly identical, just faster and more efficient.

Like in the Faceminer game, the photos sitting in the Clearview AI database are just that, faces captured from social media and other sources, but without much context other than the face itself. So, how would that be of any use or value to a government agency?

To answer that, you have to look at places where facial recognition in government works, and where it falls short. For example, reviews of the technology in certain pilot programs, such as those in use by the TSA, have generally been very positive when it comes to accuracy. Those success stories are juxtaposed against quite a few high profile failures where people have been arrested or detained based on a faulty facial recognition identification.

One key difference is that, in the case of the TSA program, it’s operating in a perfect environment, namely a well-lit airport or transportation hub using high resolution cameras focused on people standing around or moving through designated areas. But out in the rest of the world, most of the CCTV feeds are either low-resolution or capture their images from very far away. For example, a camera mounted on the side of a building looking down at a street is not generally going to be able to capture a pristine image of someone walking by in the distance. And if that person is wearing sunglasses or a hat, it further complicates recognition.

Those problems have negatively impacted facial recognition since its inception. They are generally put into a category of challenges known as Human Identification at a Distance or HID. Basically, with most biometric identification methods, a person is interacting very closely or directly with a machine, like when they place their hand on a glass plate for fingerprint recognition or lean into a special camera to have their iris scanned. But with facial recognition, images are sometimes captured in less than optimal circumstances, such as in dark conditions, from far away or with a low-resolution camera.

When that happens, the AI driving a facial recognition platform needs to rely on algorithms to fill in the missing details in a process called hallucination. How it works is that face hallucination algorithms analyze a low-resolution or blurry image to try and clean it up before submitting it to the main AI for comparison with something like a criminal database. It guesses at what features might be hidden in the subpar image, either because those elements are too blurry or blocked behind something like sunglasses. 

It works by examining photographs of people with similar facial characteristics. Those comparable faces might be of people who are about the same age or have similar skin tones. The hallucination algorithm then fills in the blank or blurry areas of the target face using an amalgamation of features from similar images. And the more close-matching, high-resolution photos it has to work with, the more accurate the algorithm will be at filling in key details for better identification.

And that is where the 60 billion faces captured by Clearview will likely provide the most value to government agencies. Yes, people should be rightly concerned about their image and likeness being captured and used without their permission, but at least the data will likely help to improve the accuracy of facial recognition systems. Armed with a massive database of diverse faces to help clean up poor quality images, the accuracy and dependability of facial recognition systems can only improve. And just about everyone’s face is probably soon going to help make that happen.

John Breeden II is an award-winning journalist and reviewer with over 20 years of experience covering technology. He is the CEO of the Tech Writers Bureau, a group that creates technological thought leadership content for organizations of all sizes. Twitter: @LabGuys