Ray-Ban Meta: Companion app contains code for facial recognition

Meta has integrated code for a facial recognition feature into its Meta-AI app over several app updates this year, Wired reports. The companion app is required for setting up and central functions of the Ray-Ban Meta glasses, as well as other smart glasses from the company.

The feature, internally referred to as “NameTag” and not yet activated, is intended to recognize people captured by the glasses' camera and notify the wearer if a known face is identified. Wired cites its analysis of the app code. Two independent security experts have also confirmed the key findings. “The feature is not yet exposed to consumers but seems nearly ready to go,” says security researcher Cooper Quintin, who examined the code.

A Meta spokesperson did not fundamentally dispute the report. The company had previously stated that it was exploring such features, and the components discovered in the code were merely evidence of this work.

“To date, no such feature has been introduced to users, and no final decision has been made on whether and how we will proceed. Should we introduce something, we will do so thoughtfully and with full transparency.”

What the code is supposed to suggest

The analysis of the app code is said to show that the inactive system called “NameTag” is based on three local AI models. One recognizes faces, another creates facial crops, and a third converts the captured faces into biometric characteristic profiles, so-called Faceprints. In an activated state, the function would compare faces captured by the glasses' camera with Faceprints stored on the smartphone.

According to the Wired analysis, this database is designed to retrieve Faceprints from Meta servers and store them locally on users' devices. If the system detects a match, it is supposed to notify the wearer. Unrecognized faces, on the other hand, would be cropped, indexed, and stored in a folder labeled “Pending”.

So far, according to Wired, only fragments of the user interface have been found. However, the function was revised in May and renamed “Connections.” It remains unclear whose faces could be included in the recognition database, how corresponding profiles would be created, and how many people would ultimately be identifiable. Meta itself denies in its statement that it is building a central facial database.

From Facebook tags to Smart Glasses

The clarification addresses a sensitive point: In 2021, Meta announced it would discontinue its previous facial recognition system for automatic tagging suggestions and delete the Faceprints of more than one billion users. The system had brought the company years of privacy criticism and expensive settlements, including $650 million in Illinois and $1.4 billion with Texas.

There have been indications for some time that Meta is working on facial recognition specifically for its smart glasses. In February, the New York Times, among others, reported on this. However, it is surprising how far the development is said to have progressed and that key components are apparently already integrated into the Meta-AI app.

A detailed analysis of the app code by independent security and privacy researcher Buchodi is available online.

(dahe)