When Phones Learn to Read Signs
A phone keyboard usually assumes that its user will type or speak. Google DeepMind is beginning to test a different premise: that someone should also be able to sign.
On August 12, 2026, the company introduced SL2T, a sign-language-to-text model powering new features in Gboard and Live Transcribe. The initial release translates American Sign Language into English on Pixel 11 phones, with additional devices and sign languages promised later. A user can sign a search query, message, document passage, or Gemini request instead of entering it through written or spoken English. Live Transcribe can also turn a signed response into text during an informal conversation.
A language of movement
That may sound like another input method, but the human difference is larger. Sign languages are independent natural languages, not spoken languages transferred onto the hands. Meaning can depend on the simultaneous movement of hands, face, head, arms, and torso, as well as the use of physical space.
The World Federation of the Deaf estimates that more than 70 million people belong to signing Deaf communities and recognizes more than 200 national sign languages. For people who use one of those languages most naturally, access should not always require communicating through written English first.
What the model sees
DeepMind says SL2T was trained on more than 100,000 hours of material spanning over 50 sign languages, with roughly one-quarter of the data in ASL. Instead of sending ordinary camera video to the translation system, MediaPipe Holistic converts the signer’s movements into geometric pose landmarks on the device. The video is then discarded, while those coordinates are sent to Google’s servers for translation.
The joint impact report says user inputs and outputs are not logged unless a person explicitly authorizes collection for an evaluation study. This approach reduces the exposure of faces, physical appearance, and surroundings, but privacy and performance cannot be separated neatly. Removing visual detail also removes information. The report notes that the representation does not track the tongue, capture environmental references, or reliably distinguish some forms of contact and depth.
Useful does not mean universal
The system still has important boundaries. Lighting, camera angle, regional language, slang, complex grammar, and anatomical or motor differences can affect its output. It currently follows one signer rather than interpreting a multi-person exchange.
The release is intended for short, low-stakes tasks such as messaging, searches, assistant commands, and casual one-to-one conversations. Its own impact report says it should not be used for medical care, legal proceedings, formal education, employment decisions, government services, or other situations where mistranslation could cause material harm. This is an accessibility tool, not a replacement for qualified human interpreters.
Who gets to define success?
DeepMind says Deaf people participated in the project’s conception, data collection, user studies, and evaluation. It also convened the AI Sign Language Advisory Committee, whose contributors included representatives from Deaf advocacy, cultural, professional, and academic organizations.
That is more consequential than consultation added at the end of development. Still, a committee established by a company is not the same thing as community control. The lasting test will be whether Deaf communities can influence which languages are supported, how performance is measured, what data practices are acceptable, and when a product should not be deployed.
Benchmark scores can show improvement, but they cannot decide whether a system works across regional signing, Black ASL, left-handed signing, motor diversity, dim rooms, older phones, or the uneven conditions of daily life. Nor can they answer whether users trust it enough to rely on it.
SL2T changes what a consumer phone can accept as language. What happens next depends on whether its development treats sign languages as living languages—and Deaf users as decision-makers rather than merely beneficiaries.
Sources and Further Reading
- Google DeepMind: Putting sign language AI into users’ hands
- Google DeepMind and AISLAC: Joint Impact Report for SL2T 1.0
- FLEURS-ASL research paper
- FSboard research paper
- World Federation of the Deaf: Frequently Asked Questions
Subscribe to Science and Society
Get thoughtful stories about science, technology, and the choices shaping our future delivered to your inbox.
Science changes possibilities. Society decides what happens next.
