Google has unveiled a groundbreaking accessibility feature for its new Pixel 11 smartphone that converts American Sign Language into written text. Announced ahead of the Made by Google event, the technology leverages an AI model called SL2T developed by Google DeepMind. The feature integrates into Gboard and Live Transcribe, enabling users to communicate through sign language instead of typing for everyday tasks like composing emails, performing searches, and drafting documents.
The SL2T system works by using computer vision to track 130 distinct points across a user’s face, hands, and body, transforming physical movements into geometric coordinates that the AI translates directly into English text. Unlike previous approaches that converted signs into intermediate written labels before translation, this direct method captures facial expressions and spatial movements that carry essential meaning. Google trained the model using over 100,000 hours of footage across more than 50 sign languages, with roughly one-quarter dedicated to ASL, and programmed it to recognize both one-handed signing and left-handed users.
Despite its innovation, Google acknowledges significant limitations in the initial release. The technology struggles with regional variations, complex grammar, rapid fingerspelling, and low-light conditions. The company emphasizes that SL2T functions as an assistive input tool for casual communication only and explicitly cannot replace professional interpreters for medical appointments, legal proceedings, or formal settings. Google collaborated with Deaf employees, advocacy organizations, and sign language experts throughout development and plans to expand the feature to additional sign languages in the future.
Originally reported by Mashable →
About the Author
