Sign Language to Text Translation

Sign language AI is brought out of the lab and into consumer products for the first time with a massively multilingual sign-language-to-text (SL2T) translation model. SL2T powers sign-to-text dictation in Gboard and Live Transcribe, starting with American Sign Language (ASL) to English. The model is trained on over 100,000 hours of data across more than 50 sign languages. To protect user privacy, SL2T sees sign language as a sequence of pose landmark locations rather than a raw camera feed. It translates this coordinate sequence directly into text, bypassing intermediate annotations known as "glosses" that are widely used in prior work on sign language translation. SL2T achieves a remarkable zero-shot score of 70 BLEURT on the FLEURS-ASL benchmark. The team worked on practical issues like minimizing streaming latency, preventing hallucination on non-signing inputs, ensuring fairness for left-handed signers, and improving performance for one-handed signing.