Meta’s live translation feature on its AI glasses marks a significant leap toward realizing the long-held dream of real-time language translation in everyday life. Inspired by fictional devices like the babel fish and Star Trek’s universal translator, this technology is now a reality—powered by innovation, relentless engineering, and a deep commitment to human connection. The journey began as a prototype for the then-unannounced Meta Ray-Ban Display. However, the team quickly realized that the already-available Ray-Ban Meta glasses offered the ideal platform. Its five-microphone array enabled beamforming, allowing the glasses to distinguish between the wearer and their conversation partner—critical for accurate translation. Rather than relying on a visual display, the system uses open-ear speakers to deliver translated speech in near-real time, keeping users engaged with their surroundings. The technical challenge was immense. When a French speaker talks to someone wearing Ray-Ban Meta glasses who speaks Spanish, the audio is first transcribed into text. That text is then translated from French to Spanish, and a text-to-speech model converts it back into spoken Spanish—delivered through the glasses’ speakers. All of this happens on-device, without requiring an internet connection, making it ideal for travel and low-connectivity environments. To make this possible, the team had to optimize complex models to run efficiently on the glasses’ hardware, avoiding overheating and memory limitations. They also reduced latency from over 5 seconds to just 2.7 seconds—a 46% improvement—by enabling the model to process speech in a streaming fashion, translating and generating audio word by word without waiting for full sentences. This on-device processing, while powerful, presented unique challenges. With no server-side logging, the team had to rely on constant, real-world testing to ensure quality and performance. “We had to test it non-stop,” says Product Manager Emerson Qin. “There’s no better way to catch issues than by using it in real situations.” The feature is still evolving. Accuracy isn’t perfect, and latency is still noticeable. But Meta launched it early to gather feedback and improve over time. Adding new languages is a major undertaking—each requires custom model training and evaluation tailored to the specific device. As Qin notes, “Everything has to be redone per device. It’s not scalable yet, so we ask for patience.” Despite these hurdles, early users are already experiencing life-changing benefits. People are using live translation to connect with family members who speak different languages, navigate foreign countries, and participate in international meetings. It’s helping travelers understand museum guides, allowing future in-laws to bond over shared stories, and enabling global collaboration without the need for third-party translators. The open-ear design stands out—unlike earbuds, it doesn’t isolate users from their environment. Instead, it keeps them present in the moment. The Meta AI app also provides real-time transcriptions in both languages, allowing users to share their phone with conversation partners for added clarity. At the heart of the project is a team driven by a shared vision. “This is the best team I’ve ever worked on,” says Research Scientist Baiyang Liu. “It’s not just about the tech—it’s about people who believe in it and are willing to go the extra mile.” Product Designer Amy Pu adds, “Many of us have multilingual backgrounds. We know how powerful it is to understand a loved one’s words, even if you don’t share a language. That’s the real goal: to make understanding accessible, anytime, anywhere.” Meta’s live translation on AI glasses is more than a feature—it’s a step toward a more connected world. As the team continues to expand language support and refine performance, the dream of an everyday universal translator is becoming real, one conversation at a time.