Android Q's Live Caption feature adds real-time subtitles to any audio or video playing on your phone

Google waited until I/O 2019 to demonstrate one of the most impressive features of Android Q. It's called Live Caption, and when enabled, you'll see any video or audio you play on your phone transcribed in real time — with extremely accurate results. Live Captions are overlaid on top of whatever app you're using, be it YouTube, Instagram, Pocket Casts, or anything else, and it also supports video chat apps like Skye and Google's own Duo. It'll even work with video or audio that you record yourself.

"For 466 million deaf and hard of hearing people around the world, captions are more than a convenience — they make content more accessible. We worked closely with the Deaf community to develop a feature that would improve access to digital media," Google wrote in a blog post.

Google CEO Sundar Pichai echoed that sentiment on stage during today's I/O keynote. "Building for everyone means ensuring that everyone can access our products," he said. "We believe technology can help us be more inclusive, and AI is providing us with new tools to dramatically improve the experience for people with disabilities."

Live Caption subtitles are created through on-device machine learning, so they work offline and don't require sending any data about your activity to the cloud. The transcription appears in a black box that can freely be moved around your phone's screen to wherever's most convenient. Even better, the feature still works even if your volume is turned down or muted; it can still analyze the source audio. Transcriptions cannot be saved for later review, however. They're there when the content is playing and gone when it's done.

This is a big addition to Google's accessibility initiatives, but it's something that could prove useful for just about everyone in various circumstances. "You can imagine all the use cases for the broader community, too," Pichai said. "For example, the ability to watch any video if you're in a meeting or on the subway without disturbing the people around you."

Live Caption can be accessed by one of the volume buttons on your phone; it appears as a software icon when the volume UI pops up. But by default in Android Q, this helpful functionality is turned off and must be enabled in accessibility settings for the icon to show up.

Source: The Verge