Microsoft Unveils AI-Driven Customer Service for Blind and Visually Impaired

Microsoft Corp. is partnering with Be My Eyes, an app for the blind and visually impaired, to improve customer service access for this population.

Founded in 2015, Be My Eyes connects visually impaired people with sighted volunteers who can help them tackle tasks that would otherwise be impossible to complete. The startup also has an artificial intelligence tool called Be My AI, which uses OpenAI's ChatGPT-4 model to generate a description of a photo the person has taken, whether it's a product label or a store product.

The Redmond, Washington-based software giant will integrate Be My AI technology into its Microsoft Disability Answer Desk, which handles customer service calls, Be My Eyes said in a statement Wednesday. The collaboration will allow blind or visually impaired users of Microsoft products to resolve hardware problems or perform tasks such as installing a new version of Windows software or describing a PowerPoint presentation, all without human assistance.

Sam Altman praised Be My Eyes' use of OpenAI's GPT-4 at the startup's developer conference this month as an example of how his company's language model can use images as inputs and generate natural language responses.

"OpenAI is proud to work with Be My Eyes. They've used our AI models to significantly enhance the daily lives of people with low vision or blindness," Brad Lightcap, OpenAI's COO, said in a statement.

Be My Eyes, which tested AI-powered visual customer service with Microsoft users earlier this year, found that only 10% of people interacting with AI escalated to a human agent. The company also said queries were resolved faster with the Be My AI tool, with users spending an average of four minutes on a call with the AI, compared to 12 minutes with a human.