Microsoft Announces New AI Feature, Auto-Captions Images For Visually Impaired

A new Artificial Intelligence-based system from Microsoft intends to automatically caption images in documents and emails so that software for visual issues can read it out.

A group of researchers from Microsoft elucidated their machine learning model in a paper on preprint repository arXiv.

Microsoft’s model uses VIsual VOcabulary pre-training (VIVO). The system leverages hefty amounts of paired image-tag data to learn a visual language. A 2nd dataset of appropriately captioned pictures is then used to aid teach the AI system how to describe the pictures properly.

A software engineering manager with Microsoft’s Artificial Intelligence platform group, Saqib Shaikh mentioned,

“Ideally, everyone would include alt text for all images in documents, on the web, in social media – as this enables people who are blind to access the content and participate in the conversation. But, alas, people don’t.”

Overall, the researchers anticipate Artificial Intelligence to deliver two times the performance of Microsoft’s present captioning system.

Microsoft’s CTO of Azure AI Cognitive Services, Xuedong Huang said that they are taking the AI breakthrough to Azure as a platform to serve a comprehensive set of users. Huang further added that it is not just a breakthrough on the research. Rather, the time it took to turn that breakthrough into production on Azure is also a breakthrough, as per him.

The enhanced auto-captioning feature is also anticipated to be presented on Outlook, Word, and PowerPoint later this year.