Multi-Device Experiences (MDX) Toolkit

The Multi-Device Experiences (MDX) toolkit is an extensible cloud AI service hosted in Azure that enables research and development on task-focused scenarios spanning multiple devices. The toolkit enables developers to create intelligent, interactive experiences that capitalize on the strengths of using multiple devices simultaneously, such as an Echo plus an iPad. It provides support for user authentication and authorization, session management, and state reflection across devices. The toolkit leverages machine intelligence for intent understanding and the contextual answering of questions. Our demo features a proof-of-concept cooking scenario powered by the MDX toolkit, which allows users to follow a recipe hands-free, with voice navigation and contextual Q&A. The toolkit is designed to be extended to scenarios beyond cooking, including education, gaming, and accessibility. Our exhibit also describes the MDX research platform, which enables researchers to stand up new multi-device scenarios with minimal infrastructure investment, easily onboard new devices and device form factors, and access collated and synchronized logs from the service, model providers, and devices.

Series: Microsoft Research Faculty Summit