MindJourney is a framework that equips AI agents with a “simulation loop” to explore hypothetical 3D viewpoints before answering spatial reasoning questions—tackling a core limitation of vision-language models (VLMs), which recognize objects well in 2D…
Deterministic security for AI agents AI agents perform consequential actions while processing data from various sources, including trusted collaborators and the public Web. It is crucial that AI agents handle this data with care: confidential…
The goal of this project is to understand the broad phenomenon of how people relate to AI, how they make sense of what it is, what it can do, what it might be; what kinds…
Autonomous malware classification Project Ire is an LLM-powered autonomous malware classification system. For inquiries, please direct mail to: ireinquiries@service.microsoft.com
Optimize ANY agent with ANY framework We present Agent Lightning (opens in new tab), a flexible and extensible framework that enables seamless agent optimization for any existing agent framework. Here agent optimization includes various data-driven…
Coordinate-Free Visual Grounding for GUI Agents One of the principal challenges in building VLM-powered GUI agents is visual grounding, i.e., localizing the appropriate screen region for action execution based on both the visual content and…
Project Aqua develops tools for automatic quantization of LLMs TBD
Discovering a periodic table of machine learning connecting over 23 different methods As the field of representation learning grows, there has been a proliferation of different loss functions to solve different classes of problems. We…
This project aims to inform UX interventions for fostering appropriate reliance on generative AI systems by: See also: Appropriate reliance research initiative
