Jianfeng Gao
Distinguished Scientist & Vice President
Nouvelles et reportages
MindJourney enables AI to explore simulated 3D worlds to improve spatial interpretation
| Yuncong Yang, Reuben Tan, Swadheen Shukla, et Jianfeng Gao
MindJourney can enable AI to navigate and interpret 3D environments from limited visual input, potentially improving performance in navigation, planning, and safety-critical tasks.
CollabLLM: Teaching LLMs to collaborate with users
| Shirley Wu, Michel Galley, Baolin Peng, Swadheen Shukla, et Jianfeng Gao
Recipient of an ICML 2025 Outstanding Paper Award, CollabLLM improves how LLMs collaborate with users, including knowing when to ask questions and how to adapt tone and communication style to different situations. This approach helps move AI toward more user-centric…
Research Focus: Week of April 21, 2025
In this issue: our CHI 2025 & ICLR 2025 contributions, plus research on causal reasoning & LLMs; countering LLM jailbreak attacks; and how people use AI vs. AI-alone. Also, SVP of Microsoft Health Jim Weinstein talks rural healthcare innovation.
Research Focus: Week of March 24, 2025
In this issue, we examine a new conversation segmentation method that delivers more coherent and personalized agent conversation, and we review efforts to improve MLLMs’ understanding of geologic maps. Check out the latest research and other updates.
Magma: A foundation model for multimodal AI agents across digital and physical worlds
| Swadheen Shukla, Jianwei Yang, Reuben Tan, Qianhui Wu, et Jianfeng Gao
Explore Magma, a foundation model that can empower AI assistants to interpret environments, plan actions, and execute tasks across digital and physical spaces. Now available, learn how it advances the field of agentic AI.
ExACT: Improving AI agents’ decision-making via test-time compute scaling
| Baolin Peng, Xiao Yu, Hao Cheng, Michel Galley, Zhou Yu, et Jianfeng Gao
ExACT combines Reflective-MCTS and Exploratory Learning to improve AI agents' decision-making, enabling test-time compute scaling. Learn how these methods help agents refine strategies for state-of-the-art performance and improved computational efficiency.
Data Formulator: Exploring how AI can help analysts create rich data visualizations
| Chenglong Wang, Steven Drucker, Dan Marshall, Jeevana Priya Inala, Kori Inkpen, et Jianfeng Gao
Data Formulator investigates combining UI interactions with natural language input. Powered by AI, it can help users create or adapt visualizations and supports continuous refinement through an iterative process. Now available on GitHub.
Microsoft Research Forum Episode 4: The future of multimodal models, a new “small” language model, and other AI updates
Explore multimodal & small language models, plus advanced benchmarks for AI evaluation. Microsoft researchers are working on breakthroughs in weather prediction, materials design, even a new kind of computer for AI inference and hard optimization problems.
LLM profiling guides KV cache optimization
| Liyuan Liu et Jianfeng Gao
LLMs rely on memory-intensive mechanisms like the key-value (KV) cache to store and quickly retrieve data. FastGen optimizes KV cache usage, reducing LLM memory demands by up to 50% while maintaining performance.