Research talk: WebQA: Multihop and multimodal
- Yonatan Bisk | Carnegie Mellon University
- Microsoft Research Summit 2021 | Deep Learning & Large-Scale AI
Web search is fundamentally multimodal and multihop. Often, even before asking a question, individuals go directly to image search to find answers. Further, rarely do we find an answer from a single source, opting instead to aggregate information and reason through implications. Despite the frequency of this everyday occurrence, at present there is no unified question-answering benchmark that requires a single model to answer long-form natural language questions from text and open-ended visual sources that is akin to human experience. The researchers propose to bridge this gap between the natural language and computer vision communities with WebQA. They show that multihop text queries are difficult for a large-scale transformer model, and they also show that existing multi-modal transformers and visual representations do not perform well on open-domain visual queries. Our challenge for the community is to create a unified multimodal reasoning model that seamlessly transitions and reasons regardless of the source modality.
Learn more about the 2021 Microsoft Research Summit: https://Aka.ms/researchsummit (opens in new tab)
-
-
Yonatan Bisk
Professor
CMU
-
-
Deep Learning & Large-Scale AI
-
Opening remarks: Deep Learning and Large-Scale AI
- Ahmed Awadallah
-
-
-
-
-
-
Roundtable discussion: Efficient and adaptable large-scale AI
- Ahmed Awadallah,
- Jianfeng Gao,
- Danqi Chen
-
-
-
-
Panel: Large-scale neural platform models: Opportunities, concerns, and directions
- Eric Horvitz,
- Miles Brundage,
- Yejin Choi
-
-
-
Research talk: WebQA: Multihop and multimodal
- Yonatan Bisk
-
-
Roundtable discussion: Beyond language models: Knowledge, multiple modalities, and more
- Yonatan Bisk,
- Daniel McDuff,
- Dragomir Radev
-
Closing remarks: Deep Learning and Large Scale AI
- Jianfeng Gao