Agent AI

Publications

View by:

- LLM Agent – Retrieve What You Need: A Mutual Learning Framework for Open-domain Question Answering
  
  Dingmin Wang, Qiuyuan Huang, Matthew Jackson, Jianfeng Gao
  
  Proceedings of Transactions of the Association for Computational Linguistics (TACL) | April 2024, pp. 247-263
  
  Publication Project
- Agent AI Towards a Holistic Intelligence
  
  Qiuyuan Huang, Naoki Wake, Bidipta Sarkar, Zane Durante, Ran Gong, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Noboru Kuno, Ade Famoti, Ashley J. Llorens, John Langford, Hoi Vo, Fei-Fei Li, Katsushi Ikeuchi, Jianfeng Gao
  
  arXiv: 2403.00833 | February 2024
  
  Publication Publication
- An Interactive Agent Foundation Model
  
  Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein, Demetri Terzopoulos, Ade Famoti, Noboru Kuno, Ashley J. Llorens, Hoi Vo, Katsushi Ikeuchi, Fei-Fei Li, Jianfeng Gao, Naoki Wake, Qiuyuan Huang
  
  February 2024
  
  Publication Publication
- MindAgent: Emergent Gaming Interaction
  
  Steven Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-chun Zhu, Demetri Terzopoulos, Feifei Li, Jianfeng Gao
  
  Processings of NAACL 2024 | January 2024
  
  Publication Project
- Agent AI: Surveying the Horizons of Multimodal Interaction
  
  Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong , Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, Jianfeng Gao
  
  arXiv: 2401.03568 | January 2024
  
  Publication
- Multimodal Agent — Localized Symbolic Knowledge Distillation for Visual Commonsense Models
  
  Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Peter West, Qiuyuan Huang, Jianfeng Gao, Ali Farhadi, Yejin Choi
  
  Proceedings of NeurIPS 2023 | October 2023
  
  Publication
- Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models
  
  Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron L Halfaker, Liqun Shao, Daniel McDuff, Ahmed Awadallah, Dragomir Radev, Jianfeng Gao
  
  Proceedings of ACL 2023 | May 2023
  
  Publication Publication Project
- ArK: Augmented Reality with Knowledge Emergent Infrastructure
  
  Qiuyuan Huang, James Park, Abhinav Gupta, Pan Lu, Paul Bennett, Ran Gong, Subhojit Som, Baolin Peng, Owais Khan Mohammed, Chris Pal, Yejin Choi, Jianfeng Gao
  
  Selected as the project in HackBox 2023; Shipped to Office Teams | April 2023
  
  Publication Project
- VLC: Training Vision-Language Transformers from Captions
  
  Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  Proceedings of Transactions on Machine Learning Research | May 2022
  
  Publication Project
- KAT: A Knowledge Augmented Transformer for Vision-and-Language
  
  Liangke Gui, Borui Wang, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  NAACL 2022. Long paper, Oral. | January 2022
  
  No.1 of OK-VQA Leaderboard (2021).
  
  Publication
- NICE: Neural Image Commenting with Empathy
  
  Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Xiang Gao, Hamid Palangi, Jianfeng Wang, Kenneth Forbus, Jianfeng Gao
  
  EMNLP 2021 | October 2021
  
  Publication
- Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
  
  Kezhen Chen, Qiuyuan Huang, Hamid Palangi, Paul Smolensky, Kenneth D. Forbus, Jianfeng Gao
  
  ICML 2020 | July 2020
  
  Publication Project
- Vision-Language Navigation Policy Learning and Adaptation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-fang Wang, William Wang, Lei Zhang
  
  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 | February 2020
  
  Publication Publication Project
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang
  
  CVPR 2019 | June 2019
  
  Best Student Paper Award in CVPR 2019.
  
  Publication Video Project Project
- Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
  
  Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He
  
  AAAI 2019 | January 2019
  
  Publication Project Project
- Turbo Learning for CaptionBot and DrawingBot
  
  Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang
  
  NeurIPS 2018 | December 2018
  
  PDF Publication Project

- LLM Agent – Retrieve What You Need: A Mutual Learning Framework for Open-domain Question Answering
  
  Dingmin Wang, Qiuyuan Huang, Matthew Jackson, Jianfeng Gao
  
  Proceedings of Transactions of the Association for Computational Linguistics (TACL) | April 2024, pp. 247-263
  
  Publication Project
- Agent AI Towards a Holistic Intelligence
  
  Qiuyuan Huang, Naoki Wake, Bidipta Sarkar, Zane Durante, Ran Gong, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Noboru Kuno, Ade Famoti, Ashley J. Llorens, John Langford, Hoi Vo, Fei-Fei Li, Katsushi Ikeuchi, Jianfeng Gao
  
  arXiv: 2403.00833 | February 2024
  
  Publication Publication
- An Interactive Agent Foundation Model
  
  Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein, Demetri Terzopoulos, Ade Famoti, Noboru Kuno, Ashley J. Llorens, Hoi Vo, Katsushi Ikeuchi, Fei-Fei Li, Jianfeng Gao, Naoki Wake, Qiuyuan Huang
  
  February 2024
  
  Publication Publication
- MindAgent: Emergent Gaming Interaction
  
  Steven Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-chun Zhu, Demetri Terzopoulos, Feifei Li, Jianfeng Gao
  
  Processings of NAACL 2024 | January 2024
  
  Publication Project
- Agent AI: Surveying the Horizons of Multimodal Interaction
  
  Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong , Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, Jianfeng Gao
  
  arXiv: 2401.03568 | January 2024
  
  Publication
- Multimodal Agent — Localized Symbolic Knowledge Distillation for Visual Commonsense Models
  
  Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Peter West, Qiuyuan Huang, Jianfeng Gao, Ali Farhadi, Yejin Choi
  
  Proceedings of NeurIPS 2023 | October 2023
  
  Publication
- Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models
  
  Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron L Halfaker, Liqun Shao, Daniel McDuff, Ahmed Awadallah, Dragomir Radev, Jianfeng Gao
  
  Proceedings of ACL 2023 | May 2023
  
  Publication Publication Project
- ArK: Augmented Reality with Knowledge Emergent Infrastructure
  
  Qiuyuan Huang, James Park, Abhinav Gupta, Pan Lu, Paul Bennett, Ran Gong, Subhojit Som, Baolin Peng, Owais Khan Mohammed, Chris Pal, Yejin Choi, Jianfeng Gao
  
  Selected as the project in HackBox 2023; Shipped to Office Teams | April 2023
  
  Publication Project
- VLC: Training Vision-Language Transformers from Captions
  
  Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  Proceedings of Transactions on Machine Learning Research | May 2022
  
  Publication Project
- KAT: A Knowledge Augmented Transformer for Vision-and-Language
  
  Liangke Gui, Borui Wang, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  NAACL 2022. Long paper, Oral. | January 2022
  
  No.1 of OK-VQA Leaderboard (2021).
  
  Publication
- NICE: Neural Image Commenting with Empathy
  
  Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Xiang Gao, Hamid Palangi, Jianfeng Wang, Kenneth Forbus, Jianfeng Gao
  
  EMNLP 2021 | October 2021
  
  Publication
- Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
  
  Kezhen Chen, Qiuyuan Huang, Hamid Palangi, Paul Smolensky, Kenneth D. Forbus, Jianfeng Gao
  
  ICML 2020 | July 2020
  
  Publication Project
- Vision-Language Navigation Policy Learning and Adaptation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-fang Wang, William Wang, Lei Zhang
  
  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 | February 2020
  
  Publication Publication Project
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang
  
  CVPR 2019 | June 2019
  
  Best Student Paper Award in CVPR 2019.
  
  Publication Video Project Project
- Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
  
  Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He
  
  AAAI 2019 | January 2019
  
  Publication Project Project
- Turbo Learning for CaptionBot and DrawingBot
  
  Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang
  
  NeurIPS 2018 | December 2018
  
  PDF Publication Project
- Agent AI Towards a Holistic Intelligence
  
  Qiuyuan Huang, Naoki Wake, Bidipta Sarkar, Zane Durante, Ran Gong, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Noboru Kuno, Ade Famoti, Ashley J. Llorens, John Langford, Hoi Vo, Fei-Fei Li, Katsushi Ikeuchi, Jianfeng Gao
  
  arXiv: 2403.00833 | February 2024
  
  Publication Publication
- Agent AI: Surveying the Horizons of Multimodal Interaction
  
  Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong , Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, Jianfeng Gao
  
  arXiv: 2401.03568 | January 2024
  
  Publication
- ArK: Augmented Reality with Knowledge Emergent Infrastructure
  
  Qiuyuan Huang, James Park, Abhinav Gupta, Pan Lu, Paul Bennett, Ran Gong, Subhojit Som, Baolin Peng, Owais Khan Mohammed, Chris Pal, Yejin Choi, Jianfeng Gao
  
  Selected as the project in HackBox 2023; Shipped to Office Teams | April 2023
  
  Publication Project
- VLC: Training Vision-Language Transformers from Captions
  
  Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  Proceedings of Transactions on Machine Learning Research | May 2022
  
  Publication Project
- KAT: A Knowledge Augmented Transformer for Vision-and-Language
  
  Liangke Gui, Borui Wang, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  NAACL 2022. Long paper, Oral. | January 2022
  
  No.1 of OK-VQA Leaderboard (2021).
  
  Publication
- NICE: Neural Image Commenting with Empathy
  
  Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Xiang Gao, Hamid Palangi, Jianfeng Wang, Kenneth Forbus, Jianfeng Gao
  
  EMNLP 2021 | October 2021
  
  Publication
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang
  
  CVPR 2019 | June 2019
  
  Best Student Paper Award in CVPR 2019.
  
  Publication Video Project Project
- Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
  
  Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He
  
  AAAI 2019 | January 2019
  
  Publication Project Project
- Turbo Learning for CaptionBot and DrawingBot
  
  Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang
  
  NeurIPS 2018 | December 2018
  
  PDF Publication Project
- LLM Agent – Retrieve What You Need: A Mutual Learning Framework for Open-domain Question Answering
  
  Dingmin Wang, Qiuyuan Huang, Matthew Jackson, Jianfeng Gao
  
  Proceedings of Transactions of the Association for Computational Linguistics (TACL) | April 2024, pp. 247-263
  
  Publication Project
- Agent AI Towards a Holistic Intelligence
  
  Qiuyuan Huang, Naoki Wake, Bidipta Sarkar, Zane Durante, Ran Gong, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Noboru Kuno, Ade Famoti, Ashley J. Llorens, John Langford, Hoi Vo, Fei-Fei Li, Katsushi Ikeuchi, Jianfeng Gao
  
  arXiv: 2403.00833 | February 2024
  
  Publication Publication
- An Interactive Agent Foundation Model
  
  Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein, Demetri Terzopoulos, Ade Famoti, Noboru Kuno, Ashley J. Llorens, Hoi Vo, Katsushi Ikeuchi, Fei-Fei Li, Jianfeng Gao, Naoki Wake, Qiuyuan Huang
  
  February 2024
  
  Publication Publication
- Agent AI: Surveying the Horizons of Multimodal Interaction
  
  Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong , Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, Jianfeng Gao
  
  arXiv: 2401.03568 | January 2024
  
  Publication
- Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models
  
  Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron L Halfaker, Liqun Shao, Daniel McDuff, Ahmed Awadallah, Dragomir Radev, Jianfeng Gao
  
  Proceedings of ACL 2023 | May 2023
  
  Publication Publication Project
- ArK: Augmented Reality with Knowledge Emergent Infrastructure
  
  Qiuyuan Huang, James Park, Abhinav Gupta, Pan Lu, Paul Bennett, Ran Gong, Subhojit Som, Baolin Peng, Owais Khan Mohammed, Chris Pal, Yejin Choi, Jianfeng Gao
  
  Selected as the project in HackBox 2023; Shipped to Office Teams | April 2023
  
  Publication Project
- NICE: Neural Image Commenting with Empathy
  
  Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Xiang Gao, Hamid Palangi, Jianfeng Wang, Kenneth Forbus, Jianfeng Gao
  
  EMNLP 2021 | October 2021
  
  Publication
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang
  
  CVPR 2019 | June 2019
  
  Best Student Paper Award in CVPR 2019.
  
  Publication Video Project Project
- Turbo Learning for CaptionBot and DrawingBot
  
  Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang
  
  NeurIPS 2018 | December 2018
  
  PDF Publication Project
- LLM Agent – Retrieve What You Need: A Mutual Learning Framework for Open-domain Question Answering
  
  Dingmin Wang, Qiuyuan Huang, Matthew Jackson, Jianfeng Gao
  
  Proceedings of Transactions of the Association for Computational Linguistics (TACL) | April 2024, pp. 247-263
  
  Publication Project
- An Interactive Agent Foundation Model
  
  Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein, Demetri Terzopoulos, Ade Famoti, Noboru Kuno, Ashley J. Llorens, Hoi Vo, Katsushi Ikeuchi, Fei-Fei Li, Jianfeng Gao, Naoki Wake, Qiuyuan Huang
  
  February 2024
  
  Publication Publication
- MindAgent: Emergent Gaming Interaction
  
  Steven Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-chun Zhu, Demetri Terzopoulos, Feifei Li, Jianfeng Gao
  
  Processings of NAACL 2024 | January 2024
  
  Publication Project
- Agent AI: Surveying the Horizons of Multimodal Interaction
  
  Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong , Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, Jianfeng Gao
  
  arXiv: 2401.03568 | January 2024
  
  Publication
- Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models
  
  Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron L Halfaker, Liqun Shao, Daniel McDuff, Ahmed Awadallah, Dragomir Radev, Jianfeng Gao
  
  Proceedings of ACL 2023 | May 2023
  
  Publication Publication Project
- ArK: Augmented Reality with Knowledge Emergent Infrastructure
  
  Qiuyuan Huang, James Park, Abhinav Gupta, Pan Lu, Paul Bennett, Ran Gong, Subhojit Som, Baolin Peng, Owais Khan Mohammed, Chris Pal, Yejin Choi, Jianfeng Gao
  
  Selected as the project in HackBox 2023; Shipped to Office Teams | April 2023
  
  Publication Project
- VLC: Training Vision-Language Transformers from Captions
  
  Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  Proceedings of Transactions on Machine Learning Research | May 2022
  
  Publication Project
- KAT: A Knowledge Augmented Transformer for Vision-and-Language
  
  Liangke Gui, Borui Wang, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  NAACL 2022. Long paper, Oral. | January 2022
  
  No.1 of OK-VQA Leaderboard (2021).
  
  Publication
- NICE: Neural Image Commenting with Empathy
  
  Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Xiang Gao, Hamid Palangi, Jianfeng Wang, Kenneth Forbus, Jianfeng Gao
  
  EMNLP 2021 | October 2021
  
  Publication
- Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
  
  Kezhen Chen, Qiuyuan Huang, Hamid Palangi, Paul Smolensky, Kenneth D. Forbus, Jianfeng Gao
  
  ICML 2020 | July 2020
  
  Publication Project
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang
  
  CVPR 2019 | June 2019
  
  Best Student Paper Award in CVPR 2019.
  
  Publication Video Project Project
- Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
  
  Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He
  
  AAAI 2019 | January 2019
  
  Publication Project Project
- Turbo Learning for CaptionBot and DrawingBot
  
  Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang
  
  NeurIPS 2018 | December 2018
  
  PDF Publication Project
- An Interactive Agent Foundation Model
  
  Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein, Demetri Terzopoulos, Ade Famoti, Noboru Kuno, Ashley J. Llorens, Hoi Vo, Katsushi Ikeuchi, Fei-Fei Li, Jianfeng Gao, Naoki Wake, Qiuyuan Huang
  
  February 2024
  
  Publication Publication
- Agent AI: Surveying the Horizons of Multimodal Interaction
  
  Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong , Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, Jianfeng Gao
  
  arXiv: 2401.03568 | January 2024
  
  Publication
- Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models
  
  Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron L Halfaker, Liqun Shao, Daniel McDuff, Ahmed Awadallah, Dragomir Radev, Jianfeng Gao
  
  Proceedings of ACL 2023 | May 2023
  
  Publication Publication Project
- ArK: Augmented Reality with Knowledge Emergent Infrastructure
  
  Qiuyuan Huang, James Park, Abhinav Gupta, Pan Lu, Paul Bennett, Ran Gong, Subhojit Som, Baolin Peng, Owais Khan Mohammed, Chris Pal, Yejin Choi, Jianfeng Gao
  
  Selected as the project in HackBox 2023; Shipped to Office Teams | April 2023
  
  Publication Project
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang
  
  CVPR 2019 | June 2019
  
  Best Student Paper Award in CVPR 2019.
  
  Publication Video Project Project

- Agent AI Towards a Holistic Intelligence
  
  Qiuyuan Huang, Naoki Wake, Bidipta Sarkar, Zane Durante, Ran Gong, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Noboru Kuno, Ade Famoti, Ashley J. Llorens, John Langford, Hoi Vo, Fei-Fei Li, Katsushi Ikeuchi, Jianfeng Gao
  
  arXiv: 2403.00833 | February 2024
  
  Publication Publication
- An Interactive Agent Foundation Model
  
  Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein, Demetri Terzopoulos, Ade Famoti, Noboru Kuno, Ashley J. Llorens, Hoi Vo, Katsushi Ikeuchi, Fei-Fei Li, Jianfeng Gao, Naoki Wake, Qiuyuan Huang
  
  February 2024
  
  Publication Publication
- MindAgent: Emergent Gaming Interaction
  
  Steven Gong, Qiuyuan Huang, Xiaojian Ma, Hoi Vo, Zane Durante, Yusuke Noda, Zilong Zheng, Song-chun Zhu, Demetri Terzopoulos, Feifei Li, Jianfeng Gao
  
  Processings of NAACL 2024 | January 2024
  
  Publication Project
- Agent AI: Surveying the Horizons of Multimodal Interaction
  
  Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong , Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, Jianfeng Gao
  
  arXiv: 2401.03568 | January 2024
  
  Publication
- Multimodal Agent — Localized Symbolic Knowledge Distillation for Visual Commonsense Models
  
  Jae Sung Park, Jack Hessel, Khyathi Chandu, Paul Pu Liang, Ximing Lu, Peter West, Qiuyuan Huang, Jianfeng Gao, Ali Farhadi, Yejin Choi
  
  Proceedings of NeurIPS 2023 | October 2023
  
  Publication
- Logical Transformers: Infusing Logical Structures into Pre-Trained Language Models
  
  Borui Wang, Qiuyuan Huang, Budhaditya Deb, Aaron L Halfaker, Liqun Shao, Daniel McDuff, Ahmed Awadallah, Dragomir Radev, Jianfeng Gao
  
  Proceedings of ACL 2023 | May 2023
  
  Publication Publication Project
- ArK: Augmented Reality with Knowledge Emergent Infrastructure
  
  Qiuyuan Huang, James Park, Abhinav Gupta, Pan Lu, Paul Bennett, Ran Gong, Subhojit Som, Baolin Peng, Owais Khan Mohammed, Chris Pal, Yejin Choi, Jianfeng Gao
  
  Selected as the project in HackBox 2023; Shipped to Office Teams | April 2023
  
  Publication Project
- KAT: A Knowledge Augmented Transformer for Vision-and-Language
  
  Liangke Gui, Borui Wang, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  NAACL 2022. Long paper, Oral. | January 2022
  
  No.1 of OK-VQA Leaderboard (2021).
  
  Publication
- NICE: Neural Image Commenting with Empathy
  
  Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Xiang Gao, Hamid Palangi, Jianfeng Wang, Kenneth Forbus, Jianfeng Gao
  
  EMNLP 2021 | October 2021
  
  Publication
- Mapping Natural-language Problems to Formal-language Solutions Using Structured Neural Representations
  
  Kezhen Chen, Qiuyuan Huang, Hamid Palangi, Paul Smolensky, Kenneth D. Forbus, Jianfeng Gao
  
  ICML 2020 | July 2020
  
  Publication Project
- Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Weng, William Yang Wang, Lei Zhang
  
  CVPR 2019 | June 2019
  
  Best Student Paper Award in CVPR 2019.
  
  Publication Video Project Project
- Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation
  
  Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Oliver Wu, Jianfeng Wang, Xiaodong He
  
  AAAI 2019 | January 2019
  
  Publication Project Project
- Turbo Learning for CaptionBot and DrawingBot
  
  Qiuyuan Huang, Pengchuan Zhang, Oliver Wu, Lei Zhang
  
  NeurIPS 2018 | December 2018
  
  PDF Publication Project
- LLM Agent – Retrieve What You Need: A Mutual Learning Framework for Open-domain Question Answering
  
  Dingmin Wang, Qiuyuan Huang, Matthew Jackson, Jianfeng Gao
  
  Proceedings of Transactions of the Association for Computational Linguistics (TACL) | April 2024, pp. 247-263
  
  Publication Project
- VLC: Training Vision-Language Transformers from Captions
  
  Liangke Gui, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, Jianfeng Gao
  
  Proceedings of Transactions on Machine Learning Research | May 2022
  
  Publication Project
- Vision-Language Navigation Policy Learning and Adaptation
  
  Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-fang Wang, William Wang, Lei Zhang
  
  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 | February 2020
  
  Publication Publication Project