Shuohang Wang

Principal Researcher

Publications

View by:

- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- Sparse Modular Activation for Efficient Sequence Modeling
  
  Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai
  
  NeurIPS 2023 | July 2023
  
  Publication
- In-Context Demonstration Selection with Cross Entropy Difference
  
  Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Chenguang Zhu
  
  May 2023
  
  Arxiv
  
  Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- Generate rather than Retrieve: Large Language Models are Strong Context Generators
  
  Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
  
  ICLR 2023 | May 2023
  
  Publication
- InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
  
  Yichong Xu, Ruochen Xu, Dan Iter, Yang Liu, Shuohang Wang, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
  
  Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
  
  arXiv 2303.16634 | March 2023
  
  Publication
- AdaPrompt: Adaptive Model Training for Prompt-based NLP
  
  Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang
  
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022. | December 2022
  
  Publication
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication Project
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  
  Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Leveraging Knowledge in Multilingual Commonsense Reasoning
  
  Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Project
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Project
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication Project
- NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset
  
  Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Accelerating Real-Time Question Answering via Question Generation
  
  Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu, Chenguang Zhu
  
  arXiv: Computation and Language | September 2021
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  Project
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Contrastive Distillation on Intermediate Representations for Language Model Compression
  
  Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Cross-Thought for Sentence Encoder Pre-training
  
  Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Multi-Fact Correction in Abstractive Text Summarization
  
  Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Hierarchical Graph Network for Multi-hop Question Answering
  
  Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, JJ (Jingjing) Liu
  
  2020 Empirical Methods in Natural Language Processing | April 2020
  
  DOI PDF Github

- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- Sparse Modular Activation for Efficient Sequence Modeling
  
  Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai
  
  NeurIPS 2023 | July 2023
  
  Publication
- In-Context Demonstration Selection with Cross Entropy Difference
  
  Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Chenguang Zhu
  
  May 2023
  
  Arxiv
  
  Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
  
  Yichong Xu, Ruochen Xu, Dan Iter, Yang Liu, Shuohang Wang, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Project
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Project
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication Project
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  Project
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Cross-Thought for Sentence Encoder Pre-training
  
  Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Multi-Fact Correction in Abstractive Text Summarization
  
  Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Contrastive Distillation on Intermediate Representations for Language Model Compression
  
  Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Hierarchical Graph Network for Multi-hop Question Answering
  
  Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, JJ (Jingjing) Liu
  
  2020 Empirical Methods in Natural Language Processing | April 2020
  
  DOI PDF Github
- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- Generate rather than Retrieve: Large Language Models are Strong Context Generators
  
  Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
  
  ICLR 2023 | May 2023
  
  Publication
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
  
  Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
  
  arXiv 2303.16634 | March 2023
  
  Publication
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- AdaPrompt: Adaptive Model Training for Prompt-based NLP
  
  Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang
  
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022. | December 2022
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication Project
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Leveraging Knowledge in Multilingual Commonsense Reasoning
  
  Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  
  Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Project
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Project
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset
  
  Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Accelerating Real-Time Question Answering via Question Generation
  
  Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu, Chenguang Zhu
  
  arXiv: Computation and Language | September 2021
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  Project
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication Project
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Project
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Project
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication Project
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication

- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- Sparse Modular Activation for Efficient Sequence Modeling
  
  Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai
  
  NeurIPS 2023 | July 2023
  
  Publication
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- Generate rather than Retrieve: Large Language Models are Strong Context Generators
  
  Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
  
  ICLR 2023 | May 2023
  
  Publication
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- AdaPrompt: Adaptive Model Training for Prompt-based NLP
  
  Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang
  
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022. | December 2022
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication Project
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Leveraging Knowledge in Multilingual Commonsense Reasoning
  
  Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  
  Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Project
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication Project
- NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset
  
  Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  Project
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Cross-Thought for Sentence Encoder Pre-training
  
  Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Multi-Fact Correction in Abstractive Text Summarization
  
  Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Contrastive Distillation on Intermediate Representations for Language Model Compression
  
  Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Hierarchical Graph Network for Multi-hop Question Answering
  
  Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, JJ (Jingjing) Liu
  
  2020 Empirical Methods in Natural Language Processing | April 2020
  
  DOI PDF Github
- In-Context Demonstration Selection with Cross Entropy Difference
  
  Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Chenguang Zhu
  
  May 2023
  
  Arxiv
  
  Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
  
  Yichong Xu, Ruochen Xu, Dan Iter, Yang Liu, Shuohang Wang, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Project
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
  
  Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
  
  arXiv 2303.16634 | March 2023
  
  Publication
- Accelerating Real-Time Question Answering via Question Generation
  
  Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu, Chenguang Zhu
  
  arXiv: Computation and Language | September 2021
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication