Linjie Li

Chercheur

Papiers de Collaboration

Afficher par:

- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
  
  Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
  
  Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- What can Foundation Models’ Embeddings do?
  
  Linjie Li, Jianwei Yang, Lijuan Wang
  
  October 2024
- Bring Metric Functions into Diffusion Models
  
  Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo
  
  IJCAI 2024 | August 2024
  
  DOI Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- VideoGUI: A Benchmark for GUI Automation from Instructional Videos
  
  Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen Wu, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou
  
  NeurIPS 2024 | June 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
  
  Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | November 2023
  
  DOI Publication Publication
- The Generative AI Paradox: “What It Can Create, It May Not Understand”
  
  Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian R. Fisher, Abhilasha Ravichander, Khyathi Raghavi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
  
  ICLR 2024 | October 2023
  
  DOI Publication
- Multimodal Foundation Models: From Specialists to General-Purpose Assistants
  
  Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao
  
  September 2023
  
  118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/
  
  Publication Projet
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
  
  Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang
  
  ICML 2024 | August 2023
  
  DOI Publication PDF Publication
- DisCo: Disentangled Control for Realistic Human Dance Generation
  
  Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
  
  Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Y. Yacoob, Lijuan Wang
  
  ICLR 2024 | June 2023
  
  Publication
- MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
  
  Jielin Qiu, Jiacheng Zhu, William Jongwon Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication Projet
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication Projet
- Cross-modal Representation Learning for Zero-shot Action Recognition
  
  Chung-Ching Lin, Kevin Lin, Linjie Li, Lijuan Wang, Zicheng Liu
  
  2022 Computer Vision and Pattern Recognition | May 2022
  
  Publication Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Projet
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Projet
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication Projet
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication Projet
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
  
  Linjie Li, Jie Lei, Zhe Gan, Jingjing Liu
  
  ICCV 2021 | October 2021
  
  Publication Projet
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github Projet
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication Projet
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF Projet
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication Projet
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF Projet
- Graph Optimal Transport for Cross-Domain Alignment.
  
  Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, JJ (Jingjing) Liu
  
  International Conference on Machine Learning (ICML 2020) | July 2020
  
  PDF

- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
  
  Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
  
  Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- What can Foundation Models’ Embeddings do?
  
  Linjie Li, Jianwei Yang, Lijuan Wang
  
  October 2024
- Bring Metric Functions into Diffusion Models
  
  Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo
  
  IJCAI 2024 | August 2024
  
  DOI Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- VideoGUI: A Benchmark for GUI Automation from Instructional Videos
  
  Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen Wu, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou
  
  NeurIPS 2024 | June 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- The Generative AI Paradox: “What It Can Create, It May Not Understand”
  
  Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian R. Fisher, Abhilasha Ravichander, Khyathi Raghavi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
  
  ICLR 2024 | October 2023
  
  DOI Publication
- Multimodal Foundation Models: From Specialists to General-Purpose Assistants
  
  Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao
  
  September 2023
  
  118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/
  
  Publication Projet
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
  
  Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang
  
  ICML 2024 | August 2023
  
  DOI Publication PDF Publication
- Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
  
  Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Y. Yacoob, Lijuan Wang
  
  ICLR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication Projet
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication Projet
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Projet
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Projet
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication Projet
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication Projet
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github Projet
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication Projet
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF Projet
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication Projet
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF Projet
- Graph Optimal Transport for Cross-Domain Alignment.
  
  Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, JJ (Jingjing) Liu
  
  International Conference on Machine Learning (ICML 2020) | July 2020
  
  PDF
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
  
  Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | November 2023
  
  DOI Publication Publication
- DisCo: Disentangled Control for Realistic Human Dance Generation
  
  Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
  
  Jielin Qiu, Jiacheng Zhu, William Jongwon Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication Projet
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication Projet
- Cross-modal Representation Learning for Zero-shot Action Recognition
  
  Chung-Ching Lin, Kevin Lin, Linjie Li, Lijuan Wang, Zicheng Liu
  
  2022 Computer Vision and Pattern Recognition | May 2022
  
  Publication Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Projet
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Projet
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication Projet
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication Projet
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
  
  Linjie Li, Jie Lei, Zhe Gan, Jingjing Liu
  
  ICCV 2021 | October 2021
  
  Publication Projet
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github Projet
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication Projet
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF Projet
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication Projet
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF Projet
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication Projet
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Projet
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Projet
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication Projet
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication Projet
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github Projet
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication Projet

- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback
  
  Yafu Li, Xuyang Hu, Xiaoye Qu, Linjie Li, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
  
  Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng
  
  ICML 2025 | January 2025
  
  DOI Publication Publication
- GenXD: Generating Any 3D and 4D Scenes
  
  Yuyang Zhao, Chung-Ching Lin, K. Lin, Zhiwen Yan, Linjie Li, Zhengyuan Yang, Jianfeng Wang, Gim Hee Lee, Lijuan Wang
  
  ICLR 2025 | November 2024
  
  DOI Publication Publication
- SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation
  
  Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Linjie Li, K. Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models
  
  Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing
  
  Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, K. Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang, Linjie Li
  
  ICLR 2025 | October 2024
  
  DOI Publication Publication
- Bring Metric Functions into Diffusion Models
  
  Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo
  
  IJCAI 2024 | August 2024
  
  DOI Publication
- Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness
  
  Khyathi Raghavi Chandu, Linjie Li, Anas Awadalla, Ximing Lu, J. Park, Jack Hessel, Lijuan Wang, Yejin Choi
  
  ICLR 2025 | July 2024
  
  DOI Publication Publication
- VideoGUI: A Benchmark for GUI Automation from Instructional Videos
  
  Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen Wu, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou
  
  NeurIPS 2024 | June 2024
  
  DOI Publication Publication
- MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos
  
  Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, K. Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang
  
  ICLR 2025 | June 2024
  
  DOI Publication Publication
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning
  
  Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | November 2023
  
  DOI Publication Publication
- The Generative AI Paradox: “What It Can Create, It May Not Understand”
  
  Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian R. Fisher, Abhilasha Ravichander, Khyathi Raghavi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi
  
  ICLR 2024 | October 2023
  
  DOI Publication
- MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities
  
  Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang
  
  ICML 2024 | August 2023
  
  DOI Publication PDF Publication
- DisCo: Disentangled Control for Realistic Human Dance Generation
  
  Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
  
  Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Y. Yacoob, Lijuan Wang
  
  ICLR 2024 | June 2023
  
  Publication
- MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
  
  Jielin Qiu, Jiacheng Zhu, William Jongwon Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang
  
  CVPR 2024 | June 2023
  
  Publication
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
  
  Zi-Yi Dou, Aishwarya Kamath, Zhe Gan, Pengchuan Zhang, Jianfeng Wang, Linjie Li, Zicheng Liu, Ce Liu, Yann LeCun, Nanyun Peng, Jianfeng Gao, Lijuan Wang
  
  NeurIPS 2022 | November 2022
  
  Publication Projet
- SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
  
  Kevin Lin, Linjie Li, Chung-Ching Lin, Faisal Ahmed, Zhe Gan, Zicheng Liu, Yumao Lu, Lijuan Wang
  
  CVPR 2022 | June 2022
  
  Publication Projet
- Cross-modal Representation Learning for Zero-shot Action Recognition
  
  Chung-Ching Lin, Kevin Lin, Linjie Li, Lijuan Wang, Zicheng Liu
  
  2022 Computer Vision and Pattern Recognition | May 2022
  
  Publication Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication Projet
- VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation
  
  Linjie Li, Jie Lei, Zhe Gan, Licheng Yu, Yen-Chun Chen, Rohit Pillai, Yu Cheng, Luowei Zhou, Xin Eric Wang, William Yang Wang, Tamara Lee Berg, Mohit Bansal, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  NeurIPS 2021 | December 2021
  
  Publication Projet
- Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models
  
  Linjie Li, Jie Lei, Zhe Gan, Jingjing Liu
  
  ICCV 2021 | October 2021
  
  Publication Projet
- Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
  
  Jie Lei, Linjie Li, Luowei Zhou, Zhe Gan, Tamara L. Berg, Mohit Bansal, Jingjing Liu
  
  CVPR 2021 | June 2021
  
  CVPR 2021 Best Paper
  
  Publication Github Projet
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- Meta Module Network for Compositional Visual Reasoning
  
  Wenhu Chen, Zhe Gan, Linjie Li, Yu Cheng, William Yang Wang, JJ (Jingjing) Liu
  
  Winter Conference on Applications of Computer Vision (WACV 2021) | March 2021
  
  Publication
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, JJ (Jingjing) Liu
  
  NeurIPS 2020 | December 2020
  
  PDF Projet
- HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
  
  Linjie Li, Yen-Chun Chen, Yu Cheng, Zhe Gan, Licheng Yu, Jingjing Liu
  
  Conf. on Empirical Methods in Natural Language Processing (EMNLP 2020) | November 2020
  
  Publication Projet
- UNITER: UNiversal Image-TExt Representation Learning
  
  Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, JJ (Jingjing) Liu
  
  ECCV 2020 | August 2020
  
  PDF Projet
- Graph Optimal Transport for Cross-Domain Alignment.
  
  Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, JJ (Jingjing) Liu
  
  International Conference on Machine Learning (ICML 2020) | July 2020
  
  PDF
- What can Foundation Models’ Embeddings do?
  
  Linjie Li, Jianwei Yang, Lijuan Wang
  
  October 2024
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint Projet
- Multimodal Foundation Models: From Specialists to General-Purpose Assistants
  
  Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao
  
  September 2023
  
  118-page book/survey on the literature review, evolution, trends and our position on multimodal foundation models. CVPR 2023 Tutorial: https://vlp-tutorial.github.io/2023/
  
  Publication Projet
- VIOLET: End-to-End Video-Language Transformers with Masked Visual-token Modeling
  
  Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, Zicheng Liu
  
  arXiv preprint arXiv:2111.12681 | November 2021
  
  Publication Projet
- A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
  
  Linjie Li, Zhe Gan, Jingjing Liu
  
  December 2020
  
  Publication Projet
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication