Bei Liu

Senior Researcher

Connect on LinkedIn

Publications

View by:

- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
  
  Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo
  
  November 2024
  
  DOI Publication Publication
- MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
  
  Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo
  
  arXiv | December 2022
  
  Publication Project
- Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
  
  Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu
  
  NeurIPS 2022 | November 2022
  
  Publication
- Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
  
  Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo
  
  CVPR 2022 | June 2022
  
  Publication Project
- Debiasing Event Understanding for Visual Commonsense Tasks
  
  Minji Seo, YeonJoon Jung, Seungtaek Choi, seung-won hwang, Bei Liu
  
  May 2022
- Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
  
  Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo
  
  NeurIPS 2021 | December 2021
  
  Publication
- Searching the Search Space of Vision Transformer
  
  Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling
  
  NeurIPS 2021 | December 2021
  
  Publication
- Learning Fine-Grained Motion Embedding for Landscape Animation
  
  Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
  
  ACM Multimedia 2021 | October 2021
  
  Oral
  
  Publication Project
- A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
  
  Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- Unifying multimodal transformer for bi-directional image and text generation
  
  Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
  
  Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
  
  ICMR 2021 | August 2021
  
  DOI Project
- Reference-Based Defect Detection Network
  
  Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao
  
  IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647
  
  Publication
- Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
  
  Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  CVPR 2021 | June 2021
  
  Oral
  
  Publication Project
- Aesthetic-Aware Image Style Transfer
  
  Bei Liu, Jianlong Fu
  
  ACM Multimedia | October 2020
  
  Publication Project
- Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
  
  Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  MSR-TR-2020-51 | April 2020
  
  Published by Microsoft
  
  Publication Project
- Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
  
  Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
  
  ACM Multimedia | October 2019
  
  Publication Project
- SMP Challenge: An Overview of Social Media Prediction Challenge 2019
  
  Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo
  
  ACM Multimedia | October 2019
  
  Publication
- WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
  
  Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang
  
  ICCV | October 2019
  
  Publication Project
- Emotion Reinforced Visual Storytelling
  
  Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
  
  ICMR | June 2019
  
  Publication Project
- Learning Rich Image Region Representation for Visual Question Answering
  
  Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
  
  MSR-TR-2020-1 | June 2019
  
  Published by Microsoft
  
  CVPR 2019, VQA Workshop
  
  Publication Project
- Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
  
  Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
  
  2018 ACM Multimedia | October 2018
  
  Best Paper Award
  
  DOI Publication PDF Project

- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
  
  Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo
  
  November 2024
  
  DOI Publication Publication
- Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
  
  Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu
  
  NeurIPS 2022 | November 2022
  
  Publication
- Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
  
  Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  CVPR 2021 | June 2021
  
  Oral
  
  Publication Project
- Aesthetic-Aware Image Style Transfer
  
  Bei Liu, Jianlong Fu
  
  ACM Multimedia | October 2020
  
  Publication Project
- Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
  
  Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  MSR-TR-2020-51 | April 2020
  
  Published by Microsoft
  
  Publication Project
- Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
  
  Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
  
  ACM Multimedia | October 2019
  
  Publication Project
- Learning Rich Image Region Representation for Visual Question Answering
  
  Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
  
  MSR-TR-2020-1 | June 2019
  
  Published by Microsoft
  
  CVPR 2019, VQA Workshop
  
  Publication Project
- Emotion Reinforced Visual Storytelling
  
  Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
  
  ICMR | June 2019
  
  Publication Project
- Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
  
  Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
  
  2018 ACM Multimedia | October 2018
  
  Best Paper Award
  
  DOI Publication PDF Project
- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
  
  Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo
  
  November 2024
  
  DOI Publication Publication
- MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
  
  Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo
  
  arXiv | December 2022
  
  Publication Project
- Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
  
  Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo
  
  CVPR 2022 | June 2022
  
  Publication Project
- Searching the Search Space of Vision Transformer
  
  Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling
  
  NeurIPS 2021 | December 2021
  
  Publication
- Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
  
  Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo
  
  NeurIPS 2021 | December 2021
  
  Publication
- Learning Fine-Grained Motion Embedding for Landscape Animation
  
  Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
  
  ACM Multimedia 2021 | October 2021
  
  Oral
  
  Publication Project
- Unifying multimodal transformer for bi-directional image and text generation
  
  Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
  
  Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
  
  Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
  
  ICMR 2021 | August 2021
  
  DOI Project
- Reference-Based Defect Detection Network
  
  Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao
  
  IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647
  
  Publication
- Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
  
  Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  CVPR 2021 | June 2021
  
  Oral
  
  Publication Project
- Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
  
  Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  MSR-TR-2020-51 | April 2020
  
  Published by Microsoft
  
  Publication Project
- WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
  
  Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang
  
  ICCV | October 2019
  
  Publication Project
- Learning Rich Image Region Representation for Visual Question Answering
  
  Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
  
  MSR-TR-2020-1 | June 2019
  
  Published by Microsoft
  
  CVPR 2019, VQA Workshop
  
  Publication Project
- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
  
  Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo
  
  November 2024
  
  DOI Publication Publication
- Debiasing Event Understanding for Visual Commonsense Tasks
  
  Minji Seo, YeonJoon Jung, Seungtaek Choi, seung-won hwang, Bei Liu
  
  May 2022
- Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
  
  Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
  
  2018 ACM Multimedia | October 2018
  
  Best Paper Award
  
  DOI Publication PDF Project
- Learning Fine-Grained Motion Embedding for Landscape Animation
  
  Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
  
  ACM Multimedia 2021 | October 2021
  
  Oral
  
  Publication Project
- Unifying multimodal transformer for bi-directional image and text generation
  
  Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
  
  Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
  
  Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
  
  ICMR 2021 | August 2021
  
  DOI Project
- SMP Challenge: An Overview of Social Media Prediction Challenge 2019
  
  Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo
  
  ACM Multimedia | October 2019
  
  Publication
- Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
  
  Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
  
  ACM Multimedia | October 2019
  
  Publication Project
- Emotion Reinforced Visual Storytelling
  
  Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
  
  ICMR | June 2019
  
  Publication Project
- Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
  
  Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
  
  2018 ACM Multimedia | October 2018
  
  Best Paper Award
  
  DOI Publication PDF Project

- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
  
  Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, Baining Guo
  
  November 2024
  
  DOI Publication Publication
- MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generation
  
  Ludan Ruan, Yiyang Ma, Huan Yang, Huiguo He, Bei Liu, Jianlong Fu, Nicholas Jing Yuan, Qin Jin, Baining Guo
  
  arXiv | December 2022
  
  Publication Project
- Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning
  
  Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu
  
  NeurIPS 2022 | November 2022
  
  Publication
- Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions
  
  Hongwei Xue, Tiankai Hang, Yanhong Zeng, Yuchong Sun, Bei Liu, Huan Yang, Jianlong Fu, Baining Guo
  
  CVPR 2022 | June 2022
  
  Publication Project
- Searching the Search Space of Vision Transformer
  
  Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, Haibin Ling
  
  NeurIPS 2021 | December 2021
  
  Publication
- Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training
  
  Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo
  
  NeurIPS 2021 | December 2021
  
  Publication
- Learning Fine-Grained Motion Embedding for Landscape Animation
  
  Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo
  
  ACM Multimedia 2021 | October 2021
  
  Oral
  
  Publication Project
- Unifying multimodal transformer for bi-directional image and text generation
  
  Yupan Huang, Hongwei Xue, Bei Liu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
  
  Yupan Huang, Bei Liu, Jianlong Fu, Yutong Lu
  
  ACM Multimedia 2021 | October 2021
  
  Publication Project
- MMPT’21: International Joint Workshop on Multi-Modal Pre-Training for Multimedia Understanding
  
  Bei Liu, Jianlong Fu, Shizhe Chen, Qin Jin, Alexander Hauptmann, Yong Rui
  
  ICMR 2021 | August 2021
  
  DOI Project
- Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
  
  Zhicheng Huang*, Zhaoyang Zeng*, Yupan Huang*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  CVPR 2021 | June 2021
  
  Oral
  
  Publication Project
- Aesthetic-Aware Image Style Transfer
  
  Bei Liu, Jianlong Fu
  
  ACM Multimedia | October 2020
  
  Publication Project
- SMP Challenge: An Overview of Social Media Prediction Challenge 2019
  
  Bo Wu, Wen-Huang Cheng, Peiye Liu, Bei Liu, Zhaoyang Zeng, Jiebo Luo
  
  ACM Multimedia | October 2019
  
  Publication
- Neural Storyboard Artist: Visualizing Stories with Coherent Image Sequences
  
  Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, Jin Zhou
  
  ACM Multimedia | October 2019
  
  Publication Project
- WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
  
  Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao, Lei Zhang
  
  ICCV | October 2019
  
  Publication Project
- Emotion Reinforced Visual Storytelling
  
  Nanxing Li, Bei Liu, Zhizhong Han, Yu-Shen Liu, Jianlong Fu
  
  ICMR | June 2019
  
  Publication Project
- Beyond Narrative Description: Generating Poetry from Images by Multi-Adversarial Training
  
  Bei Liu, Jianlong Fu, Makoto P. Kato, Masatoshi Yoshikawa
  
  2018 ACM Multimedia | October 2018
  
  Best Paper Award
  
  DOI Publication PDF Project
- Debiasing Event Understanding for Visual Commonsense Tasks
  
  Minji Seo, YeonJoon Jung, Seungtaek Choi, seung-won hwang, Bei Liu
  
  May 2022
- Reference-Based Defect Detection Network
  
  Zhaoyang Zeng, Bei Liu, Jianlong Fu, Hongyang Chao
  
  IEEE Transactions on Image Processing | July 2021, Vol 30: pp. 6637-6647
  
  Publication
- Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers
  
  Zhicheng Huang*, Zhaoyang Zeng*, Bei Liu, Dongmei Fu, Jianlong Fu
  
  MSR-TR-2020-51 | April 2020
  
  Published by Microsoft
  
  Publication Project
- Learning Rich Image Region Representation for Visual Question Answering
  
  Bei Liu, Zhicheng Huang, Zhaoyang Zeng, Zheyu Chen, Jianlong Fu
  
  MSR-TR-2020-1 | June 2019
  
  Published by Microsoft
  
  CVPR 2019, VQA Workshop
  
  Publication Project