View from MSRA Shanghai Lab

System and Engineering Group

MSR Asia-Shanghai

论文与出版物

查看方式：

- MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention
  
  Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICML 2025 | July 2025
  
  Preprint Github 项目
- SCBench: A KV Cache-Centric Analysis of Long-Context Methods
  
  Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICLR 2025 | December 2024
  
  Preprint Github 项目
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
  
  Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  NeurIPS 2024 | December 2024
  
  spotlight
  
  Preprint Github 项目
- LORASC: Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning
  
  Yifan Yang, Yifei Shen, Yuqing Yang, Lili Qiu, Fangyun Wei
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  Publication
- LLM2CLIP: Powerful Language Model Unlocks Richer Visual Representation
  
  Weiquan Huang, Aoqi Wu, Yifan Yang, Xufang Luo, Yuqing Yang, Liang Hu, Qi Dai, Xiyang Dai, Dongdong Chen, Chong Luo, Lili Qiu
  
  November 2024
  
  Publication Github 项目
- Reviving Cloud Gaming Sessions
  
  Yifan Yang, Lili Qiu, Yuqing Yang
  
  CoNEXT 2024 – International Conference on emerging Networking EXperiments and Technologies | November 2024
  
  Preprint
- RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
  
  Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, Lili Qiu
  
  ENLSP NeurIPS Workshop 2024 | September 2024
  
  Best Paper Award
  
  Preprint
- Mitigate Position Bias in Large Language Models via Scaling a Single Dimension
  
  Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, Lili Qiu
  
  June 2024
  
  Preprint
- Real-Time Neural Video Recovery and Enhancement on Mobile Devices
  
  Yifan Yang, Yuqing Yang, Lili Qiu
  
  CoNEXT 2024 – International Conference on emerging Networking EXperiments and Technologies | April 2024
  
  Preprint
- LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression
  
  Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024
  
  Preprint 项目
- LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
  
  Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023) | December 2023
  
  Publication 项目
- Large Multimodal Model for Real-World Radiology Report Generation
  
  Brian Nlong Zhao, Xinyang Jiang, Xufang Luo, Yifan Yang, Bo Li, Zilong Wang, Javier Alvarez-Valle, Matthew P. Lungren, Dongsheng Li, Lili Qiu
  
  December 2023
  
  Publication
- PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
  
  Ningxin Zheng, Huiqiang Jiang, Quanlu Zhang, Zhenhua Han, Lingxiao Ma, Yuqing Yang, Fan Yang, Chengruidong Zhang, Lili Qiu, Mao Yang, Lidong Zhou
  
  The 29th ACM Symposium on Operating Systems Principles | October 2023
  
  Publication
- SPFresh: Incremental In-Place Update for Billion-Scale Vector Search
  
  Yuming Xu, Hengyu Liang, Jin Li, Shuotao Xu, Qi Chen, Qianxi Zhang, Cheng Li , Ziyue Yang, Fan Yang, Yuqing Yang, Peng Cheng, Mao Yang
  
  SOSP’23 | October 2023
  
  Microsoft Research Blog
  
  https://www.rarnonalumber.com/en-us/research/blog/unified-database-laying-the-foundation-for-large-language-model-vertical-applications/
  
  Publication
- LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
  
  Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) | October 2023
  
  Preprint 项目
- ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
  
  Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
  
  Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- Attentive Mask CLIP
  
  Yifan Yang, Houwen Peng, Xinyang Jiang, Huiqiang Jiang, Yuqing Yang, Lili Qiu
  
  2023 International Conference on Computer Vision | October 2023
  
  Publication
- ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation
  
  Yasheng Sun, Yifan Yang, Houwen Peng, Yifei Shen, Yuqing Yang, Han Hu, Lili Qiu, Hideki Koike, Yuqing Yang
  
  NeurIPS 2023 | October 2023
  
  Publication
- Accurate and Structured Pruning for Efficient Automatic Speech Recognition
  
  Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
  
  INTERSPEECH 2023 | August 2023
  
  Publication
- Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
  
  Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
  
  KDD 2023 | August 2023
  
  Publication
- Optimizing Dynamic Neural Networks with Brainstorm
  
  Weihao Cui, Zhenhua Han, Lingji Ouyang, Yichuan Wang, Ningxin Zheng, Lingxiao Ma, Yuqing Yang, Fan Yang, Jilong Xue, Lili Qiu, Lidong Zhou, Quan Chen, Haisheng Tan, Minyi Guo
  
  OSDI’23 | July 2023
  
  Publication
- VBASE: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity
  
  Qianxi Zhang, Shuotao Xu, Qi Chen, Guoxin Sui, Jiadong Xie, Zhizhen Cai, Yaoqi Chen, Yinxuan He, Yuqing Yang, Fan Yang, Mao Yang, Lidong Zhou
  
  OSDI’23 | July 2023
  
  Publication
- Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
  
  Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
  
  Sixth Conference on Machine Learning and Systems (MLSys’23) | June 2023
  
  Publication 项目
- EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
  
  Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, Yixuan Yuan
  
  June 2023
  
  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  
  Publication
- SiloD: A Co-design of Caching and Scheduling for Deep Learning Clusters
  
  Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Mingxia Li, Fan Yang, Qianxi Zhang, Binyang Li, Yuqing Yang, Lili Qiu, Lintao Zhang, Lidong Zhou
  
  EuroSys 2023 | May 2023
  
  Publication
- Reinforcement Learning with Automated Auxiliary Loss Search
  
  Tairan He, Yuge Zhang, Kan Ren, Minghuan Liu, Che Wang, Weinan Zhang, Yuqing Yang, Dongsheng Li
  
  2022 Neural Information Processing Systems | October 2022
  
  Publication
- PilotFish: Harvesting Free Cycles of Cloud Gaming with Deep Learning Training
  
  Wei Zhang , Binghao Chen, Zhenhua Han, Quan Chen, Peng Cheng, Fan Yang, Ran Shu, Yuqing Yang, Minyi Guo
  
  2022 USENIX Annual Technical Conference | July 2022
  
  PDF
- SparTA: Deep-Learning Model Sparsity via Tensor-with-Sparsity-Attribute
  
  Ningxin Zheng, Bin Lin, Quanlu Zhang, Lingxiao Ma, Yuqing Yang, Fan Yang, Yang Wang, Mao Yang, Lidong Zhou
  
  OSDI 2022 | July 2022
  
  Publication 项目项目
- Privacy-preserving Online AutoML for Domain-Specific Face Detection
  
  Chenqian Yan, Yuge Zhang, Quanlu Zhang, Yaming Yang, Xinyang Jiang, Yuqing Yang, Baoyuan Wang
  
  CVPR 2022 | May 2022
  
  Publication
- nn-METER: Towards Accurate Latency Prediction of DNN Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  GetMobile: Mobile Computing and Communications | December 2021, 第 25 卷(4): pp. 19-23
  
  SigMobile Research Highlight
  
  Publication
- AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing
  
  Yuge Zhang, Chenqian Yan, Quanlu Zhang, Li Lyna Zhang, Yaming Yang, Xiaotian Gao, Yuqing Yang
  
  August 2021
  
  发布者： arXiv | August 2021
  
  Publication
- nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys’21) | June 2021
  
  Best Paper Award
  
  Publication
- HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees
  
  Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Fan Yang, Lidong Zhou, Mao Yang, Yuqi Wang, Yifan Xiong, Bin Wang
  
  2020 Operating Systems Design and Implementation | November 2020
  
  Publication 项目
- Retiarii: A Deep Learning Exploratory-Training Framework
  
  Quanlu Zhang, Zhenhua Han, Fan Yang, Yuge Zhang, Zhe Liu, Mao Yang, Lidong Zhou
  
  2020 Operating Systems Design and Implementation | November 2020
  
  Publication 视频
- Fast Hardware-Aware Neural Architecture Search
  
  Li Lyna Zhang, Yuqing Yang, Yuhang Jiang, Wenwu Zhu, Yunxin Liu
  
  CVPRW (Joint Workshop on Efficient Deep Learning in Computer Vision) | June 2020
  
  DOI PDF Publication

- MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention
  
  Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICML 2025 | July 2025
  
  Preprint Github 项目
- SCBench: A KV Cache-Centric Analysis of Long-Context Methods
  
  Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICLR 2025 | December 2024
  
  Preprint Github 项目
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
  
  Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  NeurIPS 2024 | December 2024
  
  spotlight
  
  Preprint Github 项目
- LORASC: Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning
  
  Yifan Yang, Yifei Shen, Yuqing Yang, Lili Qiu, Fangyun Wei
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  Publication
- RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
  
  Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, Lili Qiu
  
  ENLSP NeurIPS Workshop 2024 | September 2024
  
  Best Paper Award
  
  Preprint
- Mitigate Position Bias in Large Language Models via Scaling a Single Dimension
  
  Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, Lili Qiu
  
  June 2024
  
  Preprint
- LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression
  
  Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024
  
  Preprint 项目
- LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
  
  Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023) | December 2023
  
  Publication 项目
- Large Multimodal Model for Real-World Radiology Report Generation
  
  Brian Nlong Zhao, Xinyang Jiang, Xufang Luo, Yifan Yang, Bo Li, Zilong Wang, Javier Alvarez-Valle, Matthew P. Lungren, Dongsheng Li, Lili Qiu
  
  December 2023
  
  Publication
- PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
  
  Ningxin Zheng, Huiqiang Jiang, Quanlu Zhang, Zhenhua Han, Lingxiao Ma, Yuqing Yang, Fan Yang, Chengruidong Zhang, Lili Qiu, Mao Yang, Lidong Zhou
  
  The 29th ACM Symposium on Operating Systems Principles | October 2023
  
  Publication
- LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
  
  Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) | October 2023
  
  Preprint 项目
- ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
  
  Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
  
  Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- Attentive Mask CLIP
  
  Yifan Yang, Houwen Peng, Xinyang Jiang, Huiqiang Jiang, Yuqing Yang, Lili Qiu
  
  2023 International Conference on Computer Vision | October 2023
  
  Publication
- ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation
  
  Yasheng Sun, Yifan Yang, Houwen Peng, Yifei Shen, Yuqing Yang, Han Hu, Lili Qiu, Hideki Koike, Yuqing Yang
  
  NeurIPS 2023 | October 2023
  
  Publication
- Accurate and Structured Pruning for Efficient Automatic Speech Recognition
  
  Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
  
  INTERSPEECH 2023 | August 2023
  
  Publication
- Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
  
  Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
  
  KDD 2023 | August 2023
  
  Publication
- Reinforcement Learning with Automated Auxiliary Loss Search
  
  Tairan He, Yuge Zhang, Kan Ren, Minghuan Liu, Che Wang, Weinan Zhang, Yuqing Yang, Dongsheng Li
  
  2022 Neural Information Processing Systems | October 2022
  
  Publication
- AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing
  
  Yuge Zhang, Chenqian Yan, Quanlu Zhang, Li Lyna Zhang, Yaming Yang, Xiaotian Gao, Yuqing Yang
  
  August 2021
  
  发布者： arXiv | August 2021
  
  Publication
- nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys’21) | June 2021
  
  Best Paper Award
  
  Publication
- MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention
  
  Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICML 2025 | July 2025
  
  Preprint Github 项目
- SCBench: A KV Cache-Centric Analysis of Long-Context Methods
  
  Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICLR 2025 | December 2024
  
  Preprint Github 项目
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
  
  Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  NeurIPS 2024 | December 2024
  
  spotlight
  
  Preprint Github 项目
- RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
  
  Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, Lili Qiu
  
  ENLSP NeurIPS Workshop 2024 | September 2024
  
  Best Paper Award
  
  Preprint
- Mitigate Position Bias in Large Language Models via Scaling a Single Dimension
  
  Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, Lili Qiu
  
  June 2024
  
  Preprint
- LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression
  
  Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024
  
  Preprint 项目
- LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
  
  Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023) | December 2023
  
  Publication 项目
- LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
  
  Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) | October 2023
  
  Preprint 项目
- Accurate and Structured Pruning for Efficient Automatic Speech Recognition
  
  Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
  
  INTERSPEECH 2023 | August 2023
  
  Publication
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
  
  Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  NeurIPS 2024 | December 2024
  
  spotlight
  
  Preprint Github 项目
- Reviving Cloud Gaming Sessions
  
  Yifan Yang, Lili Qiu, Yuqing Yang
  
  CoNEXT 2024 – International Conference on emerging Networking EXperiments and Technologies | November 2024
  
  Preprint
- RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
  
  Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, Lili Qiu
  
  ENLSP NeurIPS Workshop 2024 | September 2024
  
  Best Paper Award
  
  Preprint
- Real-Time Neural Video Recovery and Enhancement on Mobile Devices
  
  Yifan Yang, Yuqing Yang, Lili Qiu
  
  CoNEXT 2024 – International Conference on emerging Networking EXperiments and Technologies | April 2024
  
  Preprint
- SPFresh: Incremental In-Place Update for Billion-Scale Vector Search
  
  Yuming Xu, Hengyu Liang, Jin Li, Shuotao Xu, Qi Chen, Qianxi Zhang, Cheng Li , Ziyue Yang, Fan Yang, Yuqing Yang, Peng Cheng, Mao Yang
  
  SOSP’23 | October 2023
  
  Microsoft Research Blog
  
  https://www.rarnonalumber.com/en-us/research/blog/unified-database-laying-the-foundation-for-large-language-model-vertical-applications/
  
  Publication
- PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
  
  Ningxin Zheng, Huiqiang Jiang, Quanlu Zhang, Zhenhua Han, Lingxiao Ma, Yuqing Yang, Fan Yang, Chengruidong Zhang, Lili Qiu, Mao Yang, Lidong Zhou
  
  The 29th ACM Symposium on Operating Systems Principles | October 2023
  
  Publication
- ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
  
  Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
  
  Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
  
  Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
  
  KDD 2023 | August 2023
  
  Publication
- Optimizing Dynamic Neural Networks with Brainstorm
  
  Weihao Cui, Zhenhua Han, Lingji Ouyang, Yichuan Wang, Ningxin Zheng, Lingxiao Ma, Yuqing Yang, Fan Yang, Jilong Xue, Lili Qiu, Lidong Zhou, Quan Chen, Haisheng Tan, Minyi Guo
  
  OSDI’23 | July 2023
  
  Publication
- VBASE: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity
  
  Qianxi Zhang, Shuotao Xu, Qi Chen, Guoxin Sui, Jiadong Xie, Zhizhen Cai, Yaoqi Chen, Yinxuan He, Yuqing Yang, Fan Yang, Mao Yang, Lidong Zhou
  
  OSDI’23 | July 2023
  
  Publication
- Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
  
  Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
  
  Sixth Conference on Machine Learning and Systems (MLSys’23) | June 2023
  
  Publication 项目
- SiloD: A Co-design of Caching and Scheduling for Deep Learning Clusters
  
  Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Mingxia Li, Fan Yang, Qianxi Zhang, Binyang Li, Yuqing Yang, Lili Qiu, Lintao Zhang, Lidong Zhou
  
  EuroSys 2023 | May 2023
  
  Publication
- PilotFish: Harvesting Free Cycles of Cloud Gaming with Deep Learning Training
  
  Wei Zhang , Binghao Chen, Zhenhua Han, Quan Chen, Peng Cheng, Fan Yang, Ran Shu, Yuqing Yang, Minyi Guo
  
  2022 USENIX Annual Technical Conference | July 2022
  
  PDF
- SparTA: Deep-Learning Model Sparsity via Tensor-with-Sparsity-Attribute
  
  Ningxin Zheng, Bin Lin, Quanlu Zhang, Lingxiao Ma, Yuqing Yang, Fan Yang, Yang Wang, Mao Yang, Lidong Zhou
  
  OSDI 2022 | July 2022
  
  Publication 项目项目
- nn-METER: Towards Accurate Latency Prediction of DNN Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  GetMobile: Mobile Computing and Communications | December 2021, 第 25 卷(4): pp. 19-23
  
  SigMobile Research Highlight
  
  Publication
- nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys’21) | June 2021
  
  Best Paper Award
  
  Publication
- HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees
  
  Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Fan Yang, Lidong Zhou, Mao Yang, Yuqi Wang, Yifan Xiong, Bin Wang
  
  2020 Operating Systems Design and Implementation | November 2020
  
  Publication 项目
- Retiarii: A Deep Learning Exploratory-Training Framework
  
  Quanlu Zhang, Zhenhua Han, Fan Yang, Yuge Zhang, Zhe Liu, Mao Yang, Lidong Zhou
  
  2020 Operating Systems Design and Implementation | November 2020
  
  Publication 视频
- Fast Hardware-Aware Neural Architecture Search
  
  Li Lyna Zhang, Yuqing Yang, Yuhang Jiang, Wenwu Zhu, Yunxin Liu
  
  CVPRW (Joint Workshop on Efficient Deep Learning in Computer Vision) | June 2020
  
  DOI PDF Publication
- LLM2CLIP: Powerful Language Model Unlocks Richer Visual Representation
  
  Weiquan Huang, Aoqi Wu, Yifan Yang, Xufang Luo, Yuqing Yang, Liang Hu, Qi Dai, Xiyang Dai, Dongdong Chen, Chong Luo, Lili Qiu
  
  November 2024
  
  Publication Github 项目
- EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
  
  Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, Yixuan Yuan
  
  June 2023
  
  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  
  Publication
- Privacy-preserving Online AutoML for Domain-Specific Face Detection
  
  Chenqian Yan, Yuge Zhang, Quanlu Zhang, Yaming Yang, Xinyang Jiang, Yuqing Yang, Baoyuan Wang
  
  CVPR 2022 | May 2022
  
  Publication
- Large Multimodal Model for Real-World Radiology Report Generation
  
  Brian Nlong Zhao, Xinyang Jiang, Xufang Luo, Yifan Yang, Bo Li, Zilong Wang, Javier Alvarez-Valle, Matthew P. Lungren, Dongsheng Li, Lili Qiu
  
  December 2023
  
  Publication
- nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys’21) | June 2021
  
  Best Paper Award
  
  Publication

- MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention
  
  Yucheng Li, Huiqiang Jiang, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICML 2025 | July 2025
  
  Preprint Github 项目
- SCBench: A KV Cache-Centric Analysis of Long-Context Methods
  
  Yucheng Li, Huiqiang Jiang, Qianhui Wu, Xufang Luo, Surin Ahn, Chengruidong Zhang, Amir H. Abdi, Dongsheng Li, Jianfeng Gao, Yuqing Yang, Lili Qiu
  
  ICLR 2025 | December 2024
  
  Preprint Github 项目
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention
  
  Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  NeurIPS 2024 | December 2024
  
  spotlight
  
  Preprint Github 项目
- LORASC: Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning
  
  Yifan Yang, Yifei Shen, Yuqing Yang, Lili Qiu, Fangyun Wei
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  Publication
- Reviving Cloud Gaming Sessions
  
  Yifan Yang, Lili Qiu, Yuqing Yang
  
  CoNEXT 2024 – International Conference on emerging Networking EXperiments and Technologies | November 2024
  
  Preprint
- RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
  
  Di Liu, Meng Chen, Baotong Lu, Huiqiang Jiang, Qianxi Zhang, Qi Chen, Chengruidong Zhang, Bailu Ding, Kai Zhang, Chen Chen, Fan Yang, Yuqing Yang, Lili Qiu
  
  ENLSP NeurIPS Workshop 2024 | September 2024
  
  Best Paper Award
  
  Preprint
- Real-Time Neural Video Recovery and Enhancement on Mobile Devices
  
  Yifan Yang, Yuqing Yang, Lili Qiu
  
  CoNEXT 2024 – International Conference on emerging Networking EXperiments and Technologies | April 2024
  
  Preprint
- LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression
  
  Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin 林庆维, Victor Ruehle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, Dongmei Zhang
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (Findings of ACL 2024) | March 2024
  
  Preprint 项目
- LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
  
  Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023) | December 2023
  
  Publication 项目
- SPFresh: Incremental In-Place Update for Billion-Scale Vector Search
  
  Yuming Xu, Hengyu Liang, Jin Li, Shuotao Xu, Qi Chen, Qianxi Zhang, Cheng Li , Ziyue Yang, Fan Yang, Yuqing Yang, Peng Cheng, Mao Yang
  
  SOSP’23 | October 2023
  
  Microsoft Research Blog
  
  https://www.rarnonalumber.com/en-us/research/blog/unified-database-laying-the-foundation-for-large-language-model-vertical-applications/
  
  Publication
- PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation
  
  Ningxin Zheng, Huiqiang Jiang, Quanlu Zhang, Zhenhua Han, Lingxiao Ma, Yuqing Yang, Fan Yang, Chengruidong Zhang, Lili Qiu, Mao Yang, Lidong Zhou
  
  The 29th ACM Symposium on Operating Systems Principles | October 2023
  
  Publication
- LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
  
  Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
  
  The 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) | October 2023
  
  Preprint 项目
- ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
  
  Chen Tang, Li Lyna Zhang, Huiqiang Jiang, Jiahang Xu, Ting Cao, Quanlu Zhang, Yuqing Yang, Zhi Wang, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
  
  Xudong Wang, Li Lyna Zhang, Jiahang Xu, Quanlu Zhang, Yujing Wang, Yuqing Yang, Ningxin Zheng, Ting Cao, Mao Yang
  
  ICCV 2023 | October 2023
  
  Publication
- Attentive Mask CLIP
  
  Yifan Yang, Houwen Peng, Xinyang Jiang, Huiqiang Jiang, Yuqing Yang, Lili Qiu
  
  2023 International Conference on Computer Vision | October 2023
  
  Publication
- ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulation
  
  Yasheng Sun, Yifan Yang, Houwen Peng, Yifei Shen, Yuqing Yang, Han Hu, Lili Qiu, Hideki Koike, Yuqing Yang
  
  NeurIPS 2023 | October 2023
  
  Publication
- Accurate and Structured Pruning for Efficient Automatic Speech Recognition
  
  Huiqiang Jiang, Li Lyna Zhang, Yuang Li, Yu Wu, Shijie Cao, Ting Cao, Yuqing Yang, Jinyu Li, Mao Yang, Lili Qiu
  
  INTERSPEECH 2023 | August 2023
  
  Publication
- Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
  
  Junyan Li, Li Lyna Zhang, Jiahang Xu, Yujing Wang, Shaoguang Yan, Yunqing Xia, Yuqing Yang, Ting Cao, Hao Sun, Weiwei Deng, Qi Zhang, Mao Yang
  
  KDD 2023 | August 2023
  
  Publication
- Optimizing Dynamic Neural Networks with Brainstorm
  
  Weihao Cui, Zhenhua Han, Lingji Ouyang, Yichuan Wang, Ningxin Zheng, Lingxiao Ma, Yuqing Yang, Fan Yang, Jilong Xue, Lili Qiu, Lidong Zhou, Quan Chen, Haisheng Tan, Minyi Guo
  
  OSDI’23 | July 2023
  
  Publication
- VBASE: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity
  
  Qianxi Zhang, Shuotao Xu, Qi Chen, Guoxin Sui, Jiadong Xie, Zhizhen Cai, Yaoqi Chen, Yinxuan He, Yuqing Yang, Fan Yang, Mao Yang, Lidong Zhou
  
  OSDI’23 | July 2023
  
  Publication
- Efficient GPU Kernels for N:M-SPARSE Weights in Deep Learning
  
  Bin Lin, Ningxin Zheng, Lei Wang, Shijie Cao, Lingxiao Ma, Quanlu Zhang, Yi Zhu, Ting Cao, Jilong Xue, Yuqing Yang, Fan Yang
  
  Sixth Conference on Machine Learning and Systems (MLSys’23) | June 2023
  
  Publication 项目
- SiloD: A Co-design of Caching and Scheduling for Deep Learning Clusters
  
  Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Mingxia Li, Fan Yang, Qianxi Zhang, Binyang Li, Yuqing Yang, Lili Qiu, Lintao Zhang, Lidong Zhou
  
  EuroSys 2023 | May 2023
  
  Publication
- Reinforcement Learning with Automated Auxiliary Loss Search
  
  Tairan He, Yuge Zhang, Kan Ren, Minghuan Liu, Che Wang, Weinan Zhang, Yuqing Yang, Dongsheng Li
  
  2022 Neural Information Processing Systems | October 2022
  
  Publication
- PilotFish: Harvesting Free Cycles of Cloud Gaming with Deep Learning Training
  
  Wei Zhang , Binghao Chen, Zhenhua Han, Quan Chen, Peng Cheng, Fan Yang, Ran Shu, Yuqing Yang, Minyi Guo
  
  2022 USENIX Annual Technical Conference | July 2022
  
  PDF
- SparTA: Deep-Learning Model Sparsity via Tensor-with-Sparsity-Attribute
  
  Ningxin Zheng, Bin Lin, Quanlu Zhang, Lingxiao Ma, Yuqing Yang, Fan Yang, Yang Wang, Mao Yang, Lidong Zhou
  
  OSDI 2022 | July 2022
  
  Publication 项目项目
- Privacy-preserving Online AutoML for Domain-Specific Face Detection
  
  Chenqian Yan, Yuge Zhang, Quanlu Zhang, Yaming Yang, Xinyang Jiang, Yuqing Yang, Baoyuan Wang
  
  CVPR 2022 | May 2022
  
  Publication
- nn-Meter: Towards Accurate Latency Prediction of Deep-Learning Model Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  The 19th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys’21) | June 2021
  
  Best Paper Award
  
  Publication
- HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees
  
  Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Fan Yang, Lidong Zhou, Mao Yang, Yuqi Wang, Yifan Xiong, Bin Wang
  
  2020 Operating Systems Design and Implementation | November 2020
  
  Publication 项目
- Retiarii: A Deep Learning Exploratory-Training Framework
  
  Quanlu Zhang, Zhenhua Han, Fan Yang, Yuge Zhang, Zhe Liu, Mao Yang, Lidong Zhou
  
  2020 Operating Systems Design and Implementation | November 2020
  
  Publication 视频
- Fast Hardware-Aware Neural Architecture Search
  
  Li Lyna Zhang, Yuqing Yang, Yuhang Jiang, Wenwu Zhu, Yunxin Liu
  
  CVPRW (Joint Workshop on Efficient Deep Learning in Computer Vision) | June 2020
  
  DOI PDF Publication
- LLM2CLIP: Powerful Language Model Unlocks Richer Visual Representation
  
  Weiquan Huang, Aoqi Wu, Yifan Yang, Xufang Luo, Yuqing Yang, Liang Hu, Qi Dai, Xiyang Dai, Dongdong Chen, Chong Luo, Lili Qiu
  
  November 2024
  
  Publication Github 项目
- Mitigate Position Bias in Large Language Models via Scaling a Single Dimension
  
  Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, Lili Qiu
  
  June 2024
  
  Preprint
- Large Multimodal Model for Real-World Radiology Report Generation
  
  Brian Nlong Zhao, Xinyang Jiang, Xufang Luo, Yifan Yang, Bo Li, Zilong Wang, Javier Alvarez-Valle, Matthew P. Lungren, Dongsheng Li, Lili Qiu
  
  December 2023
  
  Publication
- EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
  
  Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, Yixuan Yuan
  
  June 2023
  
  Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  
  Publication
- nn-METER: Towards Accurate Latency Prediction of DNN Inference on Diverse Edge Devices
  
  Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, Yunxin Liu
  
  GetMobile: Mobile Computing and Communications | December 2021, 第 25 卷(4): pp. 19-23
  
  SigMobile Research Highlight
  
  Publication
- AceNAS: Learning to Rank Ace Neural Architectures with Weak Supervision of Weight Sharing
  
  Yuge Zhang, Chenqian Yan, Quanlu Zhang, Li Lyna Zhang, Yaming Yang, Xiaotian Gao, Yuqing Yang
  
  August 2021
  
  发布者： arXiv | August 2021
  
  Publication