AI Infrastructure

Towards efficient AI/ML deployment

论文与出版物

查看方式：

- POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
  
  Aditya K Kamath, Ramya Prabhu, Jayashree Mohan, Simon Peter, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  Publication
- vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
  
  Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  PDF Preprint Github
- Towards Efficient Large Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  February 2025
  
  Preprint 项目
- Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
  
  Amey Agrawal, Anmol Agarwal, Nitin Kedia, Jayashree Mohan, Souvik Kundu, Nipun Kwatra, Ramachandran Ramjee, Alexey Tumanov
  
  July 2024
  
  Preprint
- VIDUR: A Large-Scale Simulation Framework for LLM Inference
  
  Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
  
  MLSys | May 2024
  
  PDF Github
- Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
  
  Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
  
  2024 European Conference on Computer Systems | April 2024
  
  DOI Publication
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
  
  Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, Ramachandran Ramjee
  
  OSDI ’24 | March 2024
  
  DOI Publication Publication Github
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
  
  Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
  
  September 2023
  
  Preprint
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication
- Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
  
  Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
  
  EuroSys 2022 | April 2022
  
  Best Paper Award
  
  Publication PDF
- Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
  
  Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
  
  February 2022
  
  Publication
- Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
  
  Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
  
  Fifteenth European Conference on Computer Systems (EuroSys’20) | April 2020
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication 项目
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication 项目项目
- Gandiva: Introspective Cluster Scheduling for Deep Learning
  
  Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
  
  13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
  
  Publication 项目项目

- POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
  
  Aditya K Kamath, Ramya Prabhu, Jayashree Mohan, Simon Peter, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  Publication
- vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
  
  Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  PDF Preprint Github
- Towards Efficient Large Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  February 2025
  
  Preprint 项目
- Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
  
  Amey Agrawal, Anmol Agarwal, Nitin Kedia, Jayashree Mohan, Souvik Kundu, Nipun Kwatra, Ramachandran Ramjee, Alexey Tumanov
  
  July 2024
  
  Preprint
- VIDUR: A Large-Scale Simulation Framework for LLM Inference
  
  Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
  
  MLSys | May 2024
  
  PDF Github
- Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
  
  Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
  
  2024 European Conference on Computer Systems | April 2024
  
  DOI Publication
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
  
  Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, Ramachandran Ramjee
  
  OSDI ’24 | March 2024
  
  DOI Publication Publication Github
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
  
  Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
  
  September 2023
  
  Preprint
- Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
  
  Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
  
  EuroSys 2022 | April 2022
  
  Best Paper Award
  
  Publication PDF
- Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
  
  Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
  
  February 2022
  
  Publication
- Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
  
  Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
  
  Fifteenth European Conference on Computer Systems (EuroSys’20) | April 2020
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication 项目
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication 项目项目
- Gandiva: Introspective Cluster Scheduling for Deep Learning
  
  Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
  
  13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
  
  Publication 项目项目
- Towards Efficient Large Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  February 2025
  
  Preprint 项目
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication 项目
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication 项目项目
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication

- POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
  
  Aditya K Kamath, Ramya Prabhu, Jayashree Mohan, Simon Peter, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  Publication
- vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention
  
  Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar
  
  Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2025 | March 2025
  
  PDF Preprint Github
- Etalon: Holistic Performance Evaluation Framework for LLM Inference Systems
  
  Amey Agrawal, Anmol Agarwal, Nitin Kedia, Jayashree Mohan, Souvik Kundu, Nipun Kwatra, Ramachandran Ramjee, Alexey Tumanov
  
  July 2024
  
  Preprint
- VIDUR: A Large-Scale Simulation Framework for LLM Inference
  
  Amey Agrawal, Nitin Kedia, Jayashree Mohan, Ashish Panwar, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee, Alexey Tumanov
  
  MLSys | May 2024
  
  PDF Github
- Just-In-Time Checkpointing: Low Cost Error Recovery from Deep Learning Training Failures
  
  Tanmaey Gupta, Sanjeev Krishnan, Rituraj Kumar, Abhishek Vijeev, Bhargav Gulavani, Nipun Kwatra, Ramachandran Ramjee, Muthian Sivathanu
  
  2024 European Conference on Computer Systems | April 2024
  
  DOI Publication
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
  
  Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, Ramachandran Ramjee
  
  OSDI ’24 | March 2024
  
  DOI Publication Publication Github
- Renee: End-to-end training of extreme classification models
  
  Vidit Jain, Jatin Prakash, Deepak Saini, Jian Jiao, Ramachandran Ramjee, Manik Varma
  
  In Proceedings of the Conference on Machine Learning and Systems, Miami, Florida | June 2023
  
  Publication
- Varuna: Scalable, Low-cost Training of Massive Deep Learning Models
  
  Sanjith Athlur, Nitika Saran, Muthian Sivathanu, Ramachandran Ramjee, Nipun Kwatra
  
  EuroSys 2022 | April 2022
  
  Best Paper Award
  
  Publication PDF
- Balancing efficiency and fairness in heterogeneous GPU clusters for deep learning
  
  Shubham Chaudhary, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Srinidhi Viswanatha
  
  Fifteenth European Conference on Computer Systems (EuroSys’20) | April 2020
  
  Publication
- Quiver: An Informed Storage Cache for Deep Learning
  
  Abhishek Vijaya Kumar, Muthian Sivathanu
  
  18th USENIX Conference on File and Storage Technologies (FAST 2020) | February 2020
  
  Publication 项目
- Astra: Exploiting Predictability to Optimize Deep Learning
  
  Muthian Sivathanu, Tapan Chugh, Sanjay S. Singapuram, Lidong Zhou
  
  ACM Symposium on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019) | April 2019
  
  Publication 项目项目
- Gandiva: Introspective Cluster Scheduling for Deep Learning
  
  Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
  
  13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2018) | October 2018
  
  Publication 项目项目
- Towards Efficient Large Multimodal Model Serving
  
  Haoran Qiu, Anish Biswas, Zihan Zhao, Jayashree Mohan, Alind Khare, Esha Choukse, Íñigo Goiri, Zeyu Zhang, Haiying Shen, Chetan Bansal, Ramachandran Ramjee, Rodrigo Fonseca
  
  February 2025
  
  Preprint 项目
- SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
  
  Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Ramachandran Ramjee
  
  September 2023
  
  Preprint
- Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
  
  Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha, Bhargav Gulavani, Rimma Nehme, Amey Agrawal, Chen Chen, Nipun Kwatra, Ramachandran Ramjee, Pankaj Sharma, Atul Katiyar, Vipul Modi, Vaibhav Sharma, Abhishek Singh, Shreshth Singhal, Kaustubh Welankar, Lu Xun, Ravi Anupindi, Karthik Elangovan, Hasibur Rehman, Zhou Lin, Rahul Seetharaman, Cheng Xu, Eddie Ailijiang, Suresh Krishnappa, Mark Russinovich
  
  February 2022
  
  Publication