Autotuning Systems: Techniques, Challenges, and Opportunities
Brian Kroth, Sergiy Matusevych, Yiwen Zhu
SIGMOD 2025 | June 2025
Brian Kroth
Principal RSDE
Publications
-
-
-
Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment
Yiwen Zhu, Rathijit Sen, Brian Kroth, Sergiy Matusevych, Andreas Mueller, Tengfei Huang, Rahul Challapalli, Weihan Tang, Xin He, Mo Liu, Estera Kot, Sule Kahraman, Arshdeep Sekhon, Dario Bernal, Aditya Lakra, Shaily Fozdar, Dhruv Relwani, Rui Fang, Long Tian, Karuna Sagar Krishna, Ashit Gosalia, Carlo Curino, Subru Krishnan
Companion of the 2025 International Conference on Management of Data | June 2025
-
-
-
MLOS in Action: Bridging the Gap Between Experimentation and Auto-Tuning in the Cloud
Brian Kroth, Sergiy Matusevych, Rana Alotaibi, Yiwen Zhu, Anja Gruenheid, Yuanyuan Tian
Proc. VLDB Endow. | October 2024, Vol 17: pp. 4269-4272
-
Vertically Autoscaling Monolithic Applications with CaaSPER: Scalable Container-as-a-Service Performance Enhanced Resizing Algorithm for the Cloud
Anna Pavlenko, Joyce Cahoon, Yiwen Zhu, Brian Kroth, Michael Nelson, Andrew Carter, David Liao, Travis Wright, Jesús Camacho-Rodríguez, Karla Saur
SIGMOD | June 2024
to appear
-
VASIM: Vertical Autoscaling Simulator Toolkit
Anna Pavlenko, Karla Saur, Yiwen Zhu, Brian Kroth, Joyce Cahoon, Jesús Camacho-Rodríguez
IEEE International Conference on Data Engineering (ICDE 2024) | May 2024
-
-
-
Performance Roulette: How Cloud Weather Affects ML-Based System Optimization
Johannes Freischuetz, Konstantinos Kanellis, Brian Kroth, Shivaram Venkataraman
ML for Systems Workshop at NeurIPS 2023 | December 2023
-
Towards Building Autonomous Data Services on Azure
Yiwen Zhu, Yuanyuan Tian, Joyce Cahoon, Subru Krishnan, Ankita Agarwal, Rana Alotaibi, Jesús Camacho-Rodríguez, Bibin A Chundatt, Andrew Chung, Niharika Dutta, Andrew Fogarty, Anja Gruenheid, Brandon Haynes, Matteo Interlandi, Minu Iyer, Nick Jurgens, Sumeet Khushalani, Brian Kroth, Manoj Kumar, Jyoti Leeka, Sergiy Matusevych, Minni Mittal, Andreas C. Müller, Kartheek Muthyala, Harsha Nagulapalli, Yoonjae Park, Hiren Patel, Anna Pavlenko, Olga Poppe, Santhosh Ravindran, Karla Saur, Rathijit Sen, Steve Suh, Arijit Tarafdar, Kunal Waghray, Demin Wang, Carlo Curino, Raghu Ramakrishnan
ACM SIGMOD | June 2023
-
-
-
VIP Hashing — Adapting to Skew in Popularity of Data on the Fly
Aarati Kakaraparthy, Jignesh M. Patel, Brian Kroth, Kwanghyun Park
VLDB 2022 | June 2022
-
Data Science Through the Looking Glass
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | June 2022, Vol 51(2): pp. 30-37
-
LlamaTune: Sample-Efficient DBMS Configuration Tuning
Konstantinos Kanellis, Cong Ding, Brian Kroth, Andreas C. Müller, Carlo Curino, Shivaram Venkataraman
VLDB 2022 | March 2022
-
Data Science Through the Looking Glass: Analysis of Millions of GitHub Notebooks and ML.NET Pipelines
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | March 2022
-
-
-
Lessons learned from the early performance evaluation of Intel Optane DC persistent memory in DBMS
Yinjun Wu, Kwanghyun Park, Rathijit Sen, Brian Kroth, Jae Young Do
2020 Data Management on New Hardware | June 2020
-
MLOS: An Infrastructure for Automated Software Performance Engineering
Carlo Curino, Neha Godwal, Brian Kroth, Sergiy Kuryata, Greg Lapinski, Siqi Liu, Slava Oks, Olga Poppe, Adam Smiechowski, Ed Thayer, Markus Weimer, Yiwen Zhu
DEEM 2020 | June 2020
-
Cloudy with high chance of DBMS: A 10-year prediction for Enterprise-Grade ML
Ashvin Agrawal, Rony Chatterjee, Carlo Curino, Avrilia Floratou, Neha Godwal, Matteo Interlandi, Alekh Jindal, Konstantinos Karanasos, Subru Krishnan, Brian Kroth, Jyoti Leeka, Kwanghyun Park, Hiren Patel, Olga Poppe, Fotis Psallidas, Raghu Ramakrishnan, Abhishek Roy, Karla Saur, Rathijit Sen, Markus Weimer, Travis Wright, Yiwen Zhu
Conference on Innovative Data Systems Research (CIDR) | January 2020
-
-
-
Optimizing databases by learning hidden parameters of solid state drives
Aarati Kakaraparthy, Jignesh M. Patel, Kwanghyun Park, Brian Kroth
Proceedings of VLDB | December 2019
-
-
-
Autotuning Systems: Techniques, Challenges, and Opportunities
Brian Kroth, Sergiy Matusevych, Yiwen Zhu
SIGMOD 2025 | June 2025
-
Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment
Yiwen Zhu, Rathijit Sen, Brian Kroth, Sergiy Matusevych, Andreas Mueller, Tengfei Huang, Rahul Challapalli, Weihan Tang, Xin He, Mo Liu, Estera Kot, Sule Kahraman, Arshdeep Sekhon, Dario Bernal, Aditya Lakra, Shaily Fozdar, Dhruv Relwani, Rui Fang, Long Tian, Karuna Sagar Krishna, Ashit Gosalia, Carlo Curino, Subru Krishnan
Companion of the 2025 International Conference on Management of Data | June 2025
-
MLOS in Action: Bridging the Gap Between Experimentation and Auto-Tuning in the Cloud
Brian Kroth, Sergiy Matusevych, Rana Alotaibi, Yiwen Zhu, Anja Gruenheid, Yuanyuan Tian
Proc. VLDB Endow. | October 2024, Vol 17: pp. 4269-4272
-
Performance Roulette: How Cloud Weather Affects ML-Based System Optimization
Johannes Freischuetz, Konstantinos Kanellis, Brian Kroth, Shivaram Venkataraman
ML for Systems Workshop at NeurIPS 2023 | December 2023
-
Towards Building Autonomous Data Services on Azure
Yiwen Zhu, Yuanyuan Tian, Joyce Cahoon, Subru Krishnan, Ankita Agarwal, Rana Alotaibi, Jesús Camacho-Rodríguez, Bibin A Chundatt, Andrew Chung, Niharika Dutta, Andrew Fogarty, Anja Gruenheid, Brandon Haynes, Matteo Interlandi, Minu Iyer, Nick Jurgens, Sumeet Khushalani, Brian Kroth, Manoj Kumar, Jyoti Leeka, Sergiy Matusevych, Minni Mittal, Andreas C. Müller, Kartheek Muthyala, Harsha Nagulapalli, Yoonjae Park, Hiren Patel, Anna Pavlenko, Olga Poppe, Santhosh Ravindran, Karla Saur, Rathijit Sen, Steve Suh, Arijit Tarafdar, Kunal Waghray, Demin Wang, Carlo Curino, Raghu Ramakrishnan
ACM SIGMOD | June 2023
-
Data Science Through the Looking Glass
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | June 2022, Vol 51(2): pp. 30-37
-
Data Science Through the Looking Glass: Analysis of Millions of GitHub Notebooks and ML.NET Pipelines
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | March 2022
-
MLOS: An Infrastructure for Automated Software Performance Engineering
Carlo Curino, Neha Godwal, Brian Kroth, Sergiy Kuryata, Greg Lapinski, Siqi Liu, Slava Oks, Olga Poppe, Adam Smiechowski, Ed Thayer, Markus Weimer, Yiwen Zhu
DEEM 2020 | June 2020
-
Cloudy with high chance of DBMS: A 10-year prediction for Enterprise-Grade ML
Ashvin Agrawal, Rony Chatterjee, Carlo Curino, Avrilia Floratou, Neha Godwal, Matteo Interlandi, Alekh Jindal, Konstantinos Karanasos, Subru Krishnan, Brian Kroth, Jyoti Leeka, Kwanghyun Park, Hiren Patel, Olga Poppe, Fotis Psallidas, Raghu Ramakrishnan, Abhishek Roy, Karla Saur, Rathijit Sen, Markus Weimer, Travis Wright, Yiwen Zhu
Conference on Innovative Data Systems Research (CIDR) | January 2020
-
-
-
Autotuning Systems: Techniques, Challenges, and Opportunities
Brian Kroth, Sergiy Matusevych, Yiwen Zhu
SIGMOD 2025 | June 2025
-
Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment
Yiwen Zhu, Rathijit Sen, Brian Kroth, Sergiy Matusevych, Andreas Mueller, Tengfei Huang, Rahul Challapalli, Weihan Tang, Xin He, Mo Liu, Estera Kot, Sule Kahraman, Arshdeep Sekhon, Dario Bernal, Aditya Lakra, Shaily Fozdar, Dhruv Relwani, Rui Fang, Long Tian, Karuna Sagar Krishna, Ashit Gosalia, Carlo Curino, Subru Krishnan
Companion of the 2025 International Conference on Management of Data | June 2025
-
Vertically Autoscaling Monolithic Applications with CaaSPER: Scalable Container-as-a-Service Performance Enhanced Resizing Algorithm for the Cloud
Anna Pavlenko, Joyce Cahoon, Yiwen Zhu, Brian Kroth, Michael Nelson, Andrew Carter, David Liao, Travis Wright, Jesús Camacho-Rodríguez, Karla Saur
SIGMOD | June 2024
to appear
-
VASIM: Vertical Autoscaling Simulator Toolkit
Anna Pavlenko, Karla Saur, Yiwen Zhu, Brian Kroth, Joyce Cahoon, Jesús Camacho-Rodríguez
IEEE International Conference on Data Engineering (ICDE 2024) | May 2024
-
Towards Building Autonomous Data Services on Azure
Yiwen Zhu, Yuanyuan Tian, Joyce Cahoon, Subru Krishnan, Ankita Agarwal, Rana Alotaibi, Jesús Camacho-Rodríguez, Bibin A Chundatt, Andrew Chung, Niharika Dutta, Andrew Fogarty, Anja Gruenheid, Brandon Haynes, Matteo Interlandi, Minu Iyer, Nick Jurgens, Sumeet Khushalani, Brian Kroth, Manoj Kumar, Jyoti Leeka, Sergiy Matusevych, Minni Mittal, Andreas C. Müller, Kartheek Muthyala, Harsha Nagulapalli, Yoonjae Park, Hiren Patel, Anna Pavlenko, Olga Poppe, Santhosh Ravindran, Karla Saur, Rathijit Sen, Steve Suh, Arijit Tarafdar, Kunal Waghray, Demin Wang, Carlo Curino, Raghu Ramakrishnan
ACM SIGMOD | June 2023
-
VIP Hashing — Adapting to Skew in Popularity of Data on the Fly
Aarati Kakaraparthy, Jignesh M. Patel, Brian Kroth, Kwanghyun Park
VLDB 2022 | June 2022
-
Data Science Through the Looking Glass
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | June 2022, Vol 51(2): pp. 30-37
-
LlamaTune: Sample-Efficient DBMS Configuration Tuning
Konstantinos Kanellis, Cong Ding, Brian Kroth, Andreas C. Müller, Carlo Curino, Shivaram Venkataraman
VLDB 2022 | March 2022
-
Data Science Through the Looking Glass: Analysis of Millions of GitHub Notebooks and ML.NET Pipelines
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | March 2022
-
Lessons learned from the early performance evaluation of Intel Optane DC persistent memory in DBMS
Yinjun Wu, Kwanghyun Park, Rathijit Sen, Brian Kroth, Jae Young Do
2020 Data Management on New Hardware | June 2020
-
MLOS: An Infrastructure for Automated Software Performance Engineering
Carlo Curino, Neha Godwal, Brian Kroth, Sergiy Kuryata, Greg Lapinski, Siqi Liu, Slava Oks, Olga Poppe, Adam Smiechowski, Ed Thayer, Markus Weimer, Yiwen Zhu
DEEM 2020 | June 2020
-
Cloudy with high chance of DBMS: A 10-year prediction for Enterprise-Grade ML
Ashvin Agrawal, Rony Chatterjee, Carlo Curino, Avrilia Floratou, Neha Godwal, Matteo Interlandi, Alekh Jindal, Konstantinos Karanasos, Subru Krishnan, Brian Kroth, Jyoti Leeka, Kwanghyun Park, Hiren Patel, Olga Poppe, Fotis Psallidas, Raghu Ramakrishnan, Abhishek Roy, Karla Saur, Rathijit Sen, Markus Weimer, Travis Wright, Yiwen Zhu
Conference on Innovative Data Systems Research (CIDR) | January 2020
-
Optimizing databases by learning hidden parameters of solid state drives
Aarati Kakaraparthy, Jignesh M. Patel, Kwanghyun Park, Brian Kroth
Proceedings of VLDB | December 2019
-
-
-
Autotuning Systems: Techniques, Challenges, and Opportunities
Brian Kroth, Sergiy Matusevych, Yiwen Zhu
SIGMOD 2025 | June 2025
-
MLOS in Action: Bridging the Gap Between Experimentation and Auto-Tuning in the Cloud
Brian Kroth, Sergiy Matusevych, Rana Alotaibi, Yiwen Zhu, Anja Gruenheid, Yuanyuan Tian
Proc. VLDB Endow. | October 2024, Vol 17: pp. 4269-4272
-
Performance Roulette: How Cloud Weather Affects ML-Based System Optimization
Johannes Freischuetz, Konstantinos Kanellis, Brian Kroth, Shivaram Venkataraman
ML for Systems Workshop at NeurIPS 2023 | December 2023
-
Lessons learned from the early performance evaluation of Intel Optane DC persistent memory in DBMS
Yinjun Wu, Kwanghyun Park, Rathijit Sen, Brian Kroth, Jae Young Do
2020 Data Management on New Hardware | June 2020
-
-
-
MLOS in Action: Bridging the Gap Between Experimentation and Auto-Tuning in the Cloud
Brian Kroth, Sergiy Matusevych, Rana Alotaibi, Yiwen Zhu, Anja Gruenheid, Yuanyuan Tian
Proc. VLDB Endow. | October 2024, Vol 17: pp. 4269-4272
-
VASIM: Vertical Autoscaling Simulator Toolkit
Anna Pavlenko, Karla Saur, Yiwen Zhu, Brian Kroth, Joyce Cahoon, Jesús Camacho-Rodríguez
IEEE International Conference on Data Engineering (ICDE 2024) | May 2024
-
Data Science Through the Looking Glass
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | June 2022, Vol 51(2): pp. 30-37
-
MLOS: An Infrastructure for Automated Software Performance Engineering
Carlo Curino, Neha Godwal, Brian Kroth, Sergiy Kuryata, Greg Lapinski, Siqi Liu, Slava Oks, Olga Poppe, Adam Smiechowski, Ed Thayer, Markus Weimer, Yiwen Zhu
DEEM 2020 | June 2020
-
-
-
Optimizing databases by learning hidden parameters of solid state drives
Aarati Kakaraparthy, Jignesh M. Patel, Kwanghyun Park, Brian Kroth
Proceedings of VLDB | December 2019
-
-
-
Rockhopper: A Robust Optimizer for Spark Configuration Tuning in Production Environment
Yiwen Zhu, Rathijit Sen, Brian Kroth, Sergiy Matusevych, Andreas Mueller, Tengfei Huang, Rahul Challapalli, Weihan Tang, Xin He, Mo Liu, Estera Kot, Sule Kahraman, Arshdeep Sekhon, Dario Bernal, Aditya Lakra, Shaily Fozdar, Dhruv Relwani, Rui Fang, Long Tian, Karuna Sagar Krishna, Ashit Gosalia, Carlo Curino, Subru Krishnan
Companion of the 2025 International Conference on Management of Data | June 2025
-
MLOS in Action: Bridging the Gap Between Experimentation and Auto-Tuning in the Cloud
Brian Kroth, Sergiy Matusevych, Rana Alotaibi, Yiwen Zhu, Anja Gruenheid, Yuanyuan Tian
Proc. VLDB Endow. | October 2024, Vol 17: pp. 4269-4272
-
Data Science Through the Looking Glass
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | June 2022, Vol 51(2): pp. 30-37
-
Data Science Through the Looking Glass: Analysis of Millions of GitHub Notebooks and ML.NET Pipelines
Fotis Psallidas, Yiwen Zhu, Bojan Karlaš, Jordan Henkel, Matteo Interlandi, Subru Krishnan, Brian Kroth, Venkatesh Emani, Wentao Wu, Ce Zhang, Markus Weimer, Avrilia Floratou, Carlo Curino, Konstantinos Karanasos
SIGMOD Record | March 2022
-
-
-
Autotuning Systems: Techniques, Challenges, and Opportunities
Brian Kroth, Sergiy Matusevych, Yiwen Zhu
SIGMOD 2025 | June 2025
-
Vertically Autoscaling Monolithic Applications with CaaSPER: Scalable Container-as-a-Service Performance Enhanced Resizing Algorithm for the Cloud
Anna Pavlenko, Joyce Cahoon, Yiwen Zhu, Brian Kroth, Michael Nelson, Andrew Carter, David Liao, Travis Wright, Jesús Camacho-Rodríguez, Karla Saur
SIGMOD | June 2024
to appear
-
VASIM: Vertical Autoscaling Simulator Toolkit
Anna Pavlenko, Karla Saur, Yiwen Zhu, Brian Kroth, Joyce Cahoon, Jesús Camacho-Rodríguez
IEEE International Conference on Data Engineering (ICDE 2024) | May 2024
-
Performance Roulette: How Cloud Weather Affects ML-Based System Optimization
Johannes Freischuetz, Konstantinos Kanellis, Brian Kroth, Shivaram Venkataraman
ML for Systems Workshop at NeurIPS 2023 | December 2023
-
Towards Building Autonomous Data Services on Azure
Yiwen Zhu, Yuanyuan Tian, Joyce Cahoon, Subru Krishnan, Ankita Agarwal, Rana Alotaibi, Jesús Camacho-Rodríguez, Bibin A Chundatt, Andrew Chung, Niharika Dutta, Andrew Fogarty, Anja Gruenheid, Brandon Haynes, Matteo Interlandi, Minu Iyer, Nick Jurgens, Sumeet Khushalani, Brian Kroth, Manoj Kumar, Jyoti Leeka, Sergiy Matusevych, Minni Mittal, Andreas C. Müller, Kartheek Muthyala, Harsha Nagulapalli, Yoonjae Park, Hiren Patel, Anna Pavlenko, Olga Poppe, Santhosh Ravindran, Karla Saur, Rathijit Sen, Steve Suh, Arijit Tarafdar, Kunal Waghray, Demin Wang, Carlo Curino, Raghu Ramakrishnan
ACM SIGMOD | June 2023
-
VIP Hashing — Adapting to Skew in Popularity of Data on the Fly
Aarati Kakaraparthy, Jignesh M. Patel, Brian Kroth, Kwanghyun Park
VLDB 2022 | June 2022
-
LlamaTune: Sample-Efficient DBMS Configuration Tuning
Konstantinos Kanellis, Cong Ding, Brian Kroth, Andreas C. Müller, Carlo Curino, Shivaram Venkataraman
VLDB 2022 | March 2022
-
Lessons learned from the early performance evaluation of Intel Optane DC persistent memory in DBMS
Yinjun Wu, Kwanghyun Park, Rathijit Sen, Brian Kroth, Jae Young Do
2020 Data Management on New Hardware | June 2020
-
MLOS: An Infrastructure for Automated Software Performance Engineering
Carlo Curino, Neha Godwal, Brian Kroth, Sergiy Kuryata, Greg Lapinski, Siqi Liu, Slava Oks, Olga Poppe, Adam Smiechowski, Ed Thayer, Markus Weimer, Yiwen Zhu
DEEM 2020 | June 2020
-
Cloudy with high chance of DBMS: A 10-year prediction for Enterprise-Grade ML
Ashvin Agrawal, Rony Chatterjee, Carlo Curino, Avrilia Floratou, Neha Godwal, Matteo Interlandi, Alekh Jindal, Konstantinos Karanasos, Subru Krishnan, Brian Kroth, Jyoti Leeka, Kwanghyun Park, Hiren Patel, Olga Poppe, Fotis Psallidas, Raghu Ramakrishnan, Abhishek Roy, Karla Saur, Rathijit Sen, Markus Weimer, Travis Wright, Yiwen Zhu
Conference on Innovative Data Systems Research (CIDR) | January 2020
-
Optimizing databases by learning hidden parameters of solid state drives
Aarati Kakaraparthy, Jignesh M. Patel, Kwanghyun Park, Brian Kroth
Proceedings of VLDB | December 2019
-