What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression
Mengnan Du, Subhabrata (Subho) Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Awadallah
EACL | April 2023
Publications
-
-
-
-
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
-
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
-
-
-
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
Subhabrata (Subho) Mukherjee, Ahmed Awadallah, Jianfeng Gao
ArXiv | June 2021
arXiv:2106.04563
-
-
-
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
-
-
-
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019
-
-
-
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
-
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
-
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
-
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019
-
-
-
What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression
Mengnan Du, Subhabrata (Subho) Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Awadallah
EACL | April 2023
-
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
-
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
-
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
Subhabrata (Subho) Mukherjee, Ahmed Awadallah, Jianfeng Gao
ArXiv | June 2021
arXiv:2106.04563
-
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
-
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019
-
-
-
What do Compressed Large Language Models Forget? Robustness Challenges in Model Compression
Mengnan Du, Subhabrata (Subho) Mukherjee, Yu Cheng, Milad Shokouhi, Xia Hu, Ahmed Awadallah
EACL | April 2023
-
LiteTransformerSearch: Training-free On-device Search for Efficient Autoregressive Language Models
Mojan Javaheripi, Shital Shah, Subhabrata (Subho) Mukherjee, Tomasz L. Religa, Caio C. T. Mendes, Gustavo H. de Rosa, Sébastien Bubeck, Farinaz Koushanfar, Debadeepta Dey
NeurIPS 2022 | December 2022
-
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Dongkuan Xu, Subhabrata (Subho) Mukherjee, Xiaodong Liu, Debadeepta Dey, Wenhui Wang, Xiang Zhang, Ahmed Awadallah, Jianfeng Gao
NeurIPS 2022 | December 2022
-
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
Annual Conference of the Association for Computational Linguistics (ACL 2020) | July 2020
-
-
-
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
Subhabrata (Subho) Mukherjee, Ahmed Awadallah, Jianfeng Gao
ArXiv | June 2021
arXiv:2106.04563
-
Distilling BERT into Simple Neural Network with Unlabeled Transfer Data
Subhabrata (Subho) Mukherjee, Ahmed Awadallah
ArXiv | October 2019
-