SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
Chao Ma, Wenbo Gong, Meyer Scetbon, Edward Meeds
ICML 2025 | February 2025
Wenbo Gong
Senior Researcher
Publications
-
-
-
Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
Wenbo Gong, Meyer Scetbon, Chao Ma, Edward Meeds
ArXiv | February 2025
-
Gradient Multi-Normalization for Stateless and Scalable LLM Training
Meyer Scetbon, Chao Ma, Wenbo Gong, Edward Meeds
ArXiv | February 2025
-
-
-
Neural Structure Learning with Stochastic Differential Equations
Benjie Wang, Joel Jennings, Wenbo Gong
ICLR 2024 | November 2023
-
-
-
Deep End-to-end Causal Inference
Tomas Geffner, Javier Antoran, Adam Foster, Wenbo Gong, Chao Ma, Emre Kiciman, Amit Sharma, Angus Lamb, Martin Kukla, Agrin Hilmkil, Joel Jennings, Nick Pawlowski, Miltiadis Allamanis, Cheng Zhang
NeurIPS 2022 Workshop on Causal Machine Learning for Real-World Impact (CML4Impact 2022) | November 2022
-
Rhino: Deep Causal Temporal Relationship Learning with history-dependent noise
Wenbo Gong, Joel Jennings, Cheng Zhang, Nick Pawlowski
NeurIPS 2022 Workshop on Causal Machine Learning for Real-World Impact (CML4Impact 2022) | November 2022
-
Simultaneous Missing Value Imputation and Structure Learning with Groups
Pablo Morales-Alvarez, Wenbo Gong, Angus Lamb, Simon Woodhead, Simon Peyton Jones, Nick Pawlowski, Miltos Allamanis, Cheng Zhang
2022 Neural Information Processing Systems | February 2022
-
-
-
Icebreaker: Element-wise Active Information Acquisition with Bayesian Deep Latent Gaussian Model
Wenbo Gong, Sebastian Tschiatschek, Richard Turner, Sebastian Nowozin, José Miguel Hernández-Lobato, Cheng Zhang
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | August 2019
-
-
-
SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
Chao Ma, Wenbo Gong, Meyer Scetbon, Edward Meeds
ICML 2025 | February 2025
-
Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
Wenbo Gong, Meyer Scetbon, Chao Ma, Edward Meeds
ArXiv | February 2025
-
Gradient Multi-Normalization for Stateless and Scalable LLM Training
Meyer Scetbon, Chao Ma, Wenbo Gong, Edward Meeds
ArXiv | February 2025
-
Neural Structure Learning with Stochastic Differential Equations
Benjie Wang, Joel Jennings, Wenbo Gong
ICLR 2024 | November 2023
-
Deep End-to-end Causal Inference
Tomas Geffner, Javier Antoran, Adam Foster, Wenbo Gong, Chao Ma, Emre Kiciman, Amit Sharma, Angus Lamb, Martin Kukla, Agrin Hilmkil, Joel Jennings, Nick Pawlowski, Miltiadis Allamanis, Cheng Zhang
NeurIPS 2022 Workshop on Causal Machine Learning for Real-World Impact (CML4Impact 2022) | November 2022
-
Rhino: Deep Causal Temporal Relationship Learning with history-dependent noise
Wenbo Gong, Joel Jennings, Cheng Zhang, Nick Pawlowski
NeurIPS 2022 Workshop on Causal Machine Learning for Real-World Impact (CML4Impact 2022) | November 2022
-
Simultaneous Missing Value Imputation and Structure Learning with Groups
Pablo Morales-Alvarez, Wenbo Gong, Angus Lamb, Simon Woodhead, Simon Peyton Jones, Nick Pawlowski, Miltos Allamanis, Cheng Zhang
2022 Neural Information Processing Systems | February 2022
-
Icebreaker: Element-wise Active Information Acquisition with Bayesian Deep Latent Gaussian Model
Wenbo Gong, Sebastian Tschiatschek, Richard Turner, Sebastian Nowozin, José Miguel Hernández-Lobato, Cheng Zhang
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | August 2019
-
-
-
Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
Wenbo Gong, Meyer Scetbon, Chao Ma, Edward Meeds
ArXiv | February 2025
-
Gradient Multi-Normalization for Stateless and Scalable LLM Training
Meyer Scetbon, Chao Ma, Wenbo Gong, Edward Meeds
ArXiv | February 2025
-
-
-
SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training
Chao Ma, Wenbo Gong, Meyer Scetbon, Edward Meeds
ICML 2025 | February 2025
-
Neural Structure Learning with Stochastic Differential Equations
Benjie Wang, Joel Jennings, Wenbo Gong
ICLR 2024 | November 2023
-
Deep End-to-end Causal Inference
Tomas Geffner, Javier Antoran, Adam Foster, Wenbo Gong, Chao Ma, Emre Kiciman, Amit Sharma, Angus Lamb, Martin Kukla, Agrin Hilmkil, Joel Jennings, Nick Pawlowski, Miltiadis Allamanis, Cheng Zhang
NeurIPS 2022 Workshop on Causal Machine Learning for Real-World Impact (CML4Impact 2022) | November 2022
-
Rhino: Deep Causal Temporal Relationship Learning with history-dependent noise
Wenbo Gong, Joel Jennings, Cheng Zhang, Nick Pawlowski
NeurIPS 2022 Workshop on Causal Machine Learning for Real-World Impact (CML4Impact 2022) | November 2022
-
Simultaneous Missing Value Imputation and Structure Learning with Groups
Pablo Morales-Alvarez, Wenbo Gong, Angus Lamb, Simon Woodhead, Simon Peyton Jones, Nick Pawlowski, Miltos Allamanis, Cheng Zhang
2022 Neural Information Processing Systems | February 2022
-
Icebreaker: Element-wise Active Information Acquisition with Bayesian Deep Latent Gaussian Model
Wenbo Gong, Sebastian Tschiatschek, Richard Turner, Sebastian Nowozin, José Miguel Hernández-Lobato, Cheng Zhang
Thirty-third Conference on Neural Information Processing Systems (NeurIPS) | August 2019
-