Emily Sheng

Senior Applied Scientist

Publications

View by:

- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023, Vol abs/2310.17750
  
  DOI Publication Publication

- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023, Vol abs/2310.17750
  
  DOI Publication Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023, Vol abs/2310.17750
  
  DOI Publication Publication

- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  Publication
- A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications
  
  Ahmed Magooda, Alec Helyar, Kyle Jackson, David Sullivan, Chad Atalla, Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird, Mei Chen
  
  ArXiv | October 2023, Vol abs/2310.17750
  
  DOI Publication Publication

Emily Sheng

Publications

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Taxonomizing Representational Harms using Speech Act Theory

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Taxonomizing Representational Harms using Speech Act Theory

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

Taxonomizing Representational Harms using Speech Act Theory

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applications

Contact Emily Sheng

Microsoft Research Lab – New York City