Sandia saves employees countless hours a month with a secure AI chat tool built on Azure OpenAI in Foundry Models

To improve efficiency across its organization, Sandia National Laboratories (Sandia) wanted to develop an AI solution that would help ensure security and could be smoothly incorporated into existing security protocols.

Sandia used Microsoft Azure and Azure OpenAI in Foundry Models to develop a custom chat tool that is built on large language models but without an open connection that could compromise Sandia’s security.

Within eight months, Sandia deployed the solution for nearly 17,000 employees. Thus, the laboratory improved security and productivity, advanced data analysis, and helped employees access information more easily and quickly.

Sandia National Laboratories (Sandia), a federally funded research and development center serving the US government, is using AI tools to assist in research, manage costs, and improve workplace efficiency. Sandia used Microsoft Azure and Azure OpenAI in Foundry Models to develop a solution that would help employees streamline research and business processes while adhering to strict security requirements.

“We wanted the value of popular AI models, but we wanted to deploy that value in a very secure Azure environment. With this tool, we could take a snapshot of each new model released and give that to our employees without having to actually connect to a public model.”

John Zepper, Information Engineering Executive Director and Chief Information Officer, Sandia National Laboratories

Empowering federal labs with the latest innovations in a secure environment

Sandia has tackled complex national security challenges for more than 75 years. As Sandia continues to solve the nation’s toughest science and engineering problems, it also works to identify cost savings and enhance operational efficiency.

“One of our big initiatives at Sandia right now is innovating and tracking savings or efficiencies associated with that innovation,” says John Zepper, Information Engineering Executive Director and Chief Information Officer at Sandia. “The second thing is improving our digital engineering practices.”

To explore AI applications within its internal operations, Sandia formed dedicated teams to study machine learning and advanced analytics. In 2022, these teams began evaluating AI-based solutions to improve Sandia’s efficiency while adhering to government security requirements.

Creating a secure AI-powered solution to enhance efficiency for nearly 17,000 employees

In October 2023, Sandia collaborated with Microsoft to develop SandiaAI Chat, an AI-powered tool designed to allow employees to work with sensitive, unclassified information within a separate, secure environment.

Using Azure OpenAI in Foundry Models, Sandia implemented a generative AI solution within a controlled Azure ecosystem. “We wanted the value of popular AI models, but we wanted to deploy that value in a very secure Azure environment,” says Zepper. “With this tool, we could take a snapshot of each new model released and give that to our employees without having to actually connect to a public model.”

SandiaAI Chat complies with the AI Risk Management Framework of the National Institute of Standards and Technology and is authorized under the Federal Risk and Authorization Management Program. Workers can use the tool to generate code in several common programming languages, automate workflows, and compose clearer emails, all while ensuring sensitive data never leaves Sandia control.

“We’re going to be making it possible to bring data into the models. That way, someone doing analysis can input a file or piece of data and ask the model to summarize it, find specific things within the data, or even predict things that might happen within some of the different parts we maintain in our business.”

John Zepper, Information Engineering Executive Director and Chief Information Officer, Sandia National Laboratories

Saving three minutes per question and countless hours a monthwith AI chat tools

By May 2024, Sandia had deployed the AI tool for use by nearly 17,000 employees. These workers have been using AI in innovative ways, such as generating and refining code, answering queries, and visualizing data with Microsoft Power BI even without previous experience.In the first month after SandiaAI Chat was deployed, employees asked more than 106,000 questions, finding answers quicker than they had using previous processes.

“We looked at those using the system in something analogous to an online search,” says Brian Post, Senior Manager and Deputy Chief Data Officer at Sandia. “In that process, you type something into a search engine, you get links, and you curate the links—that takes about three minutes. With SandiaAI Chat, users can condense that three-minute process into a nearly instantaneous one. Multiplied across thousands of users, that’s significant time savings.”

Employees also use SandiaAI Chat to analyze and summarize data, aiding in code development and research processes. “We’re going to be making it possible to bring data into the models,” says Zepper. “That way, someone doing analysis can input a file or piece of data and ask the model to summarize it, find specific things within the data, or even predict things that might happen within some of the different parts we maintain in our business.”

Not only is AI making it possible for employees to get work done faster, but it’s also helping them expand their own knowledge and increase their merit in the company. “During performance reviews, for example, staff members are using this to show their value,” says Zepper. “I used it myself and found the review was better written than I could have done on my own. Workers can use this to draft and then format how they want to tell their story.”

“Everybody wants to improve or innovate; they just don’t know how with complicated security requirements. But we’ve been sharing how we implemented this in a secure and scalable way so that they can do it too.”

John Zepper, Information Engineering Executive Director and Chief Information Officer, Sandia National Laboratories

Expanding the possibilities of secure innovation using AI

Sandia is leading the way in adopting AI capabilities in agencies where security is paramount. It is collaborating with other government agencies to use AI tools in a similar secure fashion, evaluating AI applications and sharing insights with interested agencies.

“We are getting calls from the military asking how we did this,” says Zepper. “We are helping colleagues in the Department of Defense and other government agencies mimic what we’ve done but with their own data.”

Sandia has shared its experience with organizations seeking to employ AI within strict safety frameworks. “Everybody wants to improve or innovate; they just don’t know how with complicated security requirements,” says Zepper. “But we’ve been sharing how we implemented this in a secure and scalable way so that they can do it too.”

Discover more about Sandia on Facebook, Instagram, LinkedIn, X/Twitter, and YouTube.