Square Enix uses Azure OpenAI Service for AI-enhanced game development

Generative AI streamlines support for game engines

A vast audience worldwide delights in the continuous release of new video games. In recent years, most games depict three-dimensional worlds and characters and boast increasingly realistic physics. Game engines provide functions to help developers make such games. This includes creating and managing the 3D assets of game characters and worlds; input processing and animation control for the movement of 3D assets; graphics functions to generate visual effects; and physics, sound effects, and music.

Square Enix, producer of some of the world’s biggest series, such as Final Fantasy and Dragon Quest, has infused its game engine with generative AI.

“We develop some games using commercial game engines, but generally use our in-house engine,” says Takeshi Aramaki, former Head of Luminous Productions and current General Manager of the Square Enix AI and Engine Development Division. According to Aramaki, game engines are essential for game development, and developers must understand them to create games.

“However, game engines have a vast range of features and their documents can exceed 6,000 pages,” Aramaki continues. “No developer reads the entire document. Our AI and Engine Development Division, which creates our engines, receives countless questions every day. But psychological hurdles prevent some workers from asking questions, even when they’re stumped.”

While too many questions can increase the workloads of game engine staff, having developers reluctant to ask questions hampers development productivity. To solve both problems, Aramaki turned his attention to a generative AI-based chatbot following the excitement generated by ChatGPT’s appearance in November 2022.

“At first, we thought chat would have little application to game development,” says Aramaki. “But the release of GPT-4 in July 2023 completely changed the situation. Generative AI could now create programs and data.”

Aramaki's team began considering the integration of generative AI and game engines. For its first step, the team created a chat function to answer questions from developers. They named the chatbot “Hisui-chan” and integrated it with Slack, choosing Azure OpenAI Service as generative AI. Arakami explains the reasoning below.

“For now, the cloud is the only option for powerful generative AI, but we were concerned about data security on overseas servers. However, Azure avoids this problem with servers in Japan. Another key aspect was Microsoft's assurances against using customer data for learning without permission and delivering ethically inappropriate answers.”

Creating an AI chatbot using GPT-4 in just three days

Hisui-chan consists of an event handling program running on Azure Functions, Microsoft Azure Blob for storing documents and question content, Azure AI Translator for detecting and translating languages, Azure AI Search for searching documents on blobs, and Azure OpenAI Service (GPT-4) for generating answers. When a question is posted to Slack, the Slack App Event API notifies Azure Functions, triggering retrieval-augmented generation (RAG) on content found with Azure AI Search, and Azure OpenAI Service generates an answer and posts it to Slack.

“Slack’s event-handling program uses Python code written by Microsoft Copilot,” explains Aramaki. “We configured the system and did everything else in-house, completing the project in just three days.”

Hisui-chan’s character is a young child. According to Aramaki, this is so users can laugh off any wrong answers given by the AI while minimizing resistance to answers coming from AI rather than humans. The team also chose Hisui-chan’s Slack icon from multiple Copilot-generated candidates.

The team released Hisui-chan in February 2024. Although Aramaki initially predicted there would be a lot of wrong answers, the intelligent responses surpassed expectations.

“As well as providing answers from entire documents, Hisui-chan also learns from past Slack discussions, so she sometimes gives surprisingly good answers that aren’t in the documentation,” says Aramaki. “When Hisui-chan answers a question, the system sends a mention to people in charge of the relevant part of the game engine, so if there’s a problem with the answer, they can add a comment, which Hisui-chan learns from continuously.”

”One of Hisui-chan’s best features is that she chats with many people, unlike a typical one-on-one chat,” says Teppei Ono, a senior programmer at Square Enix's AI and Engine Development Division. “This enables others, not just the asker, to see the questions and answers, which lowers the barriers to asking questions.” By having Hisui-chan tackle questions first and enhancing her precision over time, game engine developers experience lighter workloads. 

Hisui-chan’s tone, which is child-like and friendly, is another noteworthy aspect.

“Her tone is surprisingly consistent across multiple conversations, even with long prompts,” says Aramaki. Most generative AI tends to forget previous prompts over time, he continues. “GPT-4 is the only current solution that remembers past prompts. GPT-4’s outstanding capabilities have impressed us since we first began researching it, and our opinion remains unchanged.”

Aramaki’s team began working on a solution to auto-generate the various data necessary for game development. Developers use Square-Enix’s in-house engine to create and manage game data in a unique JSON-based format. With Aramaki’s solution, they can use Azure OpenAI Service to automatically generate Python code to generate that data.

“We started this initiative by testing two approaches: generating data directly with GPT versus generating Python code to generate data,” says Ono. The team opted for the latter approach for two reasons. “First, unlike our proprietary data format, there are countless samples of Python code out there from which we can get an acceptable level of learning. Second, directly generating hierarchical JSON data is less accurate than generating code to generate that data,” he says.

Square Enix incorporated the results into its game engine in June 2024. Ono explains the outcomes of Hisui-chan’s first four months of use below.

“With generative AI incorporated into game engines, developers can efficiently get answers on data creation. And non-programmers can easily create game data because the system auto-generates Python code for generating data. Developers can instantly check how that data is actually presented in games and expand their mindsets by observing data they rarely use.”

According to Ono, new developers often use Hisui-chan instead of tutorials. Hisui-chan generates authentic-looking data, so it’s easy for them to take their first steps in data creation.