DEPT brings Sinterklaas to life for holiday retailer using Azure AI Foundry

Retailers face the constant challenge of catching customer attention amidst a sea of competing offers and promotions. Joost Eerbeek, Digital Project Manager at DEPT, a global marketing and tech services company, says the holiday season is the most demanding. “With so many shopping options and advertisements, it's crucial for retailers to create distinctive experiences for their target customers,” says Eerbeek.

When bol, a prominent Dutch online marketplace, turned to DEPT to amplify its holiday presence—specifically, the annual publication of The Big Toy Book, its toy catalog for children—the agency saw a unique opportunity to help the retailer stand out.

“Starting around October, children start getting excited about Sinterklaas, the Dutch equivalent of Santa Claus,” explains Wesley Niels, Director, Cloud EMEA and Partner at DEPT. “We wanted to play with that magic by digitizing the toy book into an app, giving children a way to interact with their favorite toys and make a wish list, while also ‘talking’ with Sinterklaas.”

Security and responsible AI

With the safekeeping of children top of mind, DEPT chose to work with Microsoft for its leadership in security, AI, and responsible AI practices. “With Microsoft technology, we knew we could create personalized experiences for kids while ensuring safe, ethical use,” adds Rick van Sintmaartensdijk, Team Lead, Data Science and AI at DEPT.

The first step for DEPT was to digitize The Big Toy Book and incorporate features like augmented reality. DEPT built the app on Microsoft Azure with a serverless and scalable architecture, using Application Insights, Azure SQL databases, Azure Managed Redis, Azure Key Vault, and the Web Apps feature of Azure App Service. “With the app, children can scan pictures of toys from the printed book and play with them in 3D on their own (or a parent’s) device,” explains Niels.

Upleveling with AI and Custom Neural Voice

Bol already had a Sinterklaas actor for its video and radio advertisements, and DEPT was excited to see if AI could replicate the actor’s voice. “We didn’t want the synthetic voice to sound robotic—it needed to be warm and engaging,” says van Sintmaartensdijk.

The DEPT team used Azure OpenAI in Azure AI Foundry to create script components to fit requirements for training a relevant custom neural voice model. “Guided by Microsoft, we used the latest GPT-series models to create the necessary training data for the synthetic voice,” explains van Sintmaartensdijk. ”Members of our team were the ‘training’ voice and generated a lot of different kinds of words and sounds at the right length.” 

The team worked with the Sinterklaas actor to record around 500 utterances in both English and Dutch in a professional recording studio. Custom Neural Voice within Azure AI Speech helped the team create a synthetic voice that closely resembled the actor’s authentic voice. 

Using Azure AI Foundry, DEPT tested and refined the voice messages. “With Azure AI Foundry, we could quickly check how the messages sounded and make necessary adjustments. In addition, different team members, including copywriters and developers, were able to collaborate within the tool, so we could create and make changes rapidly,” says van Sintmaartensdijk.

The team used blocklists and phonetic fixes to help prevent inappropriate words from being used and ensured that names and other words were pronounced correctly. “In the end, we were blown away by how real the voice sounded,” says Niels.

“We built the Sinterklaas app in six weeks. Azure AI Foundry solutions helped us create a scalable, immersive experience without sacrificing quality, privacy, or performance,” adds Eerbeek.

Unleashing a flurry of engagement

Parental consent and participation are required for a child to use the app. A dashboard in the app lets parents see their children’s wish lists and add details unique to their child to further personalize Sinterklaas’s messages. “We know from customer feedback that AI text-to-speech gave kids the feeling that Sinterklaas truly knows them,” says Eerbeek. “AI text-to-speech alone triggered 1 million messages from Sinterklaas, resulting in more return visits to the app and more wish lists.”

During the 2024 season, the app had 300,000 total users, who engaged with Sinterklaas more than 3 million times. This contributed to a 233% improvement in engagement over the previous year’s holiday campaign. 

“AI and Microsoft helped us set a new standard for interactive marketing campaigns, and we’re already building on it for next season. It’s blown open the creative door for what’s possible for retailers to connect with customers in unique ways,” concludes van Sintmaartensdijk.  

Discover more about DEPT on Instagram and LinkedIn.