CoStar Group breaks ground with Azure AI Foundry and GPT-4o Realtime API audio models

Searching for a home online has been stuck in the filter-search paradigm for decades. “Our business is real estate data,” says Andy Florance, CEO of CoStar Group, which owns homes.com, apartments.com, and many other real estate marketplaces. “We provide proprietary data and insights—including market research and neighborhood details—across commercial and residential real estate. No other business can offer what we have. There aren’t enough filters in the traditional search matrix to capture the nuance we can offer a home buyer in their search.”

Seeking insightful, nuanced conversation

CoStar Group (CoStar) is rethinking the home search experience on its homes.com marketplace. It needs a solution that replicates the casual, nuanced conversations that buyers have with human real estate agents, to surface its vast amount of relevant data.

“When you talk to your human real estate agent, you don’t say, ‘Three bedroom, two bath, under $700,000, Wilmington area,’” notes Livia Sponseller, VP of Product Management at homes.com. “You have a conversation. There’s back-and-forth. Your real estate agent offers insights. It’s a richer, more nuanced and thoughtful experience. That’s what we want for our visitors to homes.com.” A conversational search experience helps home buyers be more specific about their priorities, yielding results that more closely align with their desires. This, in turn, means a home buyer is more likely to find the home that fits.

A real-time audio search experience seemed the perfect solution. But real-time audio presents latency issues, where the pause between the human speaking and the AI response is too long. CoStar knew the solution would have to be extremely fast to draw people in.

Prioritizing security and scalability

Two other considerations drove CoStar’s planning. First, the solution needs to offer a secure environment to help protect both the proprietary data owned by CoStar and consumer conversations. Second, the solution needs to scale as CoStar expands domestically and globally. Already, more than 100 million people come to homes.com to search for homes every month. With a premier search experience, that number is expected to grow. In addition, introducing a new technology to an existing website requires careful consideration to help ensure every visitor’s experience remains positive.

Succeeding with real-time audio AI

CoStar looked for a speech and audio API that would deliver low-latency performance, scalability, and reliability. The GPT-4o Realtime API audio models on Microsoft Azure OpenAI in Azure AI Foundry Models exceeded expectations. “The speed of the GPT-4o Realtime API is just incredible,” observes Andy Ventura, VP. “The ability to interrupt —it’s awesome.”

Sponseller agrees, adding, “Every time we interact with the voice tech, it feels magical. It’s exceeding our original expectations and vision, making the home buying process more engaging and efficient. The detail we can offer through this experience, from data on the heat index in a specific region to showing a view out the kitchen window, is mind-blowing.”

Discovering the right AI model

CoStar was able to experiment efficiently and cost-effectively with the generative AI models in Azure AI Foundry to find the most suitable fit for a real-time audio search solution for home buyers. Identifying the right model was crucial for addressing the requirement for a low-latency experience.

“One of the key advantages of choosing Microsoft was access to Azure AI Foundry Models. We were able to easily and affordably experiment with multiple models before deciding on the best low-latency model for our use case,” says Ventura. The ability to test and refine different models was instrumental in achieving the desired performance and efficiency.

CoStar chose Mistral 3B for natural language processing because it is optimized for low-latency, high-volume workloads. Since the company didn’t require a large model, Mistral 3B met the need with fewer parameters, making it faster to fine-tune and deploy. It also requires less computational power and memory, making it a strong choice for applications drawing on limited resources, such as mobile devices.

Because of these specifications, Mistral 3B uses significantly fewer tokens for each search query (for example, 100 tokens versus 4,000 with other models for an initial query such as “Show me homes in the Michigan area that are on a lake”). As a result, it’s faster and more cost-efficient. Despite its smaller size, Mistral 3B delivers the uptime, scale, and governance capabilities needed, thanks to its interoperability with Azure.

Seeing the whole picture

With the real-time search and navigation experience, a visitor to homes.com can say, “Show me homes within 25 miles of downtown Denver that are in the top school district.” Then they might ask, “Which of these have a big backyard?” and continue to refine until finally saying, “Show me the view out of the kitchen window on this one.” Home buyers can virtually step inside the home, thanks to CoStar’s recent acquisition of Matterport, the first 3D capture solution to deliver dimensionally accurate, photorealistic virtual tours or “digital twins” for any type of property. When they ask, “Can we fit a king-size bed in this room?” the combination of Matterport and the AI audio experience delivers an accurate, immediate response.

CoStar is unlocking access to insights that were previously undiscoverable. It’s doing this in a search experience that feels natural and intuitive with the combination of Mistral 3B and GPT-4o Realtime API audio models. Using Azure AI Foundry, the team can easily and seamlessly combine both Mistral and GPT-4o Realtime API audio models.

Meeting the need for security and scalability

A key reason that CoStar turned to Microsoft was for its robust security and compliance features. Azure AI Foundry provides advanced threat protection and regulatory compliance frameworks, helping ensure the highest standards of data protection and operational integrity. “Protecting our proprietary data is how we protect our business value. It’s no small thing that Microsoft is helping us achieve this,” says Florance.

Safeguarding customer conversations and maintaining compliance with data collection and disclosure laws is automatically managed with Azure AI Foundry.

Azure AI Foundry also offers autoscale features that automatically adjust resource limits based on real-time usage metrics. This helps guarantee that CoStar’s conversational AI solution can handle varying loads efficiently without manual intervention.

Preparing for what’s next

CoStar’s innovative real-time audio search sets it up for an exciting future. With the heavy lift of initial development complete, the company is poised to implement this solution across its other marketplaces domestically and globally. The success of the project is expected to drive more traffic to homes.com, ultimately increasing revenue.

Committing to exploration

CoStar aims to continue enhancing the buyer experience, making it even more personalized and intuitive. The foundational work done with Azure AI Foundry and the interoperability of advanced models like Mistral and Azure OpenAI GPT-4o Realtime API sets the stage for future innovations.

Florance concludes, “With Azure AI Foundry, we’re exploring the latest AI has to offer. Our data is highly secure, we can scale for millions of visitors, and in the end, we’re radically improving how people search for—and find—a place to call home."

Discover more about CoStar Group on Facebook, Instagram, LinkedIn, X/Twitter, and YouTube.