Gen AI in your browser
Foyer’s product, Merlin, is the GPTenabled extension for the browser with multiple integrations for sites such as YouTube, Outlook, Gmail, Amazon. However, Foyer wanted to ensure that the extension can be used across regions, with better reliability and more speed.
Using Azure OpenAI, Foyer was able to ensure it can enhance the core features of Merlin and ship out a feature-rich GPT enabled extension across the globe.
Serving 1 million+ users
Foyer used Azure OpenAI service to deploy GPT-3.5 and GPT-4 family of models to serve its 1 million+ Merlin users reliably through multi-region deployments (as compared to US-only servers through OpenAI API). This also enabled on-demand scaling to ensure zero downtime even when OpenAI’s API might be facing any service outage. The extensive monitoring capabilities built in Azure Log Analytics were also used to minimise application errors.
Azure OpenAI service had better reliability in terms of service, better incident reporting if and when there is an outage and potential solutions to mitigate rare outages. Azure OpenAI service reduced the response generation latency and AI completion speed by about 70-80% as measured in real-world applications. Better logging and alerting through Azure Log Analytics helped mitigate any logic errors missed during deployments.
Pratyush Rai
Co-founder and CEO
Foyer
Microsoft has been a fabulous partner that has helped us scale our services to millions of users. The latency and reliability that they can provide at scale, backed by exceptional engineering and support is unmatched across the industry.
70%-80%
reduction in response generation latency
Merlin is an all-in-one AI extension to write, summarize, code & play.
More details at https://www.getmerlin.in
Firstsource