Learning to Account for Good Abandonment in Search Success Metrics - Khabsa, Madian; Crook, Aidan; Awadallah, Ahmed; Zitouni, Imed; Anastasakos, Tasos; Williams, Kyle (Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM 2016), 2016)
Abandonment in web search has been widely used as a proxy to measure user satisfaction. Initially it was considered a signal of dissatisfaction, however with search engines mov- ing towards providing answer-like results, a new category of abandonment was introduced and referred to as Good Aban- donment. Predicting good abandonment is a hard problem and it was the subject of several previous studies. All those studies have focused, though, on predicting good abandon- ment in oine settings using manually labeled data. Thus, it remained a challenge how to have an online metric that accounts for good abandonment. In this work we describe how a search success metric can be augmented to account for good abandonment sessions using a machine learned metric that depends on user's viewport information. We use real user trac from millions of users to evaluate the proposed metric in an A/B experiment. We show that taking good abandonment into consideration has a signicant eect on the overall performance of the online metric.