Belief Dynamics in Web Search

People frequently answer consequential questions, such as those with a medical focus, using Internet search engines. Their primary goal in performing these searches is to revise or establish beliefs associated with one or more outcomes. Search engines are not designed to furnish answers, and instead provide a ranked list of documents, some of which contain answers. People’s prior beliefs about answer likelihoods can affect result selection decisions, and when these selections are aggregated across searchers they can skew machine-learned result rankings. Research on information retrieval has targeted key aspects of information access such as query formulation, result relevance, and search success. However, there are important unanswered questions surrounding how beliefs—and potential biases in those beliefs—affect search behaviors and how beliefs are shaped by search engine use. In this article, we report on a study examining changes, or dynamics, in beliefs during Web search. To understand belief dynamics, we focus on a balanced set of yes-no medical questions (e.g., “is congestive heart failure a heart attack?”), with consensus answers from physicians.

We show that:

1. pre-search beliefs are affected only slightly by searching and any changes in belief are more likely to move toward positive (yes);
2. pre-search beliefs affect search behavior;
3. search engines can shift some beliefs by manipulating the relative ordering and availability of answers in the results, but strongly-held beliefs are both difficult to move using uncongenial information and can be counterproductive, and;
4. search engines may exhibit near-random answer accuracy, but they may be able to help searchers attain higher accuracy if engines consider factual correctness.

Among other things, our findings suggest that search engines must attain, represent, and utilize correct answers in result ranking, and develop methods to encourage searchers to shift strongly-held but factually-incorrect beliefs.