The distorting effects of producer strategies: Why engagement does not reveal consumer preferences for misinformation.

A great deal of empirical research has examined who falls for misinformation and why. Here, we introduce a formal game-theoretic model of engagement with news stories that captures the strategic interplay between (mis)information consumers and producers. A key insight from the model is that observed patterns of engagement do not necessarily reflect the preferences of consumers. This is because producers seeking to promote misinformation can use strategies that lead moderately inattentive readers to engage more with false stories than true ones-even when readers prefer more accurate over less accurate information. We then empirically test people's preferences for accuracy in the news. In three studies, we find that people strongly prefer to click and share news they perceive as more accurate-both in a general population sample, and in a sample of users recruited through Twitter who had actually shared links to misinformation sites online. Despite this preference for accurate news-and consistent with the predictions of our model-we find markedly different engagement patterns for articles from misinformation versus mainstream news sites. Using 1,000 headlines from 20 misinformation and 20 mainstream news sites, we compare Facebook engagement data with 20,000 accuracy ratings collected in a survey experiment. Engagement with a headline is negatively correlated with perceived accuracy for misinformation sites, but positively correlated with perceived accuracy for mainstream sites. Taken together, these theoretical and empirical results suggest that consumer preferences cannot be straightforwardly inferred from empirical patterns of engagement.

Understanding and combatting misinformation across 16 countries on six continents.

The spread of misinformation online is a global problem that requires global solutions. To that end, we conducted an experiment in 16 countries across 6 continents (N = 34,286; 676,605 observations) to investigate predictors of susceptibility to misinformation about COVID-19, and interventions to combat the spread of this misinformation. In every country, participants with a more analytic cognitive style and stronger accuracy-related motivations were better at discerning truth from falsehood; valuing democracy was also associated with greater truth discernment, whereas endorsement of individual responsibility over government support was negatively associated with truth discernment in most countries. Subtly prompting people to think about accuracy had a generally positive effect on the veracity of news that people were willing to share across countries, as did minimal digital literacy tips. Finally, aggregating the ratings of our non-expert participants was able to differentiate true from false headlines with high accuracy in all countries via the 'wisdom of crowds'. The consistent patterns we observe suggest that the psychological factors underlying the misinformation challenge are similar across different regional settings, and that similar solutions may be broadly effective.

Scaling up fact-checking using the wisdom of crowds.

Professional fact-checking, a prominent approach to combating misinformation, does not scale easily. Furthermore, some distrust fact-checkers because of alleged liberal bias. We explore a solution to these problems: using politically balanced groups of laypeople to identify misinformation at scale. Examining 207 news articles flagged for fact-checking by Facebook algorithms, we compare accuracy ratings of three professional fact-checkers who researched each article to those of 1128 Americans from Amazon Mechanical Turk who rated each article's headline and lede. The average ratings of small, politically balanced crowds of laypeople (i) correlate with the average fact-checker ratings as well as the fact-checkers' ratings correlate with each other and (ii) predict whether the majority of fact-checkers rated a headline as "true" with high accuracy. Furthermore, cognitive reflection, political knowledge, and Democratic Party preference are positively related to agreement with fact-checkers, and identifying each headline's publisher leads to a small increase in agreement with fact-checkers.

Turking in the time of COVID.

On March 16, 2020, the US Government introduced strict social distancing protocols for the United States in an effort to stem the spread of the COVID-19 pandemic. This had an immediate major effect on the job market, with millions of Americans forced to find alternative ways to make a living from home. As online labor markets like Amazon Mechanical Turk (MTurk) play a major role in social science research, concerns have been raised that the pandemic may be reducing the diversity of subjects participating in experiments. Here, we investigate this possibility empirically. Specifically, we look at 15,539 responses gathered in 23 studies run on MTurk between February and July 2020, examining the distribution of gender, age, ethnicity, political preference, and analytic cognitive style. We find notable changes on some of the measures following the imposition of nationwide social distancing: participants are more likely to be less reflective (as measured by the Cognitive Reflection Test), and somewhat less likely to be white, Democrats (traditionally over-represented on MTurk), and experienced with MTurk. Most of these differences are explained by an influx of new participants into the MTurk subject pool who are more diverse and representative - but also less attentive - than previous MTurkers.

Shifting attention to accuracy can reduce misinformation online.

In recent years, there has been a great deal of concern about the proliferation of false and misleading news on social media1-4. Academics and practitioners alike have asked why people share such misinformation, and sought solutions to reduce the sharing of misinformation5-7. Here, we attempt to address both of these questions. First, we find that the veracity of headlines has little effect on sharing intentions, despite having a large effect on judgments of accuracy. This dissociation suggests that sharing does not necessarily indicate belief. Nonetheless, most participants say it is important to share only accurate news. To shed light on this apparent contradiction, we carried out four survey experiments and a field experiment on Twitter; the results show that subtly shifting attention to accuracy increases the quality of news that people subsequently share. Together with additional computational analyses, these findings indicate that people often share misinformation because their attention is focused on factors other than accuracy-and therefore they fail to implement a strongly held preference for accurate sharing. Our results challenge the popular claim that people value partisanship over accuracy8,9, and provide evidence for scalable attention-based interventions that social media platforms could easily implement to counter misinformation online.

Cognitive reflection correlates with behavior on Twitter.

We investigate the relationship between individual differences in cognitive reflection and behavior on the social media platform Twitter, using a convenience sample of N = 1,901 individuals from Prolific. We find that people who score higher on the Cognitive Reflection Test-a widely used measure of reflective thinking-were more discerning in their social media use, as evidenced by the types and number of accounts followed, and by the reliability of the news sources they shared. Furthermore, a network analysis indicates that the phenomenon of echo chambers, in which discourse is more likely with like-minded others, is not limited to politics: people who scored lower in cognitive reflection tended to follow a set of accounts which are avoided by people who scored higher in cognitive reflection. Our results help to illuminate the drivers of behavior on social media platforms and challenge intuitionist notions that reflective thinking is unimportant for everyday judgment and decision-making.

Information gerrymandering and undemocratic decisions.

People must integrate disparate sources of information when making decisions, especially in social contexts. But information does not always flow freely. It can be constrained by social networks1-3 and distorted by zealots and automated bots4. Here we develop a voter game as a model system to study information flow in collective decisions. Players are assigned to competing groups (parties) and placed on an 'influence network' that determines whose voting intentions each player can observe. Players are incentivized to vote according to partisan interest, but also to coordinate their vote with the entire group. Our mathematical analysis uncovers a phenomenon that we call information gerrymandering: the structure of the influence network can sway the vote outcome towards one party, even when both parties have equal sizes and each player has the same influence. A small number of zealots, when strategically placed on the influence network, can also induce information gerrymandering and thereby bias vote outcomes. We confirm the predicted effects of information gerrymandering in social network experiments with n = 2,520 human subjects. Furthermore, we identify extensive information gerrymandering in real-world influence networks, including online political discussions leading up to the US federal elections, and in historical patterns of bill co-sponsorship in the US Congress and European legislatures. Our analysis provides an account of the vulnerabilities of collective decision-making to systematic distortion by restricted information flow. Our analysis also highlights a group-level social dilemma: information gerrymandering can enable one party to sway decisions in its favour, but when multiple parties engage in gerrymandering the group loses its ability to reach consensus and remains trapped in deadlock.

Examining Spillovers between Long and Short Repeated Prisoner's Dilemma Games Played in the Laboratory.

We had participants play two sets of repeated Prisoner's Dilemma (RPD) games, one with a large continuation probability and the other with a small continuation probability, as well as Dictator Games (DGs) before and after the RPDs. We find that, regardless of which is RPD set is played first, participants typically cooperate when the continuation probability is large and defect when the continuation probability is small. However, there is an asymmetry in behavior when transitioning from one continuation probability to the other. When switching from large to small, transient higher levels of cooperation are observed in the early games of the small continuation set. Conversely, when switching from small to large, cooperation is immediately high in the first game of the large continuation set. We also observe that response times increase when transitioning between sets of RPDs, except for altruistic participants transitioning into the set of RPDs with long continuation probabilities. These asymmetries suggest a bias in favor of cooperation. Finally, we examine the link between altruism and RPD play. We find that small continuation probability RPD play is correlated with giving in DGs played before and after the RPDs, whereas high continuation probability RPD play is not.

Conducting interactive experiments online.

Online labor markets provide new opportunities for behavioral research, but conducting economic experiments online raises important methodological challenges. This particularly holds for interactive designs. In this paper, we provide a methodological discussion of the similarities and differences between interactive experiments conducted in the laboratory and online. To this end, we conduct a repeated public goods experiment with and without punishment using samples from the laboratory and the online platform Amazon Mechanical Turk. We chose to replicate this experiment because it is long and logistically complex. It therefore provides a good case study for discussing the methodological and practical challenges of online interactive experimentation. We find that basic behavioral patterns of cooperation and punishment in the laboratory are replicable online. The most important challenge of online interactive experiments is participant dropout. We discuss measures for reducing dropout and show that, for our case study, dropouts are exogenous to the experiment. We conclude that data quality for interactive experiments via the Internet is adequate and reliable, making online interactive experimentation a potentially valuable complement to laboratory studies.

Turking Overtime: How Participant Characteristics and Behavior Vary Over Time and Day on Amazon Mechanical Turk.

Online experiments allow researchers to collect datasets at times not typical of laboratory studies. We recruit 2,336 participants from Amazon Mechanical Turk to examine if participant characteristics and behaviors differ depending on whether the experiment is conducted during the day versus night, and on weekdays versus weekends. Participants make incentivized decisions involving prosociality, punishment, and discounting, and complete a demographic and personality survey. We find no time or day differences in behavior, but do find that participants at nights and on weekends are less experienced with online studies; on weekends are less reflective; and at night are less conscientious and more neurotic. These results are largely robust to finer grained measures of time and day. We also find that those who participated earlier in the course of the study are more experienced, reflective, and agreeable, but less charitable than later participants.

From good institutions to generous citizens: Top-down incentives to cooperate promote subsequent prosociality but not norm enforcement.

What makes people willing to pay costs to help others, and to punish others' selfishness? Why does the extent of such behaviors vary markedly across cultures? To shed light on these questions, we explore the role of formal institutions in shaping individuals' prosociality and punishment. In Study 1 (N=707), American participants who reported living under higher quality cooperation-enforcing institutions (police and courts) gave significantly more in a Dictator Game (DG), but did not punish significantly more in a Third-Party Punishment Game (TPPG). In Study 1R (N=1705), we replicated the positive relationship between reported institutional quality and DG giving observed in Study 1. In Study 2 (N=516), we experimentally manipulated institutional quality in a repeated Public Goods Game with a centralized punishment institution. Consistent with the correlational results of Study 1 and 1R, we found that centralized punishment led to significantly more prosociality in a subsequent DG compared to a no-punishment control, but had no significant direct effect on subsequent TPPG punishment (only an indirect effect via increased DG giving). Thus we present convergent evidence that the quality of institutions one is exposed to "spills over" to subsequent prosociality but not punishment. These findings support a theory of social heuristics, suggest boundary conditions on spillover effects of cooperation, and demonstrate the power of effective institutions for instilling habits of virtue and creating cultures of cooperation.