Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation

2022/10/27 by Stefan Wojcik, Sophie Hilgard, Wojcik, Stefan +13 · 9 voices · 19 citations
Social Sciences · Physics and Astronomy · #Misinformation and Its Impacts #Opinion Dynamics and Social Influence #Social Media and Politics

paper · pdf · doi:10.48550/arxiv.2210.15723

Abstract

We present an approach for selecting objectively informative and subjectively helpful annotations to social media posts. We draw on data from on an online environment where contributors annotate misinformation and simultaneously rate the contributions of others. Our algorithm uses a matrix-factorization (MF) based approach to identify annotations that appeal broadly across heterogeneous user groups - sometimes referred to as "bridging-based ranking." We pair these data with a survey experiment in which individuals are randomly assigned to see annotations to posts. We find that annotations selected by the algorithm improve key indicators compared with overall average and crowd-generated baselines. Further, when deployed on Twitter, people who saw annotations selected through this bridging-based approach were significantly less likely to reshare social media posts than those who did not see the annotations.

Cited by

• Paper Skygest: Personalized Academic Recommendations on Bluesky
• The Risks of Industry Influence in Tech Research
• Can We Fix Social Media? Testing Prosocial Interventions using Generative Social Simulation
• Moving towards informative and actionable social media research
• The Gray Area: Characterizing Moderator Disagreement on Reddit
• Republicans are flagged more often than Democrats for sharing misinformation on X’s Community Notes
• Community Notes are Vulnerable to Rater Bias and Manipulation
• Reducing Misinformation Credibility: How Explanations Impact the Effectiveness of Social Media Warning Labels and Fact-Checking Source Recall
• AI Fact-Checking in the Wild: A Field Evaluation of LLM-Written Community Notes on X
• COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations
• Timeliness, Consensus, and Composition of the Crowd: Community Notes on X
• Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation
• From Birdwatch to Community Notes, from Twitter to X: four years of community-based content moderation
• Threats to the sustainability of Community Notes on X
• Beyond Community Notes: A Framework for Understanding and Building Crowdsourced Context Systems for Social Media
• Request a Note: How the Request Function Shapes X's Community Notes System
• Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech
• Network Prebunking Problem: Optimizing Prebunking Targets to Suppress the Spread of Misinformation in Social Networks
• A New Incentive Model For Content Trust

Discussions

• Birdwatch: Crowd Wisdom and Algorithms to Understand and Reduce Misinformation [hn, 41 points, 6 comments]
• Mary Beth was on the original team that developed and deployed Twitter Birdwatch = X Community Notes, and at FB before that. Insightful thread. See also the Birdwatch paper, submitted to arxiv the n [bsky, 17 points, 1 comments]
• Birdwatch: Crowd Wisdom and Bridging Algorithm-Powered Misinfo Annotations [hn, 4 points, 0 comments]
• Fact-Checking Is Out, ‘Community Notes’ Are In [lemmy, 3 points, 0 comments]
• It occurs to me that it might have been helpful to share those things here for interested folks: - (1) The paper describing methods & early efficacy evals - arxiv.org/abs/2210.15723 - (2) The public [bsky, 2 points, 1 comments]
• birdwatch was really cool. the team was fucking awesome. we had a fucking vision for open source arxiv.org/abs/2210.15723 [bsky, 2 points, 0 comments]
• The implications of Zuck's message are clearly self serving and gross. But adding a Community Notes system is a good idea: Bluesky likely needs one too. Twitter was working on this well before Elon, [bsky, 2 points, 0 comments]
• See also: “Birdwatch: Crowd Wisdom And Bridging Algorithms Can Inform Understanding And Reduce The Spread Of Misinformation” [2022], Stefan Wojcik, et al (arxiv.org/abs/2210.15723). #Paper #Communit [bsky, 0 points, 0 comments]
• Wer hat denn das Anrecht genau zu wissen was die Wahrheit ist? Das "Birdwatch paper" ist eine sehr interessante Idee dem näher zu kommen. "Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Un [bsky, 0 points, 0 comments]

Related

• A Survey on the Role of Crowds in Combating Online Misinformation: Annotators, Evaluators, and Creators
• Community-Based Fact-Checking on Twitter's Birdwatch Platform
• Annotator in the Loop: A Case Study of In-Depth Rater Engagement to Create a Bridging Benchmark Dataset
• AMUSED: An Annotation Framework of Multi-modal Social Media Data
• Rescuing Counterspeech: A Bridging-Based Approach to Combating Misinformation