Dataset Information

Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements.

ABSTRACT: Professional fact-checkers and fact-checking organizations provide a critical public service. Skeptics of modern media, however, often question the accuracy and objectivity of fact-checkers. The current study assessed agreement among two independent fact-checkers, The Washington Post and PolitiFact, regarding the false and misleading statements of then President Donald J. Trump. Differences in statement selection and deceptiveness scaling were investigated. The Washington Post checked PolitiFact fact-checks 77.4% of the time (22.6% selection disagreement). Moderate agreement was observed for deceptiveness scaling. Nearly complete agreement was observed for bottom-line attributed veracity. Additional cross-checking with other sources (Snopes, FactCheck.org), original sources, and with fact-checking for the first 100 days of President Joe Biden's administration were inconsistent with potential ideology effects. Our evidence suggests fact-checking is a difficult enterprise, there is considerable variability between fact-checkers in the raw number of statements that are checked, and finally, selection and scaling account for apparent discrepancies among fact-checkers.

SUBMITTER: Markowitz DM

PROVIDER: S-EPMC10368232 | biostudies-literature | 2023

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements.

Markowitz David M DM Levine Timothy R TR Serota Kim B KB Moore Alivia D AD

PloS one 20230725 7

Professional fact-checkers and fact-checking organizations provide a critical public service. Skeptics of modern media, however, often question the accuracy and objectivity of fact-checkers. The current study assessed agreement among two independent fact-checkers, The Washington Post and PolitiFact, regarding the false and misleading statements of then President Donald J. Trump. Differences in statement selection and deceptiveness scaling were investigated. The Washington Post checked PolitiFact ...[more]

PMID: 37490489

Similar Datasets

Project description:BackgroundEffective interventions aimed at correcting COVID-19 vaccine misinformation, known as fact-checking messages, are needed to combat the mounting antivaccine infodemic and alleviate vaccine hesitancy.ObjectiveThis work investigates (1) the changes in the public's attitude toward COVID-19 vaccines over time, (2) the effectiveness of COVID-19 vaccine fact-checking information on social media engagement and attitude change, and (3) the emotional and linguistic features of the COVID-19 vaccine fact-checking information ecosystem.MethodsWe collected a data set of 12,553 COVID-19 vaccine fact-checking Facebook posts and their associated comments (N=122,362) from January 2020 to March 2022 and conducted a series of natural language processing and statistical analyses to investigate trends in public attitude toward the vaccine in COVID-19 vaccine fact-checking posts and comments, and emotional and linguistic features of the COVID-19 fact-checking information ecosystem.ResultsThe percentage of fact-checking posts relative to all COVID-19 vaccine posts peaked in May 2020 and then steadily decreased as the pandemic progressed (r=-0.92, df=21, t=-10.94, 95% CI -0.97 to -0.82, P<.001). The salience of COVID-19 vaccine entities was significantly lower in comments (mean 0.03, SD 0.03, t=39.28, P<.001) than in posts (mean 0.09, SD 0.11). Third-party fact checkers have been playing a more important role in more fact-checking over time (r=0.63, df=25, t=4.06, 95% CI 0.33-0.82, P<.001). COVID-19 vaccine fact-checking posts continued to be more analytical (r=0.81, df=25, t=6.88, 95% CI 0.62-0.91, P<.001) and more confident (r=0.59, df=25, t=3.68, 95% CI 0.27-0.79, P=.001) over time. Although comments did not exhibit a significant increase in confidence over time, tentativeness in comments significantly decreased (r=-0.62, df=25, t=-3.94, 95% CI -0.81 to -0.31, P=.001). In addition, although hospitals receive less engagement than other information sources, the comments expressed more positive attitudinal valence in comments compared to other information sources (b=0.06, 95% CI 0.00-0.12, t=2.03, P=.04).ConclusionsThe percentage of fact-checking posts relative to all posts about the vaccine steadily decreased after May 2020. As the pandemic progressed, third-party fact checkers played a larger role in posting fact-checking COVID-19 vaccine posts. COVID-19 vaccine fact-checking posts continued to be more analytical and more confident over time, reflecting increased confidence in posts. Similarly, tentativeness in comments decreased; this likewise suggests that public uncertainty diminished over time. COVID-19 fact-checking vaccine posts from hospitals yielded more positive attitudes toward vaccination than other information sources. At the same time, hospitals received less engagement than other information sources. This suggests that hospitals should invest more in generating engaging public health campaigns on social media.

Dataset Information

Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements.

Publications

Cross-checking journalistic fact-checkers: The role of sampling and scaling in interpreting false and misleading statements.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets