Using crowdsourcing to evaluate published scientific literature: methods and example.
Ontology highlight
ABSTRACT: Systematically evaluating scientific literature is a time consuming endeavor that requires hours of coding and rating. Here, we describe a method to distribute these tasks across a large group through online crowdsourcing. Using Amazon's Mechanical Turk, crowdsourced workers (microworkers) completed four groups of tasks to evaluate the question, "Do nutrition-obesity studies with conclusions concordant with popular opinion receive more attention in the scientific community than do those that are discordant?" 1) Microworkers who passed a qualification test (19% passed) evaluated abstracts to determine if they were about human studies investigating nutrition and obesity. Agreement between the first two raters' conclusions was moderate (κ = 0.586), with consensus being reached in 96% of abstr
SUBMITTER: Brown AW
PROVIDER: S-EPMC4079692 | biostudies-literature | 2014
REPOSITORIES: biostudies-literature
ACCESS DATA