Improving Consensus Scoring of Crowdsourced Data Using the Rasch Model: Development and Refinement of a Diagnostic Instrument.

ABSTRACT:

Background

Diabetic retinopathy (DR) is a leading cause of vision loss in working age individuals worldwide. While screening is effective and cost effective, it remains underutilized, and novel methods are needed to increase detection of DR. This clinical validation study compared diagnostic gradings of retinal fundus photographs provided by volunteers on the Amazon Mechanical Turk (AMT) crowdsourcing marketplace with expert-provided gold-standard grading and explored whether determination of the consensus of crowdsourced classifications could be improved beyond a simple majority vote (MV) using regression methods.

Objective

The aim of our study was to determine whether regression methods could be used to improve the consensus grading of data collected by crowdsourcing.

M

SUBMITTER: Brady CJ 

PROVIDER: S-EPMC5497070 | biostudies-literature | 2017 Jun

REPOSITORIES: biostudies-literature