The Accuracy and Reliability of Crowdsource Annotations of Digital Retinal Images.

ABSTRACT:

Purpose

Crowdsourcing is based on outsourcing computationally intensive tasks to numerous individuals in the online community who have no formal training. Our aim was to develop a novel online tool designed to facilitate large-scale annotation of digital retinal images, and to assess the accuracy of crowdsource grading using this tool, comparing it to expert classification.

Methods

We used 100 retinal fundus photograph images with predetermined disease criteria selected by two experts from a large cohort study. The Amazon Mechanical Turk Web platform was used to drive traffic to our site so anonymous workers could perform a classification and annotation task of the fundus photographs in our dataset after a short training exercise. Three groups were assessed: masters only, no

SUBMITTER: Mitry D 

PROVIDER: S-EPMC5032847 | biostudies-literature | 2016 Sep

REPOSITORIES: biostudies-literature