Producing knowledge by admitting ignorance: Enhancing data quality through an "I don't know" option in citizen science.

ABSTRACT: The "noisy labeler problem" in crowdsourced data has attracted great attention in recent years, with important ramifications in citizen science, where non-experts must produce high-quality data. Particularly relevant to citizen science is dynamic task allocation, in which the level of agreement among labelers can be progressively updated through the information-theoretic notion of entropy. Under dynamic task allocation, we hypothesized that providing volunteers with an "I don't know" option would contribute to enhancing data quality, by introducing further, useful information about the level of agreement among volunteers. We investigated the influence of an "I don't know" option on the data quality in a citizen science project that entailed classifying the image of a highly polluted canal

SUBMITTER: Torre M 

PROVIDER: S-EPMC6392254 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature