Dataset Information

The myopia of crowds: Cognitive load and collective evaluation of answers on Stack Exchange.

ABSTRACT: Crowds can often make better decisions than individuals or small groups of experts by leveraging their ability to aggregate diverse information. Question answering sites, such as Stack Exchange, rely on the "wisdom of crowds" effect to identify the best answers to questions asked by users. We analyze data from 250 communities on the Stack Exchange network to pinpoint factors affecting which answers are chosen as the best answers. Our results suggest that, rather than evaluate all available answers to a question, users rely on simple cognitive heuristics to choose an answer to vote for or accept. These cognitive heuristics are linked to an answer's salience, such as the order in which it is listed and how much screen space it occupies. While askers appear to depend on heuristics to a greater extent than voters when choosing an answer to accept as the most helpful one, voters use acceptance itself as a heuristic, and they are more likely to choose the answer after it has been accepted than before that answer was accepted. These heuristics become more important in explaining and predicting behavior as the number of available answers to a question increases. Our findings suggest that crowd judgments may become less reliable as the number of answers grows.

SUBMITTER: Burghardt K

PROVIDER: S-EPMC5354439 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The myopia of crowds: Cognitive load and collective evaluation of answers on Stack Exchange.

Burghardt Keith K Alsina Emanuel F EF Girvan Michelle M Rand William W Lerman Kristina K

PloS one 20170316 3

Crowds can often make better decisions than individuals or small groups of experts by leveraging their ability to aggregate diverse information. Question answering sites, such as Stack Exchange, rely on the "wisdom of crowds" effect to identify the best answers to questions asked by users. We analyze data from 250 communities on the Stack Exchange network to pinpoint factors affecting which answers are chosen as the best answers. Our results suggest that, rather than evaluate all available answe ...[more]

PMID: 28301531

Similar Datasets

Project description:Agent-based models of 'flocking' and 'schooling' have shown that a weighted average of neighbor velocities, with weights that decay gradually with distance, yields emergent collective motion. Weighted averaging thus offers a potential mechanism of self-organization that recruits an increasing, but self-limiting, number of individuals into collective motion. Previously, we identified and modeled such a 'soft metric' neighborhood of interaction in human crowds that decays exponentially to zero at a distance of 4-5m. Here we investigate the limits of weighted averaging in humans and find that it is surprisingly robust: pedestrians align with the mean heading direction in their neighborhood, despite high levels of noise and diverging motions in the crowd, as predicted by the model. In three Virtual Reality experiments, participants were immersed in a crowd of virtual humans in a mobile head-mounted display and were instructed to walk with the crowd. By perturbing the heading (walking direction) of virtual neighbors and measuring the participant's trajectory, we probed the limits of weighted averaging. (1) In the 'Noisy Neighbors' experiment, the neighbor headings were randomized (range 0-90°) about the crowd's mean direction (±10° or ±20°, left or right); (2) in the 'Splitting Crowd' experiment, the crowd split into two groups (heading difference = 10-40°) and the proportion of the crowd in one group was varied (50-84%); (3) in the 'Coherent Subgroup' experiment, a perturbed subgroup varied in its coherence (heading SD = 0-2°) about a mean direction (±10° or ±20°) within a noisy crowd (heading range = 180°), and the proportion of the crowd in the subgroup was varied. In each scenario, the results were predicted by the weighted averaging model, and attraction strength (turning rate) increased with the participant's deviation from the mean heading direction, not with group coherence. However, the results indicate that humans ignore highly discrepant headings (45-90°). These findings reveal that weighted averaging in humans is highly robust and generates a common heading direction that acts as a positive feedback to recruit more individuals into collective motion, in a self-reinforcing cascade. Therefore, this 'soft' metric neighborhood serves as a mechanism of self-organization in human crowds.

Project description:IntroductionUnderstanding collective behavior of moving organisms and how interactions between individuals govern their collective motion has triggered a growing number of studies. Similarities have been observed between the scale-free behavioral aspects of various systems (i.e. groups of fish, ants, and mammals). Investigation of such connections between the collective motion of non-human organisms and that of humans however, has been relatively scarce. The problem demands for particular attention in the context of emergency escape motion for which innovative experimentation with panicking ants has been recently employed as a relatively inexpensive and non-invasive approach. However, little empirical evidence has been provided as to the relevance and reliability of this approach as a model of human behaviour.MethodsThis study explores pioneer experiments of emergency escape to tackle this question and to connect two forms of experimental observations that investigate the collective movement at macroscopic level. A large number of experiments with human and panicking ants are conducted representing the escape behavior of these systems in crowded spaces. The experiments share similar architectural structures in which two streams of crowd flow merge with one another. Measures such as discharge flow rates and the probability distribution of passage headways are extracted and compared between the two systems.FindingsOur findings displayed an unexpected degree of similarity between the collective patterns emerged from both observation types, particularly based on aggregate measures. Experiments with ants and humans commonly indicated how significantly the efficiency of motion and the rate of discharge depend on the architectural design of the movement environment.Practical applicationsOur findings contribute to the accumulation of evidence needed to identify the boarders of applicability of experimentation with crowds of non-human entities as models of human collective motion as well as the level of measurements (i.e. macroscopic or microscopic) and the type of contexts at which reliable inferences can be drawn. This particularly has implications in the context of experimenting evacuation behaviour for which recruiting human subjects may face ethical restrictions. The findings, at minimum, offer promise as to the potential benefit of piloting such experiments with non-human crowds, thereby forming better-informed hypotheses.

Dataset Information

The myopia of crowds: Cognitive load and collective evaluation of answers on Stack Exchange.

Publications

The myopia of crowds: Cognitive load and collective evaluation of answers on Stack Exchange.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets