Dataset Information

Measuring the crowd within again: a pre-registered replication study.

ABSTRACT: According to the crowd within effect, the average of two estimates from one person tends to be more accurate than a single estimate of that person. The effect implies that the well documented wisdom of the crowd effect-the crowd's average estimate tends to be more accurate than the individual estimates-can be obtained within a single individual. In this paper, we performed a high-powered, pre-registered replication study of the original experiment. Our replication results are evaluated with the traditional null hypothesis significance testing approach, as well as with effect sizes and their confidence intervals. We adopted a co-pilot approach, in the sense that all analyses were performed independently by two researchers using different analysis software. Moreover, we report Bayes factors for all tests. We successfully replicated the crowd within effect, both when the second guess was made immediately after the first guess, as well as when it was made 3 weeks later. The experimental protocol, the raw data, the post-processed data and the analysis code are available online.

SUBMITTER: Steegen S

PROVIDER: S-EPMC4112915 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Measuring the crowd within again: a pre-registered replication study.

Steegen Sara S Dewitte Laura L Tuerlinckx Francis F Vanpaemel Wolf W

Frontiers in psychology 20140728

According to the crowd within effect, the average of two estimates from one person tends to be more accurate than a single estimate of that person. The effect implies that the well documented wisdom of the crowd effect-the crowd's average estimate tends to be more accurate than the individual estimates-can be obtained within a single individual. In this paper, we performed a high-powered, pre-registered replication study of the original experiment. Our replication results are evaluated with the ...[more]

PMID: 25120505

Similar Datasets

Project description:Numerous studies report brain potential evidence for the anticipation of specific words during language comprehension. In the most convincing demonstrations, highly predictable nouns exert an influence on processing even before they appear to a reader or listener, as indicated by the brain's neural response to a prenominal adjective or article when it mismatches the expectations about the upcoming noun. However, recent studies suggest that some well-known demonstrations of prediction may be hard to replicate. This could signal the use of data-contingent analysis, but might also mean that readers and listeners do not always use prediction-relevant information in the way that psycholinguistic theories typically suggest. To shed light on this issue, we performed a close replication of one of the best-cited ERP studies on word anticipation (Van Berkum, Brown, Zwitserlood, Kooijman & Hagoort, 2005; Experiment 1), in which participants listened to Dutch spoken mini-stories. In the original study, the marking of grammatical gender on pre-nominal adjectives ('groot/grote') elicited an early positivity when mismatching the gender of an unheard, highly predictable noun, compared to matching gender. The current pre-registered study involved that same manipulation, but used a novel set of materials twice the size of the original set, an increased sample size (N = 187), and Bayesian mixed-effects model analyses that better accounted for known sources of variance than the original. In our study, mismatching gender elicited more negative voltage than matching gender at posterior electrodes. However, this N400-like effect was small in size and lacked support from Bayes Factors. In contrast, we successfully replicated the original's noun effects. While our results yielded some support for prediction, they do not support the Van Berkum et al. effect and highlight the risks associated with commonly employed data-contingent analyses and small sample sizes. Our results also raise the question whether Dutch listeners reliably or consistently use adjectival inflection information to inform their noun predictions.

Project description:Certain manipulations, such as testing oneself on newly learned word associations (recall), or the act of repeating a word during training (reproduction), can lead to better learning and retention relative to simply providing more exposure to the word (restudy). Such benefit has been observed for written words. Here, we test how these training manipulations affect learning of words presented aurally, when participants are required to produce these novel phonological forms in a recall task.Participants (36 English-speaking adults) learned 27 pseudowords, which were paired with 27 unfamiliar pictures. They were given cued recall practice for 9 of the words, reproduction practice for another set of 9 words, and the remaining 9 words were restudied. Participants were tested on their recognition (3-alternative forced choice) and recall (saying the pseudoword in response to a picture) of these items immediately after training, and a week after training. Our hypotheses were that reproduction and restudy practice would lead to better learning immediately after training, but that cued recall practice would lead to better retention in the long term.In all three conditions, recognition performance was extremely high immediately after training, and a week following training, indicating that participants had acquired associations between the novel pictures and novel words. In addition, recognition and cued recall performance was better immediately after training relative to a week later, confirming that participants forgot some words over time. However, results in the cued recall task did not support our hypotheses. Immediately after training, participants showed an advantage for cued Recall over the Restudy condition, but not over the Reproduce condition. Furthermore, there was no boost for the cued Recall condition over time relative to the other two conditions. Results from a Bayesian analysis also supported this null finding. Nonetheless, we found a clear effect of word length, with shorter words being better learned than longer words, indicating that our method was sufficiently sensitive to detect an impact of condition on learning.Our primary hypothesis about training conditions conferring specific advantages for production of novel words presented aurally, especially over long intervals, was not supported by this data. Although there may be practical reasons for preferring a particular method for training expressive vocabulary, no difference in effectiveness was detected when presenting words aurally: reproducing, recalling or restudying a word led to the same production accuracy.

Project description:Introduction Internet-based self-help interventions have shown to be effective in the treatment of depression. Several meta-analyses indicated that human contact has a crucial impact on adherence and outcome. While most research focused on the role of guidance during interventions, a review by Andersson and Johansson (2012) suggested that contact before the intervention too may play an important role. Objective We investigated the impact of the degree of contact in internet-based interventions (IBIs) for depression on outcome in adults suffering from elevated symptoms of depression. Methods We conducted a preregistered meta-analysis (www.osf.io/4mzyd) and included trials comparing IBIs for depression against control conditions (treatment as usual [TAU] or waiting list [WL]) in patients with symptoms of unipolar depression searching the databases PsycINFO and Cochrane's Central Register of Controlled Trials (CENTRAL) limited to entries from EMBASE and PubMed. Following Andersson and Johansson (2012), contact before an intervention was defined as having had a diagnostic interview before the IBI, and contact during intervention was defined as having received guidance during the IBI. IBIs were grouped as providing (0) no contact, (1) contact before the IBI, (2) contact during the IBI, or (3) contact both before and during the IBI. The primary outcome was standardized mean difference (SMD) of the IBI and control in depressive symptoms at treatment termination. Secondary outcomes were study dropout and adherence to the IBI. Results We included 56 eligible trials that randomized 13,335 patients to 75 internet-based intervention conditions and control groups (TAU in 23 trials, WL in 33 trials). In total, 44 trials (78.57 %) were judged to show some concerns or a high risk of bias. Overall heterogeneity was high regarding the primary outcome (I2s < 66 %) and even higher for secondary outcomes (I2s < 91 %). Degree of contact was a robust predictor of outcome and adherence in all pre-registered and exploratory analyses. We found the effect of the IBI to increase with higher degree of contact. However, in pair-wise contrasts, only IBIs offering both contact before and during the intervention (SMD = 0.573, 95 % CI: 0.437, 0.709) significantly outperformed interventions offering no contact (SMD = 0.224, 95 % CI: 0.090, 0.340). Conclusions The results suggest that contact before and during an intervention increases the effects of IBIs. The combination of contact before and during the intervention seems to a pivotal role regarding adherence as well as treatment outcome for patients suffering from depression. Highlights • Degree of contact is a robust predictor of outcome and adherence in internet-based interventions (IBIs).• Contact before the intervention as well as contact during the IBI are independently associated with outcome.• Only IBIs with contact before and during the intervention significantly outperformed interventions without any contact.• Synchronous contact before an internet-intervention seems to play a pivotal role.

Dataset Information

Measuring the crowd within again: a pre-registered replication study.

Publications

Measuring the crowd within again: a pre-registered replication study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets