Dataset Information

Accuracy-Risk Trade-Off Due to Social Learning in Crowd-Sourced Financial Predictions.

ABSTRACT: A critical question relevant to the increasing importance of crowd-sourced-based finance is how to optimize collective information processing and decision-making. Here, we investigate an often under-studied aspect of the performance of online traders: beyond focusing on just accuracy, what gives rise to the trade-off between risk and accuracy at the collective level? Answers to this question will lead to designing and deploying more effective crowd-sourced financial platforms and to minimizing issues stemming from risk such as implied volatility. To investigate this trade-off, we conducted a large online Wisdom of the Crowd study where 2037 participants predicted the prices of real financial assets (S&P 500, WTI Oil and Gold prices). Using the data collected, we modeled the belief update process of participants using models inspired by Bayesian models of cognition. We show that subsets of predictions chosen based on their belief update strategies lie on a Pareto frontier between accuracy and risk, mediated by social learning. We also observe that social learning led to superior accuracy during one of our rounds that occurred during the high market uncertainty of the Brexit vote.

SUBMITTER: Adjodah D

PROVIDER: S-EPMC8307866 | biostudies-literature | 2021 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Accuracy-Risk Trade-Off Due to Social Learning in Crowd-Sourced Financial Predictions.

Adjodah Dhaval D Leng Yan Y Chong Shi Kai SK Krafft P M PM Moro Esteban E Pentland Alex A

Entropy (Basel, Switzerland) 20210624 7

A critical question relevant to the increasing importance of crowd-sourced-based finance is how to optimize collective information processing and decision-making. Here, we investigate an often under-studied aspect of the performance of online traders: beyond focusing on just accuracy, what gives rise to the trade-off between risk and accuracy at the collective level? Answers to this question will lead to designing and deploying more effective crowd-sourced financial platforms and to minimizing i ...[more]

PMID: 34202445

Similar Datasets

Project description:Despite limited information and knowledge, we personally form beliefs about certain properties of objects encountered in our daily life-popularity of a newly released movie, for example. Since such beliefs are prone to error, we often revise our initial beliefs according to the beliefs of others to improve accuracy. Optimal revision requires modulating the degree of accepting others' beliefs based on various cues for accuracy-number of opinions, for example-such that the more accurate others' beliefs are, the more we accept them. Although previous studies have shown that such accuracy cues can influence the degree of acceptance during social revision, they primarily investigated problems with 'factually correct' answers, and rarely problems with 'socially correct' answers. Here we examined which accuracy cues are objectively useful (utility of cues), and how those cues are used (use of cues), in the social revision of people's beliefs about problems with 'socially correct' answers. We asked people to estimate the 'shared preferences (SPs)' for sociocultural items, the answers to which are determined by socially aggregated beliefs-how popular an abstract painting will be among a large crowd, for example-and then to revise their initial estimates after being exposed to other people's estimates about the same items. We considered 'confidence', 'agreement among estimates', and 'number of estimates' as accuracy cues. We found that, while all three cues validly signaled the accuracy of SP estimates, only the 'number' cue has a significant utility, but the other cues are much less useful for optimal revision. Nevertheless, people used the cues of 'agreement' and their own 'confidence' to the extent comparable to that of the 'number' cue. Our findings suggest that the utility and use of accuracy cues for problems with 'socially correct' answers differ from those with 'factually correct' answers, as follows: (i) confidence does not have a significant utility and (ii) but people use their own confidence while ignoring others' confidence.

Project description:IntroductionTo mitigate the COVID-19 pandemic and prevent overwhelming the healthcare system, social-distancing policies such as school closure, stay-at-home orders, and indoor dining closure have been utilized worldwide. These policies function by reducing the rate of close contact within populations and result in decreased human mobility. Adherence to social distancing can substantially reduce disease spread. Thus, quantifying human mobility and social-distancing compliance, especially at high temporal resolution, can provide great insight into the impact of social distancing policies.MethodsWe used the movement of individuals around New York City (NYC), measured via traffic levels, as a proxy for human mobility and the impact of social-distancing policies (i.e., work from home policies, school closure, indoor dining closure etc.). By data mining Google traffic in real-time, and applying image processing, we derived high resolution time series of traffic in NYC. We used time series decomposition and generalized additive models to quantify changes in rush hour/non-rush hour, and weekday/weekend traffic, pre-pandemic and following the roll-out of multiple social distancing interventions.ResultsMobility decreased sharply on March 14, 2020 following declaration of the pandemic. However, levels began rebounding by approximately April 13, almost 2 months before stay-at-home orders were lifted, indicating premature increase in mobility, which we term social-distancing fatigue. We also observed large impacts on diurnal traffic congestion, such that the pre-pandemic bi-modal weekday congestion representing morning and evening rush hour was dramatically altered. By September, traffic congestion rebounded to approximately 75% of pre-pandemic levels.ConclusionUsing crowd-sourced traffic congestion data, we described changes in mobility in Manhattan, NYC, during the COVID-19 pandemic. These data can be used to inform human mobility changes during the current pandemic, in planning for responses to future pandemics, and in understanding the potential impact of large-scale traffic interventions such as congestion pricing policies.

Project description:BackgroundWhile many patients seem to recover from SARS-CoV-2 infections, many patients report experiencing SARS-CoV-2 symptoms for weeks or months after their acute COVID-19 ends, even developing new symptoms weeks after infection. These long-term effects are called post-acute sequelae of SARS-CoV-2 (PASC) or, more commonly, Long COVID. The overall prevalence of Long COVID is currently unknown, and tools are needed to help identify patients at risk for developing long COVID.MethodsA working group of the Rapid Acceleration of Diagnostics-radical (RADx-rad) program, comprised of individuals from various NIH institutes and centers, in collaboration with REsearching COVID to Enhance Recovery (RECOVER) developed and organized the Long COVID Computational Challenge (L3C), a community challenge aimed at incentivizing the broader scientific community to develop interpretable and accurate methods for identifying patients at risk of developing Long COVID. From August 2022 to December 2022, participants developed Long COVID risk prediction algorithms using the National COVID Cohort Collaborative (N3C) data enclave, a harmonized data repository from over 75 healthcare institutions from across the United States (U.S.).FindingsOver the course of the challenge, 74 teams designed and built 35 Long COVID prediction models using the N3C data enclave. The top 10 teams all scored above a 0.80 Area Under the Receiver Operator Curve (AUROC) with the highest scoring model achieving a mean AUROC of 0.895. Included in the top submission was a visualization dashboard that built timelines for each patient, updating the risk of a patient developing Long COVID in response to clinical events.InterpretationAs a result of L3C, federal reviewers identified multiple machine learning models that can be used to identify patients at risk for developing Long COVID. Many of the teams used approaches in their submissions which can be applied to future clinical prediction questions.FundingResearch reported in this RADx® Rad publication was supported by the National Institutes of Health. Timothy Bergquist, Johanna Loomba, and Emily Pfaff were supported by Axle Subcontract: NCATS-STSS-P00438.

Project description:BackgroundVideo-based assessment by experts may structurally measure surgical performance using procedure-specific competency assessment tools (CATs). A CAT for minimally invasive esophagectomy (MIE-CAT) was developed and validated previously. However, surgeon's time is scarce and video assessment is time-consuming and labor intensive. This study investigated non-procedure-specific assessment of MIE video clips by MIE experts and crowdsourcing, collective surgical performance evaluation by anonymous and untrained laypeople, to assist procedure-specific expert review.MethodsTwo surgical performance scoring frameworks were used to assess eight MIE videos. First, global performance was assessed with the non-procedure-specific Global Operative Assessment of Laparoscopic Skills (GOALS) of 64 procedural phase-based video clips < 10 min. Each clip was assessed by two MIE experts and > 30 crowd workers. Second, the same experts assessed procedure-specific performance with the MIE-CAT of the corresponding full-length video. Reliability and convergent validity of GOALS for MIE were investigated using hypothesis testing with correlations (experience, blood loss, operative time, and MIE-CAT).ResultsLess than 75% of hypothesized correlations between GOALS scores and experience of the surgical team (r < 0.3), blood loss (r = - 0.82 to 0.02), operative time (r = - 0.42 to 0.07), and the MIE-CAT scores (r = - 0.04 to 0.76) were met for both crowd workers and experts. Interestingly, experts' GOALS and MIE-CAT scores correlated strongly (r = 0.40 to 0.79), while crowd workers' GOALS and experts' MIE-CAT scores correlations were weak (r = - 0.04 to 0.49). Expert and crowd worker GOALS scores correlated poorly (ICC ≤ 0.42).ConclusionGOALS assessments by crowd workers lacked convergent validity and showed poor reliability. It is likely that MIE is technically too difficult to assess for laypeople. Convergent validity of GOALS assessments by experts could also not be established. GOALS might not be comprehensive enough to assess detailed MIE performance. However, expert's GOALS and MIE-CAT scores strongly correlated indicating video clip (instead of full-length video) assessments could be useful to shorten assessment time.

Dataset Information

Accuracy-Risk Trade-Off Due to Social Learning in Crowd-Sourced Financial Predictions.

Publications

Accuracy-Risk Trade-Off Due to Social Learning in Crowd-Sourced Financial Predictions.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets