Dataset Information

Human judgement forecasting tournaments: A feasibility study based on the COVID-19 pandemic with public health practitioners in England.

ABSTRACT:

Objectives

To explore the viability of running human judgement forecasting tournaments with public health practitioners, and to gather initial data on forecasting accuracy and participant perceptions of forecasting.

Study design

Quality improvement study comprising two COVID-19 forecasting tournaments using Brier Skill Score scoring and a follow-up participant questionnaire.

Methods

Over two forecasting tournaments, public health registrars in the East Midlands, UK, assigned probabilities to future possible binary events relating to COVID-19. Participants also completed a questionnaire on their experiences of forecasting.

Results

There were 17 participants in the first tournament and nine in the second tournament, with no new participants. In both tournaments, the majority of participants scored a Brier Skill Score above the benchmark of 0. The median Brier Skill Score improved slightly between the two tournaments. Participants reported luck and changing political climates as impacting their performance. Participants reported forecasting in their day job but had received no formal training to do so.

Conclusions

Forecasting is an important public health skill, and human judgement forecasting tournaments can be run amongst public health practitioners with little time and resource requirements. Further research would help identify whether training, teamwork or other interventions can improve public health forecasting accuracy.

SUBMITTER: Davies N

PROVIDER: S-EPMC9023363 | biostudies-literature | 2022 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Human judgement forecasting tournaments: A feasibility study based on the COVID-19 pandemic with public health practitioners in England.

Davies Nathan N Davies Nathan N Ferris Simon S

Public health in practice (Oxford, England) 20220422

<h4>Objectives</h4>To explore the viability of running human judgement forecasting tournaments with public health practitioners, and to gather initial data on forecasting accuracy and participant perceptions of forecasting.<h4>Study design</h4>Quality improvement study comprising two COVID-19 forecasting tournaments using Brier Skill Score scoring and a follow-up participant questionnaire.<h4>Methods</h4>Over two forecasting tournaments, public health registrars in the East Midlands, UK, assigne ...[more]

PMID: 35479262

Similar Datasets

Project description:Pandemics such as Covid-19 pose tremendous public health communication challenges in promoting protective behaviours, vaccination, and educating the public about risks. Segmenting audiences based on attitudes and behaviours is a means to increase the precision and potential effectiveness of such communication. The present study reports on such an audience segmentation effort for the population of England, sponsored by the United Kingdom Health Security Agency (UKHSA) and involving a collaboration of market research and academic experts. A cross-sectional online survey was conducted between 4 and 24 January 2022 with 5525 respondents (5178 used in our analyses) in England using market research opt-in panel. An additional 105 telephone interviews were conducted to sample persons without online or smartphone access. Respondents were quota sampled to be demographically representative. The primary analytic technique was k means cluster analysis, supplemented with other techniques including multi-dimensional scaling and use of respondent - as well as sample-standardized data when necessary to address differences in response set for some groups of respondents. Identified segments were profiled against demographic, behavioural self-report, attitudinal, and communication channel variables, with differences by segment tested for statistical significance. Seven segments were identified, including distinctly different groups of persons who tended toward a high level of compliance and several that were relatively low in compliance. The segments were characterized by distinctive patterns of demographics, attitudes, behaviours, trust in information sources, and communication channels preferred. Segments were further validated by comparing the segmentation variable versus a set of demographic variables as predictors of reported protective behaviours in the past two weeks and of vaccine refusal; the demographics together had about one-quarter the effect size of the single seven-level segment variable. With respect to managerial implications, different communication strategies for each segment are suggested for each segment, illustrating advantages of rich segmentation descriptions for understanding public health communication audiences. Strengths and weaknesses of the methods used are discussed, to help guide future efforts.

Project description:IntroductionThe emergence of COVID-19 and the importance of behaviour change to limit its spread created an urgent need to apply behavioural science to public health. Knowledge mobilisation, the processes whereby research leads to useful findings that are implemented to affect positive outcomes, is a goal for researchers, policy makers and practitioners alike. This study aimed to explores the experience of using behavioural science in public health during COVID-19, to discover barriers and facilitators and whether the rapidly changing context of COVID-19 influenced knowledge mobilisation.MethodsWe conducted a semi-structured interview study, with ten behavioural scientists and seven public health professionals in England, Scotland, Wales, The Netherlands and Canada. We conducted an inductive thematic analysis.ResultsWe report three key themes and 10 sub-themes: 1.Challenges and facilitators of translation of behavioural science into public health (Methods and frameworks supported translation, Lack of supportive infrastructure, Conviction and sourcing of evidence and Embracing behavioural science) 2. The unique context of translation (Rapid change in context, the multi-disciplinary team and the emotional toll). 3. Recommendations to support future behavioural science translation (Embedding experts into teams, Importance of a collaborative network and showcasing the role of behavioural science).DiscussionBarriers and facilitators included factors related to relationships between people, such as networks and teams; the expertise of individual people; and those related to materials, such as the use of frameworks and an overwhelming amount of evidence and literature.ConclusionPeople and frameworks were seen as important in facilitating behavioural science in practice. Future research could explore how different frameworks are used. We recommend a stepped competency framework for behavioural science in public health and more focus on nurturing networks to facilitate knowledge mobilisation in future emergencies.

Project description:ObjectivesTo explore public reactions to the COVID-19 pandemic across diverse ethnic groups.DesignRemote qualitative interviews and focus groups in English or Punjabi. Data were transcribed and analysed through inductive thematic analysis.SettingEngland and Wales, June to October 2020.Participants100 participants from 19 diverse 'self-identified' ethnic groups.ResultsDismay, frustration and altruism were reported across all ethnic groups during the first 6-9 months of the COVID-19 pandemic. Dismay was caused by participants' reported individual, family and community risks, and loss of support networks. Frustration was caused by reported lack of recognition of the efforts of ethnic minority groups (EMGs), inaction by government to address COVID-19 and inequalities, rule breaking by government advisors, changing government rules around: border controls, personal protective equipment, social distancing, eating out, and perceived poor communication around COVID-19 and the Public Health England COVID-19 disparities report (leading to reported increased racism and social isolation). Altruism was felt by all, in the resilience of National Health Service (NHS) staff and their communities and families pulling together. Data, participants' suggested actions and the behaviour change wheel informed suggested interventions and policies to help control COVID-19.ConclusionTo improve trust and compliance future reports or guidance should clearly explain any stated differences in health outcomes by ethnicity or other risk group, including specific messages for these groups and concrete actions to minimise any risks. Messaging should reflect the uncertainty in data or advice and how guidance may change going forward as new evidence becomes available. A contingency plan is needed to mitigate the impact of COVID-19 across all communities including EMGs, the vulnerable and socially disadvantaged individuals, in preparation for any rise in cases and for future pandemics. Equality across ethnicities for healthcare is essential, and the NHS and local communities will need to be supported to attain this.

Dataset Information

Human judgement forecasting tournaments: A feasibility study based on the COVID-19 pandemic with public health practitioners in England.

Objectives

Study design

Methods

Results

Conclusions

Publications

Human judgement forecasting tournaments: A feasibility study based on the COVID-19 pandemic with public health practitioners in England.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets