Dataset Information

Megastudies improve the impact of applied behavioural science.

ABSTRACT: Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens' decisions and outcomes¹. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals². The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy-a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research^3-6. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

SUBMITTER: Milkman KL

PROVIDER: S-EPMC8822539 | biostudies-literature | 2021 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Megastudies improve the impact of applied behavioural science.

Milkman Katherine L KL Gromet Dena D Ho Hung H Kay Joseph S JS Lee Timothy W TW Pandiloski Pepi P Park Yeji Y Rai Aneesh A Bazerman Max M Beshears John J Bonacorsi Lauri L Camerer Colin C Chang Edward E Chapman Gretchen G Cialdini Robert R Dai Hengchen H Eskreis-Winkler Lauren L Fishbach Ayelet A Gross James J JJ Horn Samantha S Hubbard Alexa A Jones Steven J SJ Karlan Dean D Kautz Tim T Kirgios Erika E Klusowski Joowon J Kristal Ariella A Ladhania Rahul R Loewenstein George G Ludwig Jens J Mellers Barbara B Mullainathan Sendhil S Saccardo Silvia S Spiess Jann J Suri Gaurav G Talloen Joachim H JH Taxer Jamie J Trope Yaacov Y Ungar Lyle L Volpp Kevin G KG Whillans Ashley A Zinman Jonathan J Duckworth Angela L AL

Nature 20211208 7889

Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens' decisions and outcomes<sup>1</sup>. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals<sup>2</sup>. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy-a ma ...[more]

PMID: 34880497

Similar Datasets

Project description:Evidence-based decision-making is most effective with comprehensive access to scientific studies. If studies face significant publication delays or barriers, the useful information they contain may not reach decision-makers in a timely manner. This represents a potential problem for mission-oriented disciplines where access to the latest data is required to ensure effective actions are undertaken. We sought to analyse the severity of publication delay in conservation science-a field that requires urgent action to prevent the loss of biodiversity. We used the Conservation Evidence database to assess the length of publication delay (time from finishing data collection to publication) in the literature that tests the effectiveness of conservation interventions. From 7,447 peer-reviewed and non-peer-reviewed studies of conservation interventions published over eleven decades, we find that the raw mean publication delay was 3.2 years (±2SD = 0.1) and varied by conservation subject. A significantly shorter delay was observed for studies focused on Bee Conservation, Sustainable Aquaculture, Management of Captive Animals, Amphibian Conservation, and Control of Freshwater Invasive Species (Estimated Marginal Mean range from 1.4-1.9 years). Publication delay was significantly shorter for the non-peer-reviewed literature (Estimated Marginal Mean delay of 1.9 years ± 0.2) compared to the peer-reviewed literature (i.e., scientific journals; Estimated Marginal Mean delay of 3.0 years ± 0.1). We found publication delay has significantly increased over time (an increase of ~1.2 years from 1912 (1.4 years ± 0.2) to 2020 (2.6 years ± 0.1)), but this change was much weaker and non-significant post-2000s; we found no evidence for any decline. There was also no evidence that studies on more threatened species were subject to a shorter delay-indeed, the contrary was true for mammals, and to a lesser extent for birds. We suggest a range of possible ways in which scientists, funders, publishers, and practitioners can work together to reduce delays at each stage of the publication process.

Project description:A time-series is any set of N time-ordered observations of a process. In veterinary epidemiology, our focus is generally on disease occurrence (the "process") over time, but animal production, welfare or other traits might also be of interest. A common source of time-series datasets are animal disease monitoring and surveillance systems. Here, we scan the application of methods to analyse time-series data in the peer-reviewed, published literature. Based on this literature scan we focus on autocorrelation and illustrate the recommended steps using ARIMA (Autoregressive Integrated Moving Average Models) methods via analysis of a time-series of canine parvovirus (CPV) events in a pet dog population in Australia, 2009 to 2015. We conclude by identifying the barriers to the application of ARIMA methods in veterinary epidemiology and suggest some possible solutions. In the literature scan the selected 37 studies focused mostly on infectious and parasitic diseases, predominantly for analytical, rather than descriptive or predictive, purposes. Trends and seasonality were investigated, and autocorrelation analyzed, in most studies, most commonly using R software. An approach to analyzing autocorrelation using ARIMA methods was then illustrated using a time-series (week and month units) of CPV events in a pet dog population in Australia, reported to a national companion animal disease surveillance system. This time-series was derived by summing veterinarian reports of confirmed CPV diagnoses. We present data analysis output generated via the R statistical environment, and make this code available for the reader to apply to this or other time-series datasets. We also illustrate prediction of CPV events by rainfall as a covariate. Time-series analysis using ARIMA methods to understand and explore autocorrelation appears to be relatively uncommon in veterinary epidemiology. Some of the reasons might include limited availability of data of sufficient time unit length, lack of familiarity with analytical methods and available software, and how to best use the information generated. We recommend that wherever feasible, such time-series data be made available both for analysis and for methods development.

Project description:BackgroundBreast cancer screening is estimated to save 1300 lives annually in the United Kingdom. Despite this, uptake of invitations has fallen over the past decade. Behavioural science-informed interventions addressing the determinants of attendance behaviour have shown variable effectiveness. This may be due to the narrow repertoire of techniques trialled, and the difficulties of implementation at a population-scale. The aim of this study is to evaluate the impact on breast screening uptake of a novel behavioural video intervention which can contain more complex combinations of behavioural change techniques.MethodsA 3-armed randomised controlled trial will be undertaken in London comparing the impact of (1) the usual care SMS reminder, to (2) a behavioural plain text SMS reminder and (3) a novel video sent as a link within the behavioural plain text SMS reminder. A total of 8391 participants (2797 per group) will be allocated to one of the three trial arms using a computer randomisation process, based upon individuals' healthcare identification numbers. The novel video has been co-designed with a diverse range of women to overcome the barriers faced by underserved communities and the wider population. The behavioural SMS content has also been co-designed through the same process as the video. Messages will be sent through the current reminder system used by the London screening programmes, with reminders 7 days and 2 days prior to a timed appointment. The primary outcome is attendance at breast cancer screening within 3 months of the initial invitation. Secondary outcomes will include evaluating the impact of each message amongst socio-demographic groups and according to the appointment type e.g. first invitation or recall.DiscussionIn addition to general declining trends in attendance, there is also concern of increasing healthcare inequalities with breast cancer screening in London. The current novel intervention, designed with underserved groups and the general population, incorporates several behavioural techniques to overcome the barriers to attendance. Understanding its potential impact in a real-world setting therefore may provide significant information on how to address reducing attendance and healthcare disparities.Trial registrationThis study was registered on ClinicalTrials.gov ( NCT05395871 ) on the 27th May 2022.

Dataset Information

Megastudies improve the impact of applied behavioural science.

Publications

Megastudies improve the impact of applied behavioural science.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets