Dataset Information

Challenges in Estimating the Effectiveness of COVID-19 Vaccination Using Observational Data.

ABSTRACT: The COVID-19 vaccines were developed and rigorously evaluated in randomized trials during 2020. However, important questions, such as the magnitude and duration of protection, their effectiveness against new virus variants, and the effectiveness of booster vaccination, could not be answered by randomized trials and have therefore been addressed in observational studies. Analyses of observational data can be biased because of confounding and because of inadequate design that does not consider the evolution of the pandemic over time and the rapid uptake of vaccination. Emulating a hypothetical "target trial" using observational data assembled during vaccine rollouts can help manage such potential sources of bias. This article describes 2 approaches to target trial emulation. In the sequential approach, on each day, eligible persons who have not yet been vaccinated are matched to a vaccinated person. The single-trial approach sets a single baseline at the start of the rollout and considers vaccination as a time-varying variable. The nature of the confounding depends on the analysis strategy: Estimating "per-protocol" effects (accounting for vaccination of initially unvaccinated persons after baseline) may require adjustment for both baseline and "time-varying" confounders. These issues are illustrated by using observational data from 2 780 931 persons in the United Kingdom aged 70 years or older to estimate the effect of a first dose of a COVID-19 vaccine. Addressing the issues discussed in this article should help authors of observational studies provide robust evidence to guide clinical and policy decisions.

SUBMITTER: Hulme WJ

PROVIDER: S-EPMC10152408 | biostudies-literature | 2023 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Challenges in Estimating the Effectiveness of COVID-19 Vaccination Using Observational Data.

Hulme William J WJ Williamson Elizabeth E Horne Elsie M F EMF Green Amelia A McDonald Helen I HI Walker Alex J AJ Curtis Helen J HJ Morton Caroline E CE MacKenna Brian B Croker Richard R Mehrkar Amir A Bacon Seb S Evans David D Inglesby Peter P Davy Simon S Bhaskaran Krishnan K Schultze Anna A Rentsch Christopher T CT Tomlinson Laurie L Douglas Ian J IJ Evans Stephen J W SJW Smeeth Liam L Palmer Tom T Goldacre Ben B Hernán Miguel A MA Sterne Jonathan A C JAC

Annals of internal medicine 20230502 5

The COVID-19 vaccines were developed and rigorously evaluated in randomized trials during 2020. However, important questions, such as the magnitude and duration of protection, their effectiveness against new virus variants, and the effectiveness of booster vaccination, could not be answered by randomized trials and have therefore been addressed in observational studies. Analyses of observational data can be biased because of confounding and because of inadequate design that does not consider the ...[more]

PMID: 37126810

Similar Datasets

Project description:IntroductionObservational studies of COVID-19 vaccines' effectiveness can provide crucial information regarding the strength and durability of protection against SARS-CoV-2 infection and whether the protective response varies across different patient subpopulations and in the context of different SARS-CoV-2 variants.MethodsWe used a test-negative study design to assess vaccine effectiveness against SARS-CoV-2 infection and severe COVID-19 resulting in hospitalization, intensive care unit admission, or death using electronic health records data of 170,741 adults who had been tested for COVID-19 at the University of Michigan Medical Center between January 1 and December 31, 2021. We estimated vaccine effectiveness by comparing the odds of vaccination between cases and controls during each 2021 calendar quarter and stratified all outcomes by vaccine type, patient demographic and clinical characteristics, and booster status.ResultsUnvaccinated individuals had more than double the rate of infections (12.1% vs 4.7%) and >3 times the rate of severe COVID-19 outcomes (1.4% vs 0.4%) than vaccinated individuals. COVID-19 vaccines were 62.1% (95% CI=60.3, 63.8) effective against a new infection, with protection waning in the last 2 quarters of 2021. The vaccine effectiveness against severe disease overall was 73.7% (95% CI=69.6, 77.3) and remained high throughout 2021. Data from the last quarter of 2021 indicated that adding a booster dose augmented effectiveness against infection up to 87.3% (95% CI=85.0, 89.2) and against severe outcomes up to 94.0% (95% CI=89.5, 96.6). Pfizer-BioNTech and Moderna vaccines showed comparable performance when controlling for vaccination timing. Vaccine effectiveness was greater in more socioeconomically affluent areas and among healthcare workers; otherwise, we did not detect any significant modification of vaccine effectiveness by covariates, including gender, race, and SES.ConclusionsCOVID-19 vaccines were highly protective against infection and severe COVID-19 resulting in hospitalization, intensive care unit admission, or death. Administration of a booster dose significantly increased vaccine effectiveness against both outcomes. Ongoing surveillance is required to assess the durability of these findings.

Project description:Because the real-world impact of new vaccines cannot be known before they are implemented in national programs, post-implementation studies at the population level are critical. Studies based on analysis of hospitalization rates of vaccine-preventable outcomes are typically used for this purpose. However, estimates of vaccine impact based on hospitalization data are particularly prone to confounding, as hospitalization rates are tightly linked to changes in the quality, access and use of the healthcare system, which often occur simultaneously with introduction of new vaccines. Here we illustrate how changes in healthcare delivery coincident with vaccine introduction can influence estimates of vaccine impact, using as an example reductions in infant pneumonia hospitalizations after introduction of the 10-valent pneumococcal conjugate vaccine (PCV10) in Brazil. To this end, we explore the effect of changes in several metrics of quality and access to public healthcare on trends in hospitalization rates before (2008-09) and after (2011-12) PCV10 introduction in 2010. Changes in infant pneumonia hospitalization rates following vaccine introduction were significantly associated with concomitant changes in hospital capacity and the fraction of the population using public hospitals. Importantly, reduction of pneumonia hospitalization rates after PCV10 were also associated with the expansion of outpatient services in several Brazilian states, falling more sharply where primary care coverage and the number of health units offering basic and emergency care increased more. We show that adjustments for unrelated (non-vaccine) trends commonly employed by impact studies, such as use of single control outcomes, are not always sufficient for accurate impact assessment. We discuss several ways to identify and overcome such biases, including sensitivity analyses using different denominators to calculate hospitalizations rates and methods that track changes in the outpatient setting. Employing these practices can improve the accuracy of vaccine impact estimates, particularly in evolving healthcare settings typical of low- and middle-income countries.

Project description:BackgroundOn July 15, 2021, with 58% of the population fully vaccinated, the start of a COVID-19 surge was observed in Puerto Rico. On July 22, 2021, the government of Puerto Rico started imposing a series of strict vaccine mandates. Two months later, over 70% of the population was vaccinated, more than in any US state, and laboratory-confirmed SARS-CoV-2 had dropped substantially. The decision to impose mandates, as well as current Department of Health recommendations related to boosters, were guided by the data and the effectiveness estimates presented here.MethodsBetween December 15, 2020, when the vaccination process began in Puerto Rico, and October 15, 2021, 2,276,966 individuals were fully vaccinated against COVID-19. During this period 112,726 laboratory-confirmed SARS-CoV-2 infections were reported. These data permitted us to quantify the outcomes of the immunization campaign and to compare effectiveness of the mRNA-1273 (Moderna), BNT162b2 (Pfizer), and Ad26.COV2.S (J&J) vaccines. We obtained vaccination status, SARS-CoV-2 test results, and COVID-19 hospitalizations and deaths, from the Department of Health. We fit statistical models that adjusted for time-varying incidence rates and age group to estimate vaccine effectiveness, since the time of vaccination, against lab-confirmed SARS-CoV-2 infection, and COVID-19 hospitalization and death.ResultsTwo weeks after final dose, the mRNA-1273, BNT162b2, and Ad26.COV2.S vaccines had an effectiveness of 90% (95% CI: 88-91), 87% (85-88), and, 64% (58-69), respectively. After five months, effectiveness waned to about 70%, 50%, and 40%, respectively. We found no evidence that effectiveness was different after the Delta variant became dominant. For those infected, the vaccines provided further protection against COVID-19 hospitalization and deaths across all age groups, and this conditional effect did not wane in time.InterpretationThe mRNA-1273 and BNT162b2 vaccines were highly effective across all age groups. They were still effective after five months although the protection against SARS-CoV-2 infection waned. The Ad26.COV2.S vaccine was effective but to a lesser degree compared to the mRNA vaccines. Although, conditional on infection, protection against adverse outcomes did not wane, the waning in effectiveness resulted in a decreased protection against serious COVID-19 outcomes across time.FundingRAI's work was partly funded by NIH Grant R35GM131802.

Project description:IntroductionThe novel coronavirus SARS-CoV-2, which emerged in December 2019, has caused millions of deaths and severe illness worldwide. Numerous vaccines are currently under development of which a few have now been authorised for population-level administration by several countries. As of 20 September 2021, over 48 million people have received their first vaccine dose and over 44 million people have received their second vaccine dose across the UK. We aim to assess the uptake rates, effectiveness, and safety of all currently approved COVID-19 vaccines in the UK.Methods and analysisWe will use prospective cohort study designs to assess vaccine uptake, effectiveness and safety against clinical outcomes and deaths. Test-negative case-control study design will be used to assess vaccine effectiveness (VE) against laboratory confirmed SARS-CoV-2 infection. Self-controlled case series and retrospective cohort study designs will be carried out to assess vaccine safety against mild-to-moderate and severe adverse events, respectively. Individual-level pseudonymised data from primary care, secondary care, laboratory test and death records will be linked and analysed in secure research environments in each UK nation. Univariate and multivariate logistic regression models will be carried out to estimate vaccine uptake levels in relation to various population characteristics. VE estimates against laboratory confirmed SARS-CoV-2 infection will be generated using a generalised additive logistic model. Time-dependent Cox models will be used to estimate the VE against clinical outcomes and deaths. The safety of the vaccines will be assessed using logistic regression models with an offset for the length of the risk period. Where possible, data will be meta-analysed across the UK nations.Ethics and disseminationWe obtained approvals from the National Research Ethics Service Committee, Southeast Scotland 02 (12/SS/0201), the Secure Anonymised Information Linkage independent Information Governance Review Panel project number 0911. Concerning English data, University of Oxford is compliant with the General Data Protection Regulation and the National Health Service (NHS) Digital Data Security and Protection Policy. This is an approved study (Integrated Research Application ID 301740, Health Research Authority (HRA) Research Ethics Committee 21/HRA/2786). The Oxford-Royal College of General Practitioners Clinical Informatics Digital Hub meets NHS Digital's Data Security and Protection Toolkit requirements. In Northern Ireland, the project was approved by the Honest Broker Governance Board, project number 0064. Findings will be made available to national policy-makers, presented at conferences and published in peer-reviewed journals.

Dataset Information

Challenges in Estimating the Effectiveness of COVID-19 Vaccination Using Observational Data.

Publications

Challenges in Estimating the Effectiveness of COVID-19 Vaccination Using Observational Data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets