Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Improving trial generalizability using observational studies.

ABSTRACT: Complementary features of randomized controlled trials (RCTs) and observational studies (OSs) can be used jointly to estimate the average treatment effect of a target population. We propose a calibration weighting estimator that enforces the covariate balance between the RCT and OS, therefore improving the trial-based estimator's generalizability. Exploiting semiparametric efficiency theory, we propose a doubly robust augmented calibration weighting estimator that achieves the efficiency bound derived under the identification assumptions. A nonparametric sieve method is provided as an alternative to the parametric approach, which enables the robust approximation of the nuisance functions and data-adaptive selection of outcome predictors for calibration. We establish asymptotic results and confirm the finite sample performances of the proposed estimators by simulation experiments and an application on the estimation of the treatment effect of adjuvant chemotherapy for early-stage non-small-cell lung patients after surgery.

SUBMITTER: Lee D

PROVIDER: S-EPMC9166225 | biostudies-literature | 2021 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Improving trial generalizability using observational studies.

Lee Dasom D Yang Shu S Dong Lin L Wang Xiaofei X Zeng Donglin D Cai Jianwen J

Biometrics 20220111 2

Complementary features of randomized controlled trials (RCTs) and observational studies (OSs) can be used jointly to estimate the average treatment effect of a target population. We propose a calibration weighting estimator that enforces the covariate balance between the RCT and OS, therefore improving the trial-based estimator's generalizability. Exploiting semiparametric efficiency theory, we propose a doubly robust augmented calibration weighting estimator that achieves the efficiency bound d ...[more]

PMID: 34862966

Similar Datasets

Selective recruitment designs for improving observational studies using electronic health records.

Project description:Large-scale electronic health records (EHRs) present an opportunity to quickly identify suitable individuals in order to directly invite them to participate in an observational study. EHRs can contain data from millions of individuals, raising the question of how to optimally select a cohort of size n from a larger pool of size N. In this article, we propose a simple selective recruitment protocol that selects a cohort in which covariates of interest tend to have a uniform distribution. We show that selectively recruited cohorts potentially offer greater statistical power and more accurate parameter estimates than randomly selected cohorts. Our protocol can be applied to studies with multiple categorical and continuous covariates. We apply our protocol to a numerically simulated prospective observational study using an EHR database of stable acute coronary disease patients from 82 089 individuals in the U.K. Selective recruitment designs require a smaller sample size, leading to more efficient and cost-effective studies.

| S-EPMC8432147 | biostudies-literature

Improving reproducibility by using high-throughput observational studies with empirical calibration.

Project description:Concerns over reproducibility in science extend to research using existing healthcare data; many observational studies investigating the same topic produce conflicting results, even when using the same data. To address this problem, we propose a paradigm shift. The current paradigm centres on generating one estimate at a time using a unique study design with unknown reliability and publishing (or not) one estimate at a time. The new paradigm advocates for high-throughput observational studies using consistent and standardized methods, allowing evaluation, calibration and unbiased dissemination to generate a more reliable and complete evidence base. We demonstrate this new paradigm by comparing all depression treatments for a set of outcomes, producing 17 718 hazard ratios, each using methodology on par with current best practice. We furthermore include control hypotheses to evaluate and calibrate our evidence generation process. Results show good transitivity and consistency between databases, and agree with four out of the five findings from clinical trials. The distribution of effect size estimates reported in the literature reveals an absence of small or null effects, with a sharp cut-off at p = 0.05. No such phenomena were observed in our results, suggesting more complete and more reliable evidence.This article is part of a discussion meeting issue 'The growing ubiquity of algorithms in society: implications, impacts and innovations'.

| S-EPMC6107542 | biostudies-literature

Generalizability of Clinical Trial Results for Adolescent Major Depressive Disorder.

Project description:Although there have been a number of clinical trials evaluating treatments for adolescents with major depressive disorder (MDD), the generalizability of those trials to samples of depressed adolescents who present for routine clinical care is unknown. Examining the generalizability of clinical trials of pharmacological and psychotherapy interventions for adolescent depression can help administrators and frontline practitioners determine the relevance of these studies for their patients and may also guide eligibility criteria for future clinical trials in this clinical population.Data on nationally representative adolescents were derived from the National Comorbidity Survey: Adolescent Supplement. To assess the generalizability of adolescent clinical trials for MDD, we applied a standard set of eligibility criteria representative of clinical trials to all adolescents in the National Comorbidity Survey: Adolescent Supplement with a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition diagnosis of MDD (N = 592).From the overall MDD sample, 61.9% would have been excluded from a typical pharmacological trial, whereas 42.2% would have been excluded from a psychotherapy trial. Among those who sought treatment (n = 412), the corresponding exclusion rates were 72.7% for a pharmacological trial and 52.2% for a psychotherapy trial. The criterion leading to the largest number of exclusions was "significant risk of suicide" in both pharmacological and psychotherapy trials.Pharmacological and, to a lesser extent, psychotherapy clinical trials likely exclude most adolescents with MDD. Careful consideration should be given to balancing eligibility criteria and internal validity with applicability in routine clinical care while ensuring patient safety.

| S-EPMC5703774 | biostudies-other

Evaluating generalizability of oncology trial results to real-world patients using machine learning-based trial emulations.

Project description:Randomized controlled trials (RCTs) evaluating anti-cancer agents often lack generalizability to real-world oncology patients. Although restrictive eligibility criteria contribute to this issue, the role of selection bias related to prognostic risk remains unclear. In this study, we developed TrialTranslator, a framework designed to systematically evaluate the generalizability of RCTs for oncology therapies. Using a nationwide database of electronic health records from Flatiron Health, this framework emulates RCTs across three prognostic phenotypes identified through machine learning models. We applied this approach to 11 landmark RCTs that investigated anti-cancer regimens considered standard of care for the four most prevalent advanced solid malignancies. Our analyses reveal that patients in low-risk and medium-risk phenotypes exhibit survival times and treatment-associated survival benefits similar to those observed in RCTs. In contrast, high-risk phenotypes show significantly lower survival times and treatment-associated survival benefits compared to RCTs. Our results were corroborated by a comprehensive robustness assessment, including examinations of specific patient subgroups, holdout validation and semi-synthetic data simulation. These findings suggest that the prognostic heterogeneity among real-world oncology patients plays a substantial role in the limited generalizability of RCT results. Machine learning frameworks may facilitate individual patient-level decision support and estimation of real-world treatment benefits to guide trial design.

| S-EPMC11835724 | biostudies-literature

Yuel: Improving the Generalizability of Structure-Free Compound-Protein Interaction Prediction.

Project description:Predicting binding affinities between small molecules and the protein target is at the core of computational drug screening and drug target identification. Deep learning-based approaches have recently been adapted to predict binding affinities and they claim to achieve high prediction accuracy in their tests; we show that these approaches do not generalize, that is, they fail to predict interactions between unknown proteins and unknown small molecules. To address these shortcomings, we develop a new compound-protein interaction predictor, Yuel, which predicts compound-protein interactions with a higher generalizability than the existing methods. Upon comprehensive tests on various data sets, we find that out of all the deep-learning approaches surveyed, Yuel manifests the best ability to predict interactions between unknown compounds and unknown proteins.

| S-EPMC9203246 | biostudies-literature

Improving the generalizability of protein-ligand binding predictions with AI-Bind.

Project description:Identifying novel drug-target interactions is a critical and rate-limiting step in drug discovery. While deep learning models have been proposed to accelerate the identification process, here we show that state-of-the-art models fail to generalize to novel (i.e., never-before-seen) structures. We unveil the mechanisms responsible for this shortcoming, demonstrating how models rely on shortcuts that leverage the topology of the protein-ligand bipartite network, rather than learning the node features. Here we introduce AI-Bind, a pipeline that combines network-based sampling strategies with unsupervised pre-training to improve binding predictions for novel proteins and ligands. We validate AI-Bind predictions via docking simulations and comparison with recent experimental evidence, and step up the process of interpreting machine learning prediction of protein-ligand binding by identifying potential active binding sites on the amino acid sequence. AI-Bind is a high-throughput approach to identify drug-target combinations with the potential of becoming a powerful tool in drug discovery.

| S-EPMC10082765 | biostudies-literature

Improving the Generalizability of Convolutional Neural Network-Based Segmentation on CMR Images.

Project description:Background: Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g., same scanner or site), their performance often degrades dramatically on images from different scanners or clinical sites. Methods: We propose a simple yet effective way for improving the network generalization ability by carefully designing data normalization and augmentation strategies to accommodate common scenarios in multi-site, multi-scanner clinical imaging data sets. We demonstrate that a neural network trained on a single-site single-scanner dataset from the UK Biobank can be successfully applied to segmenting cardiac MR images across different sites and different scanners without substantial loss of accuracy. Specifically, the method was trained on a large set of 3,975 subjects from the UK Biobank. It was then directly tested on 600 different subjects from the UK Biobank for intra-domain testing and two other sets for cross-domain testing: the ACDC dataset (100 subjects, 1 site, 2 scanners) and the BSCMR-AS dataset (599 subjects, 6 sites, 9 scanners). Results: The proposed method produces promising segmentation results on the UK Biobank test set which are comparable to previously reported values in the literature, while also performing well on cross-domain test sets, achieving a mean Dice metric of 0.90 for the left ventricle, 0.81 for the myocardium, and 0.82 for the right ventricle on the ACDC dataset; and 0.89 for the left ventricle, 0.83 for the myocardium on the BSCMR-AS dataset. Conclusions: The proposed method offers a potential solution to improve CNN-based model generalizability for the cross-scanner and cross-site cardiac MR image segmentation task.

| S-EPMC7344224 | biostudies-literature

Improving Clinical Trial Participant Tracking Tools Using Knowledge-anchored Design Methodologies.

Project description:OBJECTIVE: Rigorous human-computer interaction (HCI) design methodologies have not traditionally been applied to the development of clinical trial participant tracking (CTPT) tools. Given the frequent us of iconic HCI models in CTPTs, and prior evidence of usability problems associated with the use of ambiguous icons in complex interfaces, such approaches may be problematic. Presentation Discovery (PD), a knowledge-anchored HCI design method, has been previously demonstrated to improve the design of iconic HCI models. In this study, we compare the usability of a CTPT HCI model designed using PD and an intuitively designed CTPT HCI model. METHODS: An iconic CPTP HCI model was created using PD. The PD-generated and an existing iconic CTPT HCI model were subjected to usability testing, with an emphasis on task accuracy and completion times. Study participants also completed a qualitative survey instrument to evaluate subjective satisfaction with the two models. RESULTS: CTPT end-users reliably and reproducibly agreed on the visual manifestation and semantics of prototype graphics generated using PD. The performance of the PD-generated iconic HCI model was equivalent to an existing HCI model for tasks at multiple levels of complexity, and in some cases superior. This difference was particularly notable when tasks required an understanding of the semantic meanings of multiple icons. CONCLUSION: The use of PD to design an iconic CTPT HCI model generated beneficial results and improved end-user subjective satisfaction, while reducing task completion time. Such results are desirable in information and time intensive domains, such as clinical trials management.

| S-EPMC3225206 | biostudies-other

Improving data quality in observational research studies: Report of the Cure Glomerulonephropathy (CureGN) network.

Project description:BackgroundHigh data quality is of crucial importance to the integrity of research projects. In the conduct of multi-center observational cohort studies with increasing types and quantities of data, maintaining data quality is challenging, with few published guidelines.MethodsThe Cure Glomerulonephropathy (CureGN) Network has established numerous quality control procedures to manage the 70 participating sites in the United States, Canada, and Europe. This effort is supported and guided by the activities of several committees, including Data Quality, Recruitment and Retention, and Central Review, that work in tandem with the Data Coordinating Center to monitor the study. We have implemented coordinator training and feedback channels, data queries of questionable or missing data, and developed performance metrics for recruitment, retention, visit completion, data entry, recording of patient-reported outcomes, collection, shipping and accessing of biological samples and pathology materials, and processing, cataloging and accessing genetic data and materials.ResultsWe describe the development of data queries and site Report Cards, and their use in monitoring and encouraging excellence in site performance. We demonstrate improvements in data quality and completeness over 4 years after implementing these activities. We describe quality initiatives addressing specific challenges in collecting and cataloging whole slide images and other kidney pathology data, and novel methods of data quality assessment.ConclusionsThis paper reports the CureGN experience in optimizing data quality and underscores the importance of general and study-specific data quality initiatives to maintain excellence in the research measures of a multi-center observational study.

| S-EPMC8039553 | biostudies-literature

Using Mendelian randomisation to assess causality in observational studies.

Project description:ObjectiveMendelian randomisation (MR) is a technique that aims to assess causal effects of exposures on disease outcomes. The paper aims to present the main assumptions that underlie MR, the statistical methods used to estimate causal effects and how to account for potential violations of the key assumptions.MethodsWe discuss the key assumptions that should be satisfied in an MR setting. We list the statistical methodologies used in two-sample MR when summary data are available to estimate causal effects (ie, Wald ratio estimator, inverse-variance weighted and maximum likelihood method) and identify/adjust for potential violations of MR assumptions (ie, MR-Egger regression and weighted Median approach). We also present statistical methods and graphical tools used to evaluate the presence of heterogeneity.ResultsWe use as an illustrative example of a published two-sample MR study, investigating the causal association of body mass index with three psychiatric disorders (ie, bipolar disorder, schizophrenia and major depressive disorder). We highlight the importance of assessing the results of all available methods rather than each method alone. We also demonstrate the impact of heterogeneity in the estimation of the causal effects.ConclusionsMR is a useful tool to assess causality of risk factors in medical research. Assessment of the key assumptions underlying MR is crucial for a valid interpretation of the results.

| S-EPMC10270458 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data