A Framework for Systematic Assessment of Clinical Trial Population Representativeness Using Electronic Health Records Data.
Ontology highlight
ABSTRACT: Background
Clinical trials are the gold standard for generating robust medical evidence, but clinical trial results often raise generalizability concerns, which can be attributed to the lack of population representativeness. The electronic health records (EHRs) data are useful for estimating the population representativeness of clinical trial study population.Objectives
This research aims to estimate the population representativeness of clinical trials systematically using EHR data during the early design stage.Methods
We present an end-to-end analytical framework for transforming free-text clinical trial eligibility criteria into executable database queries conformant with the Observational Medical Outcomes Partnership Common Data Model and for systematically quantifying the population representativeness for each clinical trial.Results
We calculated the population representativeness of 782 novel coronavirus disease 2019 (COVID-19) trials and 3,827 type 2 diabetes mellitus (T2DM) trials in the United States respectively using this framework. With the use of overly restrictive eligibility criteria, 85.7% of the COVID-19 trials and 30.1% of T2DM trials had poor population representativeness.Conclusion
This research demonstrates the potential of using the EHR data to assess the clinical trials population representativeness, providing data-driven metrics to inform the selection and optimization of eligibility criteria.
SUBMITTER: Sun Y
PROVIDER: S-EPMC8426045 | biostudies-literature | 2021 Aug
REPOSITORIES: biostudies-literature
Publications
A Framework for Systematic Assessment of Clinical Trial Population Representativeness Using Electronic Health Records Data.
Sun Yingcheng Y Butler Alex A Diallo Ibrahim I Kim Jae Hyun JH Ta Casey C Rogers James R JR Liu Hao H Weng Chunhua C
Applied clinical informatics 20210801 4
<h4>Background</h4>Clinical trials are the gold standard for generating robust medical evidence, but clinical trial results often raise generalizability concerns, which can be attributed to the lack of population representativeness. The electronic health records (EHRs) data are useful for estimating the population representativeness of clinical trial study population.<h4>Objectives</h4>This research aims to estimate the population representativeness of clinical trials systematically using EHR da ...[more]
Similar Datasets
Project description:ObjectiveThe development of machine learning (ML) algorithms to address a variety of issues faced in clinical practice has increased rapidly. However, questions have arisen regarding biases in their development that can affect their applicability in specific populations. We sought to evaluate whether studies developing ML models from electronic health record (EHR) data report sufficient demographic data on the study populations to demonstrate representativeness and reproducibility.Materials and methodsWe searched PubMed for articles applying ML models to improve clinical decision-making using EHR data. We limited our search to papers published between 2015 and 2019.ResultsAcross the 164 studies reviewed, demographic variables were inconsistently reported and/or included as model inputs. Race/ethnicity was not reported in 64%; gender and age were not reported in 24% and 21% of studies, respectively. Socioeconomic status of the population was not reported in 92% of studies. Studies that mentioned these variables often did not report if they were included as model inputs. Few models (12%) were validated using external populations. Few studies (17%) open-sourced their code. Populations in the ML studies include higher proportions of White and Black yet fewer Hispanic subjects compared to the general US population.DiscussionThe demographic characteristics of study populations are poorly reported in the ML literature based on EHR data. Demographic representativeness in training data and model transparency is necessary to ensure that ML models are deployed in an equitable and reproducible manner. Wider adoption of reporting guidelines is warranted to improve representativeness and reproducibility.
| S-EPMC7727384 | biostudies-literature
Project description:BackgroundHypertension is the most prevalent risk factor for mortality globally. Uncontrolled hypertension is associated with excess morbidity and mortality, and nearly one-half of individuals with hypertension do not have the condition under control. Data from electronic health record (EHR) systems may be useful for community hypertension surveillance, filling a gap in local public health departments' community health assessments and supporting the public health data modernization initiatives currently underway. To identify patients with hypertension, computable phenotypes are required. These phenotypes leverage available data elements-such as vitals measurements and medications-to identify patients diagnosed with hypertension. However, there are multiple methodologies for creating a phenotype, and the identification of which method most accurately reflects real-world prevalence rates is needed to support data modernization initiatives.ObjectiveThis study sought to assess the comparability of 6 different EHR-based hypertension prevalence estimates with estimates from a national survey. Each of the prevalence estimates was created using a different computable phenotype. The overarching goal is to identify which phenotypes most closely align with nationally accepted estimations.MethodsUsing the 6 different EHR-based computable phenotypes, we calculated hypertension prevalence estimates for Marion County, Indiana, for the period from 2014 to 2015. We extracted hypertension rates from the Behavioral Risk Factor Surveillance System (BRFSS) for the same period. We used the two 1-sided t test (TOST) to test equivalence between BRFSS- and EHR-based prevalence estimates. The TOST was performed at the overall level as well as stratified by age, gender, and race.ResultsUsing both 80% and 90% CIs, the TOST analysis resulted in 2 computable phenotypes demonstrating rough equivalence to BRFSS estimates. Variation in performance was noted across phenotypes as well as demographics. TOST with 80% CIs demonstrated that the phenotypes had less variance compared to BRFSS estimates within subpopulations, particularly those related to racial categories. Overall, less variance occurred on phenotypes that included vitals measurements.ConclusionsThis study demonstrates that certain EHR-derived prevalence estimates may serve as rough substitutes for population-based survey estimates. These outcomes demonstrate the importance of critically assessing which data elements to include in EHR-based computer phenotypes. Using comprehensive data sources, containing complete clinical data as well as data representative of the population, are crucial to producing robust estimates of chronic disease. As public health departments look toward data modernization activities, the EHR may serve to assist in more timely, locally representative estimates for chronic disease prevalence.
| S-EPMC10973965 | biostudies-literature
Project description:BackgroundPatient access to electronic health records (EHRs) is associated with increased patient engagement and health care quality outcomes. However, the adoption of patient portals and personal health records (PHRs) that facilitate this access is impeded by barriers. The Clinical Adoption Framework (CAF) has been developed to analyze EHR adoption, but this framework does not consider the patient as an end-user.ObjectiveWe aim to extend the scope of the CAF to patient access to EHRs, develop guidance documentation for the application of the CAF, and assess the interrater reliability.MethodsWe systematically reviewed existing systematic reviews on patients' access to EHRs and PHRs. Results of each review were mapped to one of the 43 CAF categories. Categories were iteratively adapted when needed. We measured the interrater reliability with Cohen's unweighted kappa and statistics regarding the agreement among reviewers on mapping quotes of the reviews to different CAF categories.ResultsWe further defined the framework's inclusion and exclusion criteria for 33 of the 43 CAF categories and achieved a moderate agreement among the raters, which varied between categories.ConclusionsIn the reviews, categories about people, organization, system quality, system use, and the net benefits of system use were addressed more often than those about international and regional information and communication technology infrastructures, standards, politics, incentive programs, and social trends. Categories that were addressed less might have been underdefined in this study. The guidance documentation we developed can be applied to systematic literature reviews and implementation studies, patient and informal caregiver access to EHRs, and the adoption of PHRs.
| S-EPMC7154932 | biostudies-literature
Project description:Low-Density Lipoprotein (LDL) cholesterol is one of the main target for cardiovascular (CV) prevention and therapy. In the last years, Proprotein Convertase Subtilisin-Kexin type 9 inhibitors (PCSK9-i) has emerged as a key therapeutic target to lower LDL and were introduced for prevention of CV events. Recently (June 2022) the Italian Medicines Agency (AIFA) modified the eligibility criteria for the use of PCSK9-i. We designed an observational study to estimate the prevalence of eligible subjects and evaluate the effectiveness of PCSK9-i applying a Target Trial Emulation (TTE) approach based on Electronic Health Records (EHR). Subjects meeting the eligibility criteria were identified from July 2017 (when PCSK9-i became available) to December 2020. Outcomes were all-cause death and the first hospitalization. Among eligible subjects, we identified those treated at date of the first prescription. Inverse Probability of Treatment Weights (IPTW) were estimated including demographic and clinical covariates, history of treatment with statins and the month/year eligibility date. Competing risk models on weighted cohorts were used to derive the Average Treatment Effect (ATE) and the Conditional Average Treatment Effect (CATE) in subgroups of interest. Out of 1976 eligible subjects, 161 (8%) received treatment with PCSK9-i. Treated individuals were slightly younger, predominantly male, had more severe CV conditions, and were more often treated with statin compared to the untreated subjects. The latter exhibited a higher prevalence of non-CV comorbidities. A significant absolute and relative risk reduction of death and a lower relative risk for the first hospitalization was observed. The risk reduction for death was confirmed in CATE analysis. PCSk9-i were prescribed to a minority of eligible subjects. Within the TTE framework, the analysis confirmed the association between PCSK9-i and lower risk of events, aligning with findings from randomized clinical trials (RCTs). In our study, PCSK9-i provided protection specifically against all-cause death, expanding upon the evidence from RCTs that had primarily focused on composite CV outcomes.
| S-EPMC11341039 | biostudies-literature
Project description:The rise of genomically targeted therapies and immunotherapy has revolutionized the practice of oncology in the last 10-15 years. At the same time, new technologies and the electronic health record (EHR) in particular have permeated the oncology clinic. Initially designed as billing and clinical documentation systems, EHR systems have not anticipated the complexity and variety of genomic information that needs to be reviewed, interpreted, and acted upon on a daily basis. Improved integration of cancer genomic data with EHR systems will help guide clinician decision making, support secondary uses, and ultimately improve patient care within oncology clinics. Some of the key factors relating to the challenge of integrating cancer genomic data into EHRs include: the bioinformatics pipelines that translate raw genomic data into meaningful, actionable results; the role of human curation in the interpretation of variant calls; and the need for consistent standards with regard to genomic and clinical data. Several emerging paradigms for integration are discussed in this review, including: non-standardized efforts between individual institutions and genomic testing laboratories; "middleware" products that portray genomic information, albeit outside of the clinical workflow; and application programming interfaces that have the potential to work within clinical workflow. The critical need for clinical-genomic knowledge bases, which can be independent or integrated into the aforementioned solutions, is also discussed.
| S-EPMC5081968 | biostudies-literature
Project description:BackgroundDigitization offers a multitude of opportunities to gain insights into current diagnostics and therapies from retrospective data. In this context, real-world data and their accessibility are of increasing importance to support unbiased and reliable research on big data. However, routinely collected data are not readily usable for research owing to the unstructured nature of health care systems and a lack of interoperability between these systems. This challenge is evident in drug data.ObjectiveThis study aimed to present an approach that identifies and increases the structuredness of drug data while ensuring standardization according to Anatomical Therapeutic Chemical (ATC) classification.MethodsOur approach was based on available drug prescriptions and a drug catalog and consisted of 4 steps. First, we performed an initial analysis of the structuredness of local drug data to define a point of comparison for the effectiveness of the overall approach. Second, we applied 3 algorithms to unstructured data that translated text into ATC codes based on string comparisons in terms of ingredients and product names and performed similarity comparisons based on Levenshtein distance. Third, we validated the results of the 3 algorithms with expert knowledge based on the 1000 most frequently used prescription texts. Fourth, we performed a final validation to determine the increased degree of structuredness.ResultsInitially, 47.73% (n=843,980) of 1,768,153 drug prescriptions were classified as structured. With the application of the 3 algorithms, we were able to increase the degree of structuredness to 85.18% (n=1,506,059) based on the 1000 most frequent medication prescriptions. In this regard, the combination of algorithms 1, 2, and 3 resulted in a correctness level of 100% (with 57,264 ATC codes identified), algorithms 1 and 3 resulted in 99.6% (with 152,404 codes identified), and algorithms 1 and 2 resulted in 95.9% (with 39,472 codes identified).ConclusionsAs shown in the first analysis steps of our approach, the availability of a product catalog to select during the documentation process is not sufficient to generate structured data. Our 4-step approach reduces the problems and reliably increases the structuredness automatically. Similarity matching shows promising results, particularly for entries with no connection to a product catalog. However, further enhancement of the correctness of such a similarity matching algorithm needs to be investigated in future work.
| S-EPMC9909518 | biostudies-literature
Project description:BACKGROUND:Poor EHR design adds further challenges, especially in the areas of order entry and information visualization, with a net effect of increased rates of incidents, accidents, and mortality in ICU settings. OBJECTIVE:The purpose of this study was to propose a novel, mixed-methods framework to understand EHR-related information overload by identifying and characterizing areas of suboptimal usability and clinician frustration within a vendor-based, provider-facing EHR interface. METHODS:A mixed-methods, live observational usability study was conducted at a single, large, tertiary academic medical center in the Southeastern US utilizing a commercial, vendor based EHR. Physicians were asked to complete usability patient cases, provide responses to three surveys, and participant in a semi-structured interview. RESULTS:Of the 25 enrolled ICU physician participants, there were 5(20%) attending physicians, 9 (36%) fellows, and 11 (44%) residents; 52% of participants were females. On average, residents were the quickest in completing the tasks while attending physician took the longest to complete the same task. Poor usability, complex interface screens, and difficulty to navigate the EHR significantly correlated with high frustration levels. Significant association were found between the occurrence of error messages and temporal demand such that more error messages resulted in longer completion time (p?=?.03). DISCUSSION:Physicians remain frustrated with the EHR due to difficulty in finding patient information. EHR usability remains a critical challenge in healthcare, with implications for medical errors, patient safety, and clinician burnout. There is a need for scientific findings on current information needs and ways to improve EHR-related information overload.
| S-EPMC6584062 | biostudies-literature
Project description:BackgroundAs the availability of interoperable electronic health records (iEHRs) or health information exchanges (HIEs) continues to increase, there is greater need and opportunity to assess the current evidence base on what works and what does not regarding the adoption, use, and impact of iEHRs.ObjectiveThe purpose of this project is to assess the international evidence base on the adoption, use, and impact of iEHRs.MethodsWe conducted a systematic review, searching multiple databases-MEDLINE, Embase, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL)-with supplemental searches conducted in Google Scholar and grey literature sources (ie, Google, Grey Literature Report, and OpenGrey). All searches were conducted in January and February 2017. Articles were eligible for inclusion if they were published in English, were published from 2006 to 2017, and were either an original research study or a literature review. In order to be included, articles needed to focus on iEHRs and HIEs across multiple health care settings, as well as on the impact and effectiveness of iEHR adoption and use.ResultsWe included 130 articles in the synthesis (113 primary studies, 86.9%; 17 reviews, 13.1%), with the majority focused on the United States (88/130, 67.7%). The primary studies focused on a wide range of health care settings; the three most prevalent settings studied included acute care (59/113, 52.2%), primary care (44/113, 38.9%), and emergency departments (34/113, 30.1%). We identified 29 distinct measurement items in the 113 primary studies that were linked to 522 specific measurement outcomes. Productivity and quality were the two evaluation dimensions that received the most attention, accounting for 14 of 29 (48%) measurement items and 306 of 522 (58.6%) measurement outcomes identified. Overall, the majority of the 522 measurement outcomes were positive (298/522, 57.1%). We also identified 17 reviews on iEHR use and impact, 6 (35%) that focused on barriers and facilitators to adoption and implementation and 11 (65%) that focused on benefits and impacts, with the more recent reviews finding little generalizable evidence of benefit and impact.ConclusionsThis review captures the status of an evolving and active field focused on the use and impact of iEHRs. While the overall findings suggest many positive impacts, the quality of the primary studies were not evaluated systematically. When broken down by specific measurement item, the results directed attention both to measurement outcomes that were consistently positive and others that were mostly negative or equivocal.
| S-EPMC6592487 | biostudies-literature
Project description:ObjectiveWe extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.Materials and methodsWe completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.ResultsWe included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.DiscussionThere has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.ConclusionGuidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.
| S-EPMC10531113 | biostudies-literature
Project description:ObjectivesThis study aims to investigate whether different types of electronic health record (EHR) users have distinct preferences for data quality assessment indicators (DQAI) and explore how these preferences can guide the enhancement of EHR systems and the optimization of related policies.Materials and methodsHigh-frequency indicators were identified by a systematic literature review to construct a DQAI system, which was assessed by a user-oriented investigation involving doctors, nurses, hospital supervisors, and clinical researchers. The entropy weight method and fuzzy comprehensive evaluation model were employed for the system comprehensive evaluation. Exploratory factor analysis was used to construct dimensions, and visualization analysis was utilized to explore preferences at both the indicator and dimension levels.ResultsSixteen indicators were identified to construct the DQAI system and grouped into 2 dimensions: structural and relational. The DQAI system achieved a comprehensive evaluation score of 90.445, corresponding to a "very important" membership level (62.5%). Doctors and nurses exhibited a higher score mean (4.43-4.66 out of 5) than supervisors (3.73-4.55 out of 5). Researchers emphasized credibility, with a score mean of 4.79 out of 5.DiscussionThe findings reveal that different types of EHR users exhibit distinct preferences for the DQAI at both indicator and dimension levels. Doctors and nurses thought that all indicators were important, clinical researchers emphasized credibility, and supervisors focused mainly on accuracy. Indicators in the relational dimension were generally more valued than structural ones. Doctors and nurses prioritized indicators of relational dimension, while researchers and supervisors leaned towards indicators of structural dimension. These insights suggest that tailored approaches in EHR system development and policy-making could enhance EHR data quality.ConclusionThis study underscores the importance of user-centered approaches in optimizing EHR systems, highlighting diverse user preferences at both indicator and dimension levels.
| S-EPMC11633948 | biostudies-literature