A design framework and exemplar metrics for FAIRness.
Ontology highlight
ABSTRACT:
SUBMITTER: Wilkinson MD
PROVIDER: S-EPMC6018520 | biostudies-literature | 2018 Jun
REPOSITORIES: biostudies-literature
Publications
A design framework and exemplar metrics for FAIRness.
Wilkinson Mark D MD Sansone Susanna-Assunta SA Schultes Erik E Doorn Peter P Bonino da Silva Santos Luiz Olavo LO Dumontier Michel M
Scientific data 20180626
Similar Datasets
Project description:ObjectiveWe help identify subpopulations underrepresented in randomized clinical trials (RCTs) cohorts with respect to national, community-based or health system target populations by formulating population representativeness of RCTs as a machine learning (ML) fairness problem, deriving new representation metrics, and deploying them in easy-to-understand interactive visualization tools.Materials and methodsWe represent RCT cohort enrollment as random binary classification fairness problems, and then show how ML fairness metrics based on enrollment fraction can be efficiently calculated using easily computed rates of subpopulations in RCT cohorts and target populations. We propose standardized versions of these metrics and deploy them in an interactive tool to analyze 3 RCTs with respect to type 2 diabetes and hypertension target populations in the National Health and Nutrition Examination Survey.ResultsWe demonstrate how the proposed metrics and associated statistics enable users to rapidly examine representativeness of all subpopulations in the RCT defined by a set of categorical traits (eg, gender, race, ethnicity, smoking status, and blood pressure) with respect to target populations.DiscussionThe normalized metrics provide an intuitive standardized scale for evaluating representation across subgroups, which may have vastly different enrollment fractions and rates in RCT study cohorts. The metrics are beneficial complements to other approaches (eg, enrollment fractions) used to identify generalizability and health equity of RCTs.ConclusionBy quantifying the gaps between RCT and target populations, the proposed methods can support generalizability evaluation of existing RCT cohorts. The interactive visualization tool can be readily applied to identified underrepresented subgroups with respect to any desired source or target populations.
| S-EPMC8460438 | biostudies-literature
Project description:Translational science requires that scientists from multiple disciplines work together to improve the prevention, diagnosis, and treatment of human disease. Although a literature exists on the design and management of multidisciplinary teams, little has been written on multidisciplinary translational teams (MTTs). MTTs are distinct hybrid entities, with goals taken from both industry and academic models. We identified 30 design factors in 10 domains from a literature survey relevant to our MTT model: specific goals, structures, and processes. These dimensions were adapted to our own institutional environment in the selection and management of 11 MTTs that exploited resources of University of Texas Medical Branch (UTMB) Clinical and Translational Sciences Awards (CTSA). Case illustrations of two specific MTTs illustrate some of the challenges encountered and opportunities realized in terms of education and scientific advances. Network depiction of disciplinarity indicated that CTSA KRs and CTSA leadership contributed to discipline diversity especially in small (or nascent) MTTs. A separate depiction of MTT-KR utilization indicated that data analysis, translational technologies, and novel methods were heavily utilized by MTTs, whereas other KRs contributed significant effort to infrastructure development. We conclude that the CTSA can provide a rich infrastructural framework and scientific environment for the development of successful MTTs.
| S-EPMC3786412 | biostudies-literature
Project description: Not available
| S-EPMC6908569 | biostudies-literature
Project description:A strong competency and milestone framework is imperative for medical schools adopting competency-based education and assessment. Milestones can be used to align what is taught and what students are asked to demonstrate from matriculation to graduation. We describe the creation and implementation of our milestone framework as an exemplar.Supplementary informationThe online version contains supplementary material available at 10.1007/s40670-022-01558-1.
| S-EPMC9055212 | biostudies-literature
Project description:Biodiversity mitigation is a cornerstone of applied conservation. Mitigation encompasses a suite of practices, ranging from planned avoidance of impacts to creation of new natural habitats. Accurate and appropriate metrics quantifying impacts to natural systems and the effectiveness of restoration are necessary to measure the success of different mitigation strategies. Because effective mitigation requires adequate metrics, we developed a Biodiversity Metrics Framework to assist practitioners and policy makers in assessing biodiversity mitigation metrics. Based on Noss' Hierarchy of Biodiversity, the Scorecard highlights the mismatch between scientifically defined best practices and metrics required by policy. The Framework may serve a vital role in standardizing and validating mitigation projects into the future.
| S-EPMC7128020 | biostudies-literature
Project description:Questions of unfairness and inequity pose critical challenges to the successful deployment of artificial intelligence (AI) in healthcare settings. In AI models, unequal performance across protected groups may be partially attributable to the learning of spurious or otherwise undesirable correlations between sensitive attributes and disease-related information. Here, we introduce the Attribute Neutral Framework, designed to disentangle biased attributes from disease-relevant information and subsequently neutralize them to improve representation across diverse subgroups. Within the framework, we develop the Attribute Neutralizer (AttrNzr) to generate neutralized data, for which protected attributes can no longer be easily predicted by humans or by machine learning classifiers. We then utilize these data to train the disease diagnosis model (DDM). Comparative analysis with other unfairness mitigation algorithms demonstrates that AttrNzr outperforms in reducing the unfairness of the DDM while maintaining DDM's overall disease diagnosis performance. Furthermore, AttrNzr supports the simultaneous neutralization of multiple attributes and demonstrates utility even when applied solely during the training phase, without being used in the test phase. Moreover, instead of introducing additional constraints to the DDM, the AttrNzr directly addresses a root cause of unfairness, providing a model-independent solution. Our results with AttrNzr highlight the potential of data-centered and model-independent solutions for fairness challenges in AI-enabled medical systems.
| S-EPMC11464531 | biostudies-literature
Project description:Analyzing brain networks has long been a prominent research topic in neuroimaging. However, statistical methods to detect differences between these networks and relate them to phenotypic traits are still sorely needed. Our previous work developed a novel permutation testing framework to detect differences between two groups. Here we advance that work to allow both assessing differences by continuous phenotypes and controlling for confounding variables. To achieve this, we propose an innovative regression framework to relate distances (or similarities) between brain network features to functions of absolute differences in continuous covariates and indicators of difference for categorical variables. We explore several similarity metrics for comparing distances (or similarities) between connection matrices, and adapt several standard methods for estimation and inference within our framework: standard F test, F test with individual level effects (ILE), feasible generalized least squares (FGLS), and permutation. Via simulation studies, we assess all approaches for estimation and inference while comparing them with existing multivariate distance matrix regression (MDMR) methods. We then illustrate the utility of our framework by analyzing the relationship between fluid intelligence and brain network distances in Human Connectome Project (HCP) data.
| S-EPMC8942614 | biostudies-literature
Project description:BackgroundOpioid use disorder (OUD) is a critical public health crisis in the United States, affecting >5.5 million Americans in 2021. Machine learning has been used to predict patient risk of incident OUD. However, little is known about the fairness and bias of these predictive models.ObjectiveThe aims of this study are two-fold: (1) to develop a machine learning bias mitigation algorithm for sociodemographic features and (2) to develop a fairness-aware weighted majority voting (WMV) classifier for OUD prediction.MethodsWe used the 2020 National Survey on Drug and Health data to develop a neural network (NN) model using stochastic gradient descent (SGD; NN-SGD) and an NN model using Adam (NN-Adam) optimizers and evaluated sociodemographic bias by comparing the area under the curve values. A bias mitigation algorithm, based on equality of odds, was implemented to minimize disparities in specificity and recall. Finally, a WMV classifier was developed for fairness-aware prediction of OUD. To further analyze bias detection and mitigation, we did a 1-N matching of OUD to non-OUD cases, controlling for socioeconomic variables, and evaluated the performance of the proposed bias mitigation algorithm and WMV classifier.ResultsOur bias mitigation algorithm substantially reduced bias with NN-SGD, by 21.66% for sex, 1.48% for race, and 21.04% for income, and with NN-Adam by 16.96% for sex, 8.87% for marital status, 8.45% for working condition, and 41.62% for race. The fairness-aware WMV classifier achieved a recall of 85.37% and 92.68% and an accuracy of 58.85% and 90.21% using NN-SGD and NN-Adam, respectively. The results after matching also indicated remarkable bias reduction with NN-SGD and NN-Adam, respectively, as follows: sex (0.14% vs 0.97%), marital status (12.95% vs 10.33%), working condition (14.79% vs 15.33%), race (60.13% vs 41.71%), and income (0.35% vs 2.21%). Moreover, the fairness-aware WMV classifier achieved high performance with a recall of 100% and 85.37% and an accuracy of 73.20% and 89.38% using NN-SGD and NN-Adam, respectively.ConclusionsThe application of the proposed bias mitigation algorithm shows promise in reducing sociodemographic bias, with the WMV classifier confirming bias reduction and high performance in OUD prediction.
| S-EPMC11372321 | biostudies-literature
Project description:Measuring the impact of public health science or research is important especially when it comes to health outcomes. Achieving the desired health outcomes take time and may be influenced by several contributors, making attribution of credit to any one entity or effort problematic. Here we offer a science impact framework (SIF) for tracing and linking public health science to events and/or actions with recognized impact beyond journal metrics. The SIF was modeled on the Institute of Medicine's (IOM) Degrees of Impact Thermometer, but differs in that SIF is not incremental, not chronological, and has expanded scope. The SIF recognizes five domains of influence: disseminating science, creating awareness, catalyzing action, effecting change and shaping the future (scope differs from IOM). For public health, the goal is to achieve one or more specific health outcomes. What is unique about this framework is that the focus is not just on the projected impact or outcome but rather the effects that are occurring in real time with the recognition that the measurement field is complex, and it takes time for the ultimate outcome to occur. The SIF is flexible and can be tailored to measure the impact of any scientific effort: from complex initiatives to individual publications. The SIF may be used to measure impact prospectively of an ongoing or new body of work (e.g., research, guidelines and recommendations, or technology) and retrospectively of completed and disseminated work, through linking of events using indicators that are known and have been used for measuring impact. Additionally, linking events offers an approach to both tell our story and also acknowledge other players in the chain of events. The value added by science can easily be relayed to the scientific community, policy makers and the public.
| S-EPMC7755179 | biostudies-literature
Project description:BackgroundSpurred by the Coronavirus infectious disease 2019 pandemic, aerosol containment devices (ACDs) were developed to capture infectious respiratory aerosols generated by patients at their source. Prior reviews indicated that such devices had low evidence of effectiveness, but did not address how ACDs should be evaluated, how well they should perform, nor have clearly defined performance standards. Towards developing design criteria for ACDs, two questions were posed: 1) What characteristics have guided the design of ACDs? 2) How have these characteristics been evaluated?MethodsA scoping review was performed consistent with PRISMA guidelines. Data were extracted with respect to general study information, intended use of the device, device design characteristics and evaluation.ResultsFifty-four articles were included. Evaluation was most commonly performed with respect to device aerosol containment (n = 31, 61%), with only 5 (9%), 3 (6%) and 8 (15%) formally assessing providing experience, patient experience and procedure impact, respectively. Nearly all of the studies that explored provider experience and procedure impact studied intubation. Few studies provided a priori performance criteria for any evaluation metric, or referenced any external guidelines by which to bench mark performance.ConclusionWith respect to aerosol containment, ACDs should reduce exposure among HCP with the device compared with the absence of the device, and provide ≥90% reduction in respirable aerosols, equivalent in performance to N95 filtering facepiece respirators, if the goal is to reduce reliance on personal protective equipment. The ACD should not increase awkward or uncomfortable postures, or adversely impact biomechanics of the procedure itself as this could have implications for procedure outcomes. A variety of standardized instruments exist to assess the experience of patients and healthcare personnel. Integration of ACDs into routine clinical practice requires rigorous studies of aerosol containment and the user experience.
| S-EPMC9650508 | biostudies-literature