Dataset Information

Data Representativeness in Accessibility Datasets: A Meta-Analysis.

ABSTRACT: As data-driven systems are increasingly deployed at scale, ethical concerns have arisen around unfair and discriminatory outcomes for historically marginalized groups that are underrepresented in training data. In response, work around AI fairness and inclusion has called for datasets that are representative of various demographic groups. In this paper, we contribute an analysis of the representativeness of age, gender, and race & ethnicity in accessibility datasets-datasets sourced from people with disabilities and older adults-that can potentially play an important role in mitigating bias for inclusive AI-infused applications. We examine the current state of representation within datasets sourced by people with disabilities by reviewing publicly-available information of 190 datasets, we call these accessibility datasets. We find that accessibility datasets represent diverse ages, but have gender and race representation gaps. Additionally, we investigate how the sensitive and complex nature of demographic variables makes classification difficult and inconsistent (e.g., gender, race & ethnicity), with the source of labeling often unknown. By reflecting on the current challenges and opportunities for representation of disabled data contributors, we hope our effort expands the space of possibility for greater inclusion of marginalized communities in AI-infused systems.

SUBMITTER: Kamikubo R

PROVIDER: S-EPMC10024595 | biostudies-literature | 2022 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Data Representativeness in Accessibility Datasets: A Meta-Analysis.

Kamikubo Rie R Wang Lining L Marte Crystal C Mahmood Amnah A Kacorri Hernisa H

ASSETS. Annual ACM Conference on Assistive Technologies 20221022

As data-driven systems are increasingly deployed at scale, ethical concerns have arisen around unfair and discriminatory outcomes for historically marginalized groups that are underrepresented in training data. In response, work around AI fairness and inclusion has called for datasets that are representative of various demographic groups. In this paper, we contribute an analysis of the representativeness of age, gender, and race & ethnicity in accessibility datasets-datasets sourced from people ...[more]

PMID: 36939417

Similar Datasets

Project description:BackgroundA lack of available data and statistical code being published alongside journal articles provides a significant barrier to open scientific discourse, and reproducibility of research. Information governance restrictions inhibit the active dissemination of individual level data to accompany published manuscripts. Realistic, high-fidelity time-to-event synthetic data can aid in the acceleration of methodological developments in survival analysis and beyond by enabling researchers to access and test published methods using data similar to that which they were developed on.MethodsWe present methods to accurately emulate the covariate patterns and survival times found in real-world datasets using synthetic data techniques, without compromising patient privacy. We model the joint covariate distribution of the original data using covariate specific sequential conditional regression models, then fit a complex flexible parametric survival model from which to generate survival times conditional on individual covariate patterns. We recreate the administrative censoring mechanism using the last observed follow-up date information from the initial dataset. Metrics for evaluating the accuracy of the synthetic data, and the non-identifiability of individuals from the original dataset, are presented.ResultsWe successfully create a synthetic version of an example colon cancer dataset consisting of 9064 patients which aims to show good similarity to both covariate distributions and survival times from the original data, without containing any exact information from the original data, therefore allowing them to be published openly alongside research.ConclusionsWe evaluate the effectiveness of the methods for constructing synthetic data, as well as providing evidence that there is minimal risk that a given patient from the original data could be identified from their individual unique patient information. Synthetic datasets using this methodology could be made available alongside published research without breaching data privacy protocols, and allow for data and code to be made available alongside methodological or applied manuscripts to greatly improve the transparency and accessibility of medical research.

Project description:ImportanceSystematic differences between patients included in randomized clinical trials (RCTs) and the general patient population may influence the generalizability of RCT findings. Comprehensive national registries of patients with end-stage kidney disease who are undergoing dialysis provide a unique opportunity to compare trial and real-world patient cohorts.ObjectiveTo determine if participants in large, multicenter dialysis trials were similar to the general population undergoing dialysis in terms of age, comorbidities, and mortality rate.Data sourcesMEDLINE, PubMed, and the Cochrane Central Register of Controlled Trials were systematically searched on January 6, 2017, for studies published from January 1, 2007, to December 31, 2016. Data sources were published manuscripts, supplementary material, and trial registration information. Data on the general population undergoing dialysis were derived from the US Renal Data System (USRDS). Data were analyzed from March 17 to July 22, 2018.Study selectionRandomized clinical trials enrolling only participants undergoing dialysis for end-stage kidney disease with 100 or more adult participants from 2 or more sites.Data extraction and synthesisAbstract screening and data extraction were performed independently by 2 researchers. Data were pooled using a random-effects model.Main outcomes and measuresThe primary outcome was difference in mean age between the RCT and USRDS populations. Secondary outcomes included differences in mortality rate and comorbidities.ResultsThe search identified 186 RCTs, enrolling 79 104 participants. Compared with the 2011 USRDS population, RCT participants were younger (mean age, 58.9 years; 95% CI, 58.3-59.5 years vs 61.2 years; P < .001), more likely to be male (58.9%; 95% CI, 57.6%-60.1% vs 55.7%; P < .001), and have coronary artery disease (26.9%; 95% CI, 22.2%-31.7% vs 17.7%; P < .001) and less likely to have diabetes (40.2%; 95% CI, 36.7%-43.6% vs 44.2%; P = .03) or heart failure (19.6%; 95% CI, 15.1%-24.0% vs 29.8%; P < .001). The mortality rate per 100 patient-years during trial participation was less than half that of the USRDS population (8.9; 95% CI, 7.8-10.0 vs 18.6; P < .001). The differences in age, mortality, and coronary artery disease remained when studies recruiting only from the United States were considered. Diabetes was more common in RCT participants from the United States than in the registry population.Conclusions and relevanceParticipants in large, multicenter RCTs of patients with end-stage kidney disease undergoing dialysis are younger, have a different pattern of comorbidities, and have a lower mortality rate than the general population of patients undergoing dialysis. This finding has implications for the generalization of trial results to the broader patient population and for future trial design.

Dataset Information

Data Representativeness in Accessibility Datasets: A Meta-Analysis.

Publications

Data Representativeness in Accessibility Datasets: A Meta-Analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets