Considerations for addressing bias in artificial intelligence for health equity.
Ontology highlight
ABSTRACT:
SUBMITTER: Abramoff MD
PROVIDER: S-EPMC10497548 | biostudies-literature | 2023 Sep
REPOSITORIES: biostudies-literature
Publications
Considerations for addressing bias in artificial intelligence for health equity.
Abràmoff Michael D MD Tarver Michelle E ME Loyo-Berrios Nilsa N Trujillo Sylvia S Char Danton D Obermeyer Ziad Z Eydelman Malvina B MB Maisel William H WH
NPJ digital medicine 20230912 1
Health equity is a primary goal of healthcare stakeholders: patients and their advocacy groups, clinicians, other providers and their professional societies, bioethicists, payors and value based care organizations, regulatory agencies, legislators, and creators of artificial intelligence/machine learning (AI/ML)-enabled medical devices. Lack of equitable access to diagnosis and treatment may be improved through new digital health technologies, especially AI/ML, but these may also exacerbate disp ...[more]
Similar Datasets
Project description:BackgroundResources are increasingly spent on artificial intelligence (AI) solutions for medical applications aiming to improve diagnosis, treatment, and prevention of diseases. While the need for transparency and reduction of bias in data and algorithm development has been addressed in past studies, little is known about the knowledge and perception of bias among AI developers.ObjectiveThis study's objective was to survey AI specialists in health care to investigate developers' perceptions of bias in AI algorithms for health care applications and their awareness and use of preventative measures.MethodsA web-based survey was provided in both German and English language, comprising a maximum of 41 questions using branching logic within the REDCap web application. Only the results of participants with experience in the field of medical AI applications and complete questionnaires were included for analysis. Demographic data, technical expertise, and perceptions of fairness, as well as knowledge of biases in AI, were analyzed, and variations among gender, age, and work environment were assessed.ResultsA total of 151 AI specialists completed the web-based survey. The median age was 30 (IQR 26-39) years, and 67% (101/151) of respondents were male. One-third rated their AI development projects as fair (47/151, 31%) or moderately fair (51/151, 34%), 12% (18/151) reported their AI to be barely fair, and 1% (2/151) not fair at all. One participant identifying as diverse rated AI developments as barely fair, and among the 2 undefined gender participants, AI developments were rated as barely fair or moderately fair, respectively. Reasons for biases selected by respondents were lack of fair data (90/132, 68%), guidelines or recommendations (65/132, 49%), or knowledge (60/132, 45%). Half of the respondents worked with image data (83/151, 55%) from 1 center only (76/151, 50%), and 35% (53/151) worked with national data exclusively.ConclusionsThis study shows that the perception of biases in AI overall is moderately fair. Gender minorities did not once rate their AI development as fair or very fair. Therefore, further studies need to focus on minorities and women and their perceptions of AI. The results highlight the need to strengthen knowledge about bias in AI and provide guidelines on preventing biases in AI health care applications.
| S-EPMC10337406 | biostudies-literature
Project description:BackgroundThe field of oncology is at the forefront of advances in artificial intelligence (AI) in health care, providing an opportunity to examine the early integration of these technologies in clinical research and patient care. Hope that AI will revolutionize health care delivery and improve clinical outcomes has been accompanied by concerns about the impact of these technologies on health equity.ObjectiveWe aimed to conduct a scoping review of the literature to address the question, "What are the current and potential impacts of AI technologies on health equity in oncology?"MethodsFollowing PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines for scoping reviews, we systematically searched MEDLINE and Embase electronic databases from January 2000 to August 2021 for records engaging with key concepts of AI, health equity, and oncology. We included all English-language articles that engaged with the 3 key concepts. Articles were analyzed qualitatively for themes pertaining to the influence of AI on health equity in oncology.ResultsOf the 14,011 records, 133 (0.95%) identified from our review were included. We identified 3 general themes in the literature: the use of AI to reduce health care disparities (58/133, 43.6%), concerns surrounding AI technologies and bias (16/133, 12.1%), and the use of AI to examine biological and social determinants of health (55/133, 41.4%). A total of 3% (4/133) of articles focused on many of these themes.ConclusionsOur scoping review revealed 3 main themes on the impact of AI on health equity in oncology, which relate to AI's ability to help address health disparities, its potential to mitigate or exacerbate bias, and its capability to help elucidate determinants of health. Gaps in the literature included a lack of discussion of ethical challenges with the application of AI technologies in low- and middle-income countries, lack of discussion of problems of bias in AI algorithms, and a lack of justification for the use of AI technologies over traditional statistical methods to address specific research questions in oncology. Our review highlights a need to address these gaps to ensure a more equitable integration of AI in cancer research and clinical practice. The limitations of our study include its exploratory nature, its focus on oncology as opposed to all health care sectors, and its analysis of solely English-language articles.
| S-EPMC9667381 | biostudies-literature
Project description:Patients with heart failure (HF) are heterogeneous with various intrapersonal and interpersonal characteristics contributing to clinical outcomes. Bias, structural racism, and social determinants of health have been implicated in unequal treatment of patients with HF. Through several methodologies, artificial intelligence (AI) can provide models in HF prediction, prognostication, and provision of care, which may help prevent unequal outcomes. This review highlights AI as a strategy to address racial inequalities in HF; discusses key AI definitions within a health equity context; describes the current uses of AI in HF, strengths and harms in using AI; and offers recommendations for future directions.
| S-EPMC8988237 | biostudies-literature
Project description:BackgroundArtificial intelligence (AI) predictive models in primary health care have the potential to enhance population health by rapidly and accurately identifying individuals who should receive care and health services. However, these models also carry the risk of perpetuating or amplifying existing biases toward diverse groups. We identified a gap in the current understanding of strategies used to assess and mitigate bias in primary health care algorithms related to individuals' personal or protected attributes.ObjectiveThis study aimed to describe the attempts, strategies, and methods used to mitigate bias in AI models within primary health care, to identify the diverse groups or protected attributes considered, and to evaluate the results of these approaches on both bias reduction and AI model performance.MethodsWe conducted a scoping review following Joanna Briggs Institute (JBI) guidelines, searching Medline (Ovid), CINAHL (EBSCO), PsycINFO (Ovid), and Web of Science databases for studies published between January 1, 2017, and November 15, 2022. Pairs of reviewers independently screened titles and abstracts, applied selection criteria, and performed full-text screening. Discrepancies regarding study inclusion were resolved by consensus. Following reporting standards for AI in health care, we extracted data on study objectives, model features, targeted diverse groups, mitigation strategies used, and results. Using the mixed methods appraisal tool, we appraised the quality of the studies.ResultsAfter removing 585 duplicates, we screened 1018 titles and abstracts. From the remaining 189 full-text articles, we included 17 studies. The most frequently investigated protected attributes were race (or ethnicity), examined in 12 of the 17 studies, and sex (often identified as gender), typically classified as "male versus female" in 10 of the studies. We categorized bias mitigation approaches into four clusters: (1) modifying existing AI models or datasets, (2) sourcing data from electronic health records, (3) developing tools with a "human-in-the-loop" approach, and (4) identifying ethical principles for informed decision-making. Algorithmic preprocessing methods, such as relabeling and reweighing data, along with natural language processing techniques that extract data from unstructured notes, showed the greatest potential for bias mitigation. Other methods aimed at enhancing model fairness included group recalibration and the application of the equalized odds metric. However, these approaches sometimes exacerbated prediction errors across groups or led to overall model miscalibrations.ConclusionsThe results suggest that biases toward diverse groups are more easily mitigated when data are open-sourced, multiple stakeholders are engaged, and during the algorithm's preprocessing stage. Further empirical studies that include a broader range of groups, such as Indigenous peoples in Canada, are needed to validate and expand upon these findings.Trial registrationOSF Registry osf.io/9ngz5/; https://osf.io/9ngz5/.International registered report identifier (irrid)RR2-10.2196/46684.
| S-EPMC11751650 | biostudies-literature
Project description:The European Union and some of its institutions have taken significant steps to address the challenges posed by the development and use of Artificial Intelligence (AI) in various contexts. The ubiquity of AI applications in everyday life, affecting both citizens and professionals, has made AI a common topic of discussion. However, as is evident from the documents analyzed here, concerns have been raised about the possible negative social consequences of AI, in particular discriminatory bias, making it a particularly relevant issue if people-centred, rights-based AI is to be implemented. This article aims to examine the challenges of defining, identifying and mitigating discriminatory bias in AI systems from two perspectives: (1) to conduct an ethical and normative review of European Commission documents from the last 8 years (from GDPR to AI Act regulation); and (2) to expose recommendations for key stakeholders, including designers, end-users and public authorities, to minimize/mitigate this risk. The document review was carried out on 21 EU regulatory and ethical guidelines in the field of AI, from which 152 measures were extracted, differentiated between design, governance and organizational measures. It has also been observed that there is no clear conceptual framework on the issue at the European level, showing a clear problem in providing definitions of algorithmic bias and discrimination, but not in assessing their potential negative impact on individuals. Secondly, these gaps may affect the concreteness and detail of the possible mitigation/minimization measures proposed and, subsequently, their application in different contexts. Finally, the last section of this paper presents a brief discussion and conclusions on possible issues related to the implementation of the measures extracted and certain limitations of the study.
| S-EPMC11231082 | biostudies-literature
Project description:BackgroundIncorporating artificial intelligence (AI) into clinics brings the risk of automation bias, which potentially misleads the clinician's decision-making. The purpose of this study was to propose a potential strategy to mitigate automation bias.MethodsThis was a laboratory study with a randomized cross-over design. The diagnosis of anterior cruciate ligament (ACL) rupture, a common injury, on magnetic resonance imaging (MRI) was used as an example. Forty clinicians were invited to diagnose 200 ACLs with and without AI assistance. The AI's correcting and misleading (automation bias) effects on the clinicians' decision-making processes were analyzed. An ordinal logistic regression model was employed to predict the correcting and misleading probabilities of the AI. We further proposed an AI suppression strategy that retracted AI diagnoses with a higher misleading probability and provided AI diagnoses with a higher correcting probability.ResultsThe AI significantly increased clinicians' accuracy from 87.2%±13.1% to 96.4%±1.9% (P < .001). However, the clinicians' errors in the AI-assisted round were associated with automation bias, accounting for 45.5% of the total mistakes. The automation bias was found to affect clinicians of all levels of expertise. Using a logistic regression model, we identified an AI output zone with higher probability to generate misleading diagnoses. The proposed AI suppression strategy was estimated to decrease clinicians' automation bias by 41.7%.ConclusionAlthough AI improved clinicians' diagnostic performance, automation bias was a serious problem that should be addressed in clinical practice. The proposed AI suppression strategy is a practical method for decreasing automation bias.
| S-EPMC10531198 | biostudies-literature
Project description:Historically, biomedical research has been led by and focused on men. The recent introduction of Artificial Intelligence (AI) in this area has further proven this practice to be discriminatory for other sexes and genders, more noticeably for women. To move towards a fair AI development, it is essential to include sex and gender diversity both in research practices and in the workplace. In this context, the Bioinfo4women (B4W) program of the Barcelona Supercomputing Center (i) promotes the participation of women scientists by improving their visibility, (ii) fosters international collaborations between institutions and programs and (iii) advances research on sex and gender bias in AI and health. In this article, we discuss methodology and results of a series of conferences, titled “Sex and Gender Bias in Artificial Intelligence and Health, organized by B4W and La Caixa Foundation from March to June 2021 in Barcelona, Spain. The series consisted of nine hybrid events, composed of keynote sessions and seminars open to the general audience, and two working groups with invited experts from different professional backgrounds (academic fields such as biology, engineering, and sociology, as well as NGOs, journalists, lawyers, policymakers, industry). Based on this awareness-raising action, we distilled key recommendations to facilitate the inclusion of sex and gender perspective into public policies, educational programs, industry, and biomedical research, among other sectors, and help overcome sex and gender biases in AI and health.
| S-EPMC10512182 | biostudies-literature
Project description:The emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that led to the COVID-19 (Coronavirus Disease 2019) pandemic, has resulted in substantial overburdening of healthcare systems as well as an economic crisis on a global scale. This has in turn resulted in widespread efforts to identify suitable therapies to address this aggressive pathogen. Therapeutic antibody and vaccine development are being actively explored, and a phase I clinical trial of mRNA-1273 which is developed in collaboration between the National Institute of Allergy and Infectious Diseases and Moderna, Inc. is currently underway. Timelines for the broad deployment of a vaccine and antibody therapies have been estimated to be 12-18 months or longer. These are promising approaches that may lead to sustained efficacy in treating COVID-19. However, its emergence has also led to a large number of clinical trials evaluating drug combinations composed of repurposed therapies. As study results of these combinations continue to be evaluated, there is a need to move beyond traditional drug screening and repurposing by harnessing artificial intelligence (AI) to optimize combination therapy design. This may lead to the rapid identification of regimens that mediate unexpected and markedly enhanced treatment outcomes.
| S-EPMC7235485 | biostudies-literature
Project description:BackgroundThough artificial intelligence (AI) has the potential to augment the patient-physician relationship in primary care, bias in intelligent health care systems has the potential to differentially impact vulnerable patient populations.ObjectiveThe purpose of this scoping review is to summarize the extent to which AI systems in primary care examine the inherent bias toward or against vulnerable populations and appraise how these systems have mitigated the impact of such biases during their development.MethodsWe will conduct a search update from an existing scoping review to identify studies on AI and primary care in the following databases: Medline-OVID, Embase, CINAHL, Cochrane Library, Web of Science, Scopus, IEEE Xplore, ACM Digital Library, MathSciNet, AAAI, and arXiv. Two screeners will independently review all abstracts, titles, and full-text articles. The team will extract data using a structured data extraction form and synthesize the results in accordance with PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines.ResultsThis review will provide an assessment of the current state of health care equity within AI for primary care. Specifically, we will identify the degree to which vulnerable patients have been included, assess how bias is interpreted and documented, and understand the extent to which harmful biases are addressed. As of October 2020, the scoping review is in the title- and abstract-screening stage. The results are expected to be submitted for publication in fall 2021.ConclusionsAI applications in primary care are becoming an increasingly common tool in health care delivery and in preventative care efforts for underserved populations. This scoping review would potentially show the extent to which studies on AI in primary care employ a health equity lens and take steps to mitigate bias.International registered report identifier (irrid)PRR1-10.2196/27799.
| S-EPMC8486995 | biostudies-literature
Project description:Diabetic eye disease (DED) is a leading cause of blindness in the world. Annual DED testing is recommended for adults with diabetes, but adherence to this guideline has historically been low. In 2020, Johns Hopkins Medicine (JHM) began deploying autonomous AI for DED testing. In this study, we aimed to determine whether autonomous AI implementation was associated with increased adherence to annual DED testing, and how this differed across patient populations. JHM primary care sites were categorized as "non-AI" (no autonomous AI deployment) or "AI-switched" (autonomous AI deployment by 2021). We conducted a propensity score weighting analysis to compare change in adherence rates from 2019 to 2021 between non-AI and AI-switched sites. Our study included all adult patients with diabetes (>17,000) managed within JHM and has three major findings. First, AI-switched sites experienced a 7.6 percentage point greater increase in DED testing than non-AI sites from 2019 to 2021 (p < 0.001). Second, the adherence rate for Black/African Americans increased by 12.2 percentage points within AI-switched sites but decreased by 0.6% points within non-AI sites (p < 0.001), suggesting that autonomous AI deployment improved access to retinal evaluation for historically disadvantaged populations. Third, autonomous AI is associated with improved health equity, e.g. the adherence rate gap between Asian Americans and Black/African Americans shrank from 15.6% in 2019 to 3.5% in 2021. In summary, our results from real-world deployment in a large integrated healthcare system suggest that autonomous AI is associated with improvement in overall DED testing adherence, patient access, and health equity.
| S-EPMC11263546 | biostudies-literature