Examining explainable clinical decision support systems with think aloud protocols.
Ontology highlight
ABSTRACT:
SUBMITTER: Anjara SG
PROVIDER: S-EPMC10501571 | biostudies-literature | 2023
REPOSITORIES: biostudies-literature
Publications
Examining explainable clinical decision support systems with think aloud protocols.
Anjara Sabrina G SG Janik Adrianna A Dunford-Stenger Amy A Mc Kenzie Kenneth K Collazo-Lorduy Ana A Torrente Maria M Costabello Luca L Provencio Mariano M
PloS one 20230914 9
Machine learning tools are increasingly used to improve the quality of care and the soundness of a treatment plan. Explainable AI (XAI) helps users in understanding the inner mechanisms of opaque machine learning models and is a driver of trust and adoption. Explanation methods for black-box models exist, but there is a lack of user studies on the interpretability of the provided explanations. We used a Think Aloud Protocol (TAP) to explore oncologists' assessment of a lung cancer relapse predic ...[more]
Similar Datasets
Project description:BackgroundDuring the COVID-19 pandemic, a variety of clinical decision support systems (CDSS) were developed to aid patient triage. However, research focusing on the interaction between decision support systems and human experts is lacking.MethodsThirty-two physicians were recruited to rate the survival probability of 59 critically ill patients by means of chart review. Subsequently, one of two artificial intelligence systems advised the physician of a computed survival probability. However, only one of these systems explained the reasons behind its decision-making. In the third step, physicians reviewed the chart once again to determine the final survival probability rating. We hypothesized that an explaining system would exhibit a higher impact on the physicians' second rating (i.e., higher weight-on-advice).ResultsThe survival probability rating given by the physician after receiving advice from the clinical decision support system was a median of 4 percentage points closer to the advice than the initial rating. Weight-on-advice was not significantly different (p = 0.115) between the two systems (with vs without explanation for its decision). Additionally, weight-on-advice showed no difference according to time of day or between board-qualified and not yet board-qualified physicians. Self-reported post-experiment overall trust was awarded a median of 4 out of 10 points. When asked after the conclusion of the experiment, overall trust was 5.5/10 (non-explaining median 4 (IQR 3.5-5.5), explaining median 7 (IQR 5.5-7.5), p = 0.007).ConclusionsAlthough overall trust in the models was low, the median (IQR) weight-on-advice was high (0.33 (0.0-0.56)) and in line with published literature on expert advice. In contrast to the hypothesis, weight-on-advice was comparable between the explaining and non-explaining systems. In 30% of cases, weight-on-advice was 0, meaning the physician did not change their rating. The median of the remaining weight-on-advice values was 50%, suggesting that physicians either dismissed the recommendation or employed a "meeting halfway" approach. Newer technologies, such as clinical reasoning systems, may be able to augment the decision process rather than simply presenting unexplained bias.
| S-EPMC10510231 | biostudies-literature
Project description:ObjectivesThis study investigated whether medical student responses to Script Concordance Testing (SCT) items represent valid clinical reasoning. Using a think-aloud approach students provided written explanations of the reasoning that underpinned their responses, and these were reviewed for concordance with an expert reference panel.MethodsA set of 12, 11 and 15 SCT items were administered online to Year 3 (2018), Year 4 (2018) and Year 3 (2019) medical students respectively. Students' free-text descriptions of the reasoning supporting each item response were analysed, and compared with those of the expert panel. Response process validity was quantified as the rate of true positives (percentage of full and partial credit responses derived through correct clinical reasoning); and true negatives (percentage of responses with no credit derived through faulty clinical reasoning).ResultsTwo hundred and nine students completed the online tests (response rate = 68.3%). The majority of students who had chosen the response which attracted full or partial credit also provided justifications which were concordant with the experts (true positive rate of 99.6% for full credit; 99.4% for partial credit responses). Most responses that attracted no credit were based on faulty clinical reasoning (true negative of 99.0%).ConclusionsThe findings provide support for the response process validity of SCT scores in the setting of undergraduate medicine. The additional written think-aloud component, to assess clinical reasoning, provided useful information to inform student learning. However, SCT scores should be validated on each testing occasion, and in other contexts.
| S-EPMC7870454 | biostudies-literature
Project description:ObjectiveTo develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) approaches.MethodsWe extracted data on alerts generated from January 1, 2019 to December 31, 2020, at Vanderbilt University Medical Center. We developed machine learning models to predict user responses to alerts. We applied XAI techniques to generate global explanations and local explanations. We evaluated the generated suggestions by comparing with alert's historical change logs and stakeholder interviews. Suggestions that either matched (or partially matched) changes already made to the alert or were considered clinically correct were classified as helpful.ResultsThe final dataset included 2 991 823 firings with 2689 features. Among the 5 machine learning models, the LightGBM model achieved the highest Area under the ROC Curve: 0.919 [0.918, 0.920]. We identified 96 helpful suggestions. A total of 278 807 firings (9.3%) could have been eliminated. Some of the suggestions also revealed workflow and education issues.ConclusionWe developed a data-driven process to generate suggestions for improving alert criteria using XAI techniques. Our approach could identify improvements regarding clinical decision support (CDS) that might be overlooked or delayed in manual reviews. It also unveils a secondary purpose for the XAI: to improve quality by discovering scenarios where CDS alerts are not accepted due to workflow, education, or staffing issues.
| S-EPMC10990514 | biostudies-literature
Project description:Simulations and games bring the possibility to research complex processes of managerial decision-making. However, this modern field requires adequate methodological procedures. Many authors recommend the use of a combination of concurrent think-aloud (CTA) or retrospective think-aloud (RTA) with eye-tracking to investigate cognitive processes such as decision-making. Nevertheless, previous studies have little or no consideration of the possible differential impact of both think-aloud methods on data provided by eye-tracking. Therefore, the main aim of this study is to compare and assess if and how these methods differ in terms of their impact on eye-tracking. The experiment was conducted for this purpose. Participants were 14 managers who played a specific simulation game with CTA use and 17 managers who played the same game with RTA use. The results empirically prove that CTA significantly distorts data provided by eye-tracking, whereas data gathered when RTA is used, provide independent pieces of evidence about the participants' behavior. These findings suggest that RTA is more suitable for combined use with eye-tracking for the purpose of the research of decision-making in the game environment.
| S-EPMC7294419 | biostudies-literature
Project description:The overwhelming amount, production speed, multidimensionality, and potential value of data currently available-often simplified and referred to as big data -exceed the limits of understanding of the human brain. At the same time, developments in data analytics and computational power provide the opportunity to obtain new insights and transfer data-provided added value to clinical practice in real time. What is the role of the health care professional in collaboration with the data scientist in the changing landscape of modern care? We discuss how health care professionals should provide expert knowledge in each of the stages of clinical decision support design: data level, algorithm level, and decision support level. Including various ethical considerations, we advocate for health care professionals to responsibly initiate and guide interprofessional teams, including patients, and embrace novel analytic technologies to translate big data into patient benefit driven by human(e) values.
| S-EPMC6444220 | biostudies-literature
Project description:BackgroundComputerized clinical decision support systems (CDSSs) are a promising knowledge translation tool, but often fail to meaningfully influence the outcomes they target. Low CDSS provider uptake is a potential contributor to this problem but has not been systematically studied. The objective of this systematic review and meta-regression was to determine reported CDSS uptake and identify which CDSS features may influence uptake.MethodsMedline, Embase, CINAHL, and the Cochrane Database of Controlled Trials were searched from January 2000 to August 2020. Randomized, non-randomized, and quasi-experimental trials reporting CDSS uptake in any patient population or setting were included. The main outcome extracted was CDSS uptake, reported as a raw proportion, and representing the number of times the CDSS was used or accessed over the total number of times it could have been interacted with. We also extracted context, content, system, and implementation features that might influence uptake, for each CDSS. Overall weighted uptake was calculated using random-effects meta-analysis and determinants of uptake were investigated using multivariable meta-regression.ResultsAmong 7995 citations screened, 55 studies involving 373,608 patients and 3607 providers met full inclusion criteria. Meta-analysis revealed that overall CDSS uptake was 34.2% (95% CI 23.2 to 47.1%). Uptake was only reported in 12.4% of studies that otherwise met inclusion criteria. Multivariable meta-regression revealed the following factors significantly associated with uptake: (1) formally evaluating the availability and quality of the patient data needed to inform CDSS advice; and (2) identifying and addressing other barriers to the behaviour change targeted by the CDSS.Conclusions and relevanceSystem uptake was seldom reported in CDSS trials. When reported, uptake was low. This represents a major and potentially modifiable barrier to overall CDSS effectiveness. We found that features relating to CDSS context and implementation strategy best predicted uptake. Future studies should measure the impact of addressing these features as part of the CDSS implementation strategy. Uptake reporting must also become standard in future studies reporting CDSS intervention effects.RegistrationPre-registered on PROSPERO, CRD42018092337.
| S-EPMC8908582 | biostudies-literature
Project description:Breast cancer is the most frequently diagnosed cancer in women, with more than 2.1 million new diagnoses worldwide every year. Personalised treatment is critical to optimising outcomes for patients with breast cancer. A major advance in medical practice is the incorporation of Clinical Decision Support Systems (CDSSs) to assist and support healthcare staff in clinical decision-making, thus improving the quality of decisions and overall patient care whilst minimising costs. The usage and availability of CDSSs in breast cancer care in healthcare settings is increasing. However, there may be differences in how particular CDSSs are developed, the information they include, the decisions they recommend, and how they are used in practice. This systematic review examines various CDSSs to determine their availability, intended use, medical characteristics, and expected outputs concerning breast cancer therapeutic decisions, an area that is known to have varying degrees of subjectivity in clinical practice. Utilising the methodology of Kitchenham and Charter, a systematic search of the literature was performed in Springer, Science Direct, Google Scholar, PubMed, ACM, IEEE, and Scopus. An overview of CDSS which supports decision-making in breast cancer treatment is provided along with a critical appraisal of their benefits, limitations, and opportunities for improvement.
| S-EPMC7072392 | biostudies-literature
Project description:BackgroundWorldwide, the burden of allergies-in particular, drug allergies-is growing. In the process of prescribing, dispensing, or administering a drug, a medication error may occur and can have adverse consequences; for example, a drug may be given to a patient with a documented allergy to that particular drug. Computerized physician order entry (CPOE) systems with built-in clinical decision support systems (CDSS) have the potential to prevent such medication errors and adverse events.ObjectiveThe aim of this review is to provide a comprehensive overview regarding all aspects of CDSS for drug allergy, including documenting, coding, rule bases, alerts and alert fatigue, and outcome evaluation.MethodsThe Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed as much as possible and searches were conducted in 5 databases using CPOE, CDSS, alerts, and allergic or allergy as keywords. Bias could not be evaluated according to PRISMA guidelines due to the heterogeneity of study types included in the review.ResultsOf the 3160 articles considered, 60 met the inclusion criteria. A further 9 articles were added based on expert opinion, resulting in a total of 69 articles. An interrater agreement of 90.9% with a reliability ?=.787 (95% CI 0.686-0.888) was reached. Large heterogeneity across study objectives, study designs, study populations, and reported results was found. Several key findings were identified. Evidence of the usefulness of clinical decision support for drug allergies has been documented. Nevertheless, there are some important problems associated with their use. Accurate and structured documenting of information on drug allergies in electronic health records (EHRs) is difficult, as it is often not clear to healthcare providers how and where to document drug allergies. Besides the underreporting of drug allergies, outdated or inaccurate drug allergy information in EHRs poses an important problem. Research on the use of coding terminologies for documenting drug allergies is sparse. There is no generally accepted standard terminology for structured documentation of allergy information. The final key finding is the consistently reported low specificity of drug allergy alerts. Current systems have high alert override rates of up to 90%, leading to alert fatigue. Important challenges remain for increasing the specificity of drug allergy alerts. We found only one study specifically reporting outcomes related to CDSS for drug allergies. It showed that adverse drug events resulting from overridden drug allergy alerts do not occur frequently.ConclusionsAccurate and comprehensive recording of drug allergies is required for good use of CDSS for drug allergy screening. We found considerable variation in the way drug allergy are recorded in EHRs. It remains difficult to reduce drug allergy alert overload while maintaining patient safety as the highest priority. Future research should focus on improving alert specificity, thereby reducing override rates and alert fatigue. Also, the effect on patient outcomes and cost-effectiveness should be evaluated.
| S-EPMC6231757 | biostudies-literature
Project description:BackgroundThe clinical decision-making process in pressure ulcer management is complex, and its quality depends on both the nurse's experience and the availability of scientific knowledge. This process should follow evidence-based practices incorporating health information technologies to assist health care professionals, such as the use of clinical decision support systems. These systems, in addition to increasing the quality of care provided, can reduce errors and costs in health care. However, the widespread use of clinical decision support systems still has limited evidence, indicating the need to identify and evaluate its effects on nursing clinical practice.ObjectiveThe goal of the review was to identify the effects of nurses using clinical decision support systems on clinical decision making for pressure ulcer management.MethodsThe systematic review was conducted in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) recommendations. The search was conducted in April 2019 on 5 electronic databases: MEDLINE, SCOPUS, Web of Science, Cochrane, and CINAHL, without publication date or study design restrictions. Articles that addressed the use of computerized clinical decision support systems in pressure ulcer care applied in clinical practice were included. The reference lists of eligible articles were searched manually. The Mixed Methods Appraisal Tool was used to assess the methodological quality of the studies.ResultsThe search strategy resulted in 998 articles, 16 of which were included. The year of publication ranged from 1995 to 2017, with 45% of studies conducted in the United States. Most addressed the use of clinical decision support systems by nurses in pressure ulcers prevention in inpatient units. All studies described knowledge-based systems that assessed the effects on clinical decision making, clinical effects secondary to clinical decision support system use, or factors that influenced the use or intention to use clinical decision support systems by health professionals and the success of their implementation in nursing practice.ConclusionsThe evidence in the available literature about the effects of clinical decision support systems (used by nurses) on decision making for pressure ulcer prevention and treatment is still insufficient. No significant effects were found on nurses' knowledge following the integration of clinical decision support systems into the workflow, with assessments made for a brief period of up to 6 months. Clinical effects, such as outcomes in the incidence and prevalence of pressure ulcers, remain limited in the studies, and most found clinically but nonstatistically significant results in decreasing pressure ulcers. It is necessary to carry out studies that prioritize better adoption and interaction of nurses with clinical decision support systems, as well as studies with a representative sample of health care professionals, randomized study designs, and application of assessment instruments appropriate to the professional and institutional profile. In addition, long-term follow-up is necessary to assess the effects of clinical decision support systems that can demonstrate a more real, measurable, and significant effect on clinical decision making.Trial registrationPROSPERO International Prospective Register of Systematic Reviews CRD42019127663; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=127663.
| S-EPMC7600011 | biostudies-literature
Project description:BackgroundElectronic health records are central to cancer care delivery. Electronic clinical decision support (CDS) systems can potentially improve cancer care quality and safety. However, little is known regarding the use of CDS systems in clinical oncology and their impact on patient outcomes.MethodsA systematic review of peer-reviewed studies was performed to evaluate clinically relevant outcomes related to the use of CDS tools for the diagnosis, treatment, and supportive care of patients with cancer. Peer-reviewed studies published from 1995 through 2016 were included if they assessed clinical outcomes, patient-reported outcomes (PROs), costs, or care delivery process measures.ResultsElectronic database searches yielded 2,439 potentially eligible papers, with 24 studies included after final review. Most studies used an uncontrolled, pre-post intervention design. A total of 23 studies reported improvement in key study outcomes with use of oncology CDS systems, and 12 studies assessing the systems for computerized chemotherapy order entry demonstrated reductions in prescribing error rates, medication-related safety events, and workflow interruptions. The remaining studies examined oncology clinical pathways, guideline adherence, systems for collection and communication of PROs, and prescriber alerts.ConclusionsThere is a paucity of data evaluating clinically relevant outcomes of CDS system implementation in oncology care. Currently available data suggest that these systems can have a positive impact on the quality of cancer care delivery. However, there is a critical need to rigorously evaluate CDS systems in oncology to better understand how they can be implemented to improve patient outcomes.
| S-EPMC6563614 | biostudies-literature