Dataset Information

Leveraging artificial intelligence in neurosurgery-unveiling ChatGPT for neurosurgical discharge summaries and operative reports.

ABSTRACT:

Purpose

Chat generative pre-trained transformer (GPT) is a novel large pre-trained natural language processing software that can enable scientific writing amongst a litany of other features. Given this, there is a growing interest in exploring the use of ChatGPT models as a modality to facilitate/assist in the provision of clinical care.

Methods

We investigated the time taken for the composition of neurosurgical discharge summaries and operative reports at a major University hospital. In so doing, we compared currently employed speech recognition software (i.e., SpeaKING) vs novel ChatGPT for three distinct neurosurgical diseases: chronic subdural hematoma, spinal decompression, and craniotomy. Furthermore, factual correctness was analyzed for the abovementioned diseases.

Results

The composition of neurosurgical discharge summaries and operative reports with the assistance of ChatGPT leads to a statistically significant time reduction across all three diseases/report types: p < 0.001 for chronic subdural hematoma, p < 0.001 for decompression of spinal stenosis, and p < 0.001 for craniotomy and tumor resection. However, despite a high degree of factual correctness, the preparation of a surgical report for craniotomy proved to be significantly lower (p = 0.002).

Conclusion

ChatGPT assisted in the writing of discharge summaries and operative reports as evidenced by an impressive reduction in time spent as compared to standard speech recognition software. While promising, the optimal use cases and ethics of AI-generated medical writing remain to be fully elucidated and must be further explored in future studies.

SUBMITTER: Dubinski D

PROVIDER: S-EPMC10817836 | biostudies-literature | 2024 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Leveraging artificial intelligence in neurosurgery-unveiling ChatGPT for neurosurgical discharge summaries and operative reports.

Dubinski Daniel D Won Sae-Yeon SY Trnovec Svorad S Behmanesh Bedjan B Baumgarten Peter P Dinc Nazife N Konczalla Juergen J Chan Alvin A Bernstock Joshua D JD Freiman Thomas M TM Gessler Florian F

Acta neurochirurgica 20240126 1

<h4>Purpose</h4>Chat generative pre-trained transformer (GPT) is a novel large pre-trained natural language processing software that can enable scientific writing amongst a litany of other features. Given this, there is a growing interest in exploring the use of ChatGPT models as a modality to facilitate/assist in the provision of clinical care.<h4>Methods</h4>We investigated the time taken for the composition of neurosurgical discharge summaries and operative reports at a major University hospi ...[more]

PMID: 38277081

Similar Datasets

Project description:ImportanceBy law, patients have immediate access to discharge notes in their medical records. Technical language and abbreviations make notes difficult to read and understand for a typical patient. Large language models (LLMs [eg, GPT-4]) have the potential to transform these notes into patient-friendly language and format.ObjectiveTo determine whether an LLM can transform discharge summaries into a format that is more readable and understandable.Design, setting, and participantsThis cross-sectional study evaluated a sample of the discharge summaries of adult patients discharged from the General Internal Medicine service at NYU (New York University) Langone Health from June 1 to 30, 2023. Patients discharged as deceased were excluded. All discharge summaries were processed by the LLM between July 26 and August 5, 2023.InterventionsA secure Health Insurance Portability and Accountability Act-compliant platform, Microsoft Azure OpenAI, was used to transform these discharge summaries into a patient-friendly format between July 26 and August 5, 2023.Main outcomes and measuresOutcomes included readability as measured by Flesch-Kincaid Grade Level and understandability using Patient Education Materials Assessment Tool (PEMAT) scores. Readability and understandability of the original discharge summaries were compared with the transformed, patient-friendly discharge summaries created through the LLM. As balancing metrics, accuracy and completeness of the patient-friendly version were measured.ResultsDischarge summaries of 50 patients (31 female [62.0%] and 19 male [38.0%]) were included. The median patient age was 65.5 (IQR, 59.0-77.5) years. Mean (SD) Flesch-Kincaid Grade Level was significantly lower in the patient-friendly discharge summaries (6.2 [0.5] vs 11.0 [1.5]; P < .001). PEMAT understandability scores were significantly higher for patient-friendly discharge summaries (81% vs 13%; P < .001). Two physicians reviewed each patient-friendly discharge summary for accuracy on a 6-point scale, with 54 of 100 reviews (54.0%) giving the best possible rating of 6. Summaries were rated entirely complete in 56 reviews (56.0%). Eighteen reviews noted safety concerns, mostly involving omissions, but also several inaccurate statements (termed hallucinations).Conclusions and relevanceThe findings of this cross-sectional study of 50 discharge summaries suggest that LLMs can be used to translate discharge summaries into patient-friendly language and formats that are significantly more readable and understandable than discharge summaries as they appear in electronic health records. However, implementation will require improvements in accuracy, completeness, and safety. Given the safety concerns, initial implementation will require physician review.

Project description:ObjectivePatients now have direct access to their radiology reports, which can include complex terminology and be difficult to understand. We assessed ChatGPT's ability to generate summarized MRI reports for patients with prostate cancer and evaluated physician satisfaction with the artificial intelligence (AI)-summarized report.MethodsWe used ChatGPT to summarize five full MRI reports for patients with prostate cancer performed at a single institution from 2021 to 2022. Three summarized reports were generated for each full MRI report. Full MRI and summarized reports were assessed for readability using Flesch-Kincaid Grade Level (FK) score. Radiation oncologists were asked to evaluate the AI-summarized reports via an anonymous questionnaire. Qualitative responses were given on a 1-5 Likert-type scale. Fifty newly diagnosed prostate cancer patient MRIs performed at a single institution were additionally assessed for physician online portal response rates.ResultsFifteen summarized reports were generated from five full MRI reports using ChatGPT. The median FK score for the full MRI reports and summarized reports was 9.6 vs. 5.0, (p < 0.05), respectively. Twelve radiation oncologists responded to our questionnaire. The mean [SD] ratings for summarized reports were factual correctness (4.0 [0.6], understanding 4.0 [0.7]), completeness (4.1 [0.5]), potential for harm (3.5 [0.9]), overall quality (3.4 [0.9]), and likelihood to send to patient (3.1 [1.1]). Current physician online portal response rates were 14/50 (28%) at our institution.ConclusionsWe demonstrate a novel application of ChatGPT to summarize MRI reports at a reading level appropriate for patients. Physicians were likely to be satisfied with the summarized reports with respect to factual correctness, ease of understanding, and completeness. Physicians were less likely to be satisfied with respect to potential for harm, overall quality, and likelihood to send to patients. Further research is needed to optimize ChatGPT's ability to summarize radiology reports and understand what factors influence physician trust in AI-summarized reports.

Project description:BackgroundAcute kidney injury (AKI) increases the risk of hospital readmission, chronic kidney disease, and death. Therefore, effective communication in discharge summaries is essential for safe transitions of care.ObjectiveThe objectives of this study were to determine the quality of discharge summaries in AKI survivors and identify predictors of higher quality discharge summaries.DesignRetrospective chart review.SettingTertiary care academic center in Ontario, Canada.PatientsWe examined the discharge summary quality of 300 randomly selected adult patients who survived a hospitalization with AKI at our tertiary care hospital, stratified by AKI severity. We included 150 patients each from 2015 to 2016 and 2018 to 2019, before and after introduction of a post-AKI clinic in 2017.MeasurementsWe reviewed charts for 9 elements of AKI care to create a composite score summarizing discharge summary quality.MethodsWe used multivariable logistic regression to identify predictors of discharge summary quality.ResultsThe median discharge summary composite score was 4/9 (interquartile range, 2-6). The least frequently mentioned elements were baseline creatinine (n = 55, 18%), AKI-specific follow-up labs (n = 66, 22%), and medication recommendations (n = 80, 27%). The odds of having a higher quality discharge summary (composite score ≥4/9) was greater for every increase in baseline creatinine of 25 μmol/L (adjusted odds ratio [aOR]: 1.27; 95% confidence interval [CI]: 1.03, 1.56), intrarenal etiology (aOR: 2.32; 95% CI: 1.26, 4.27), and increased AKI severity (stage 2 aOR: 2.57; 95% CI: 1.35, 4.91 and stage 3 aOR: 3.36; 95% CI: 1.56, 7.22). There was no association between discharge summary quality and the years before and after introduction of a post-AKI clinic (aOR: 0.77; 95% CI: 0.46, 1.29).LimitationsThe single-center study design limits generalizability.ConclusionsMost discharge summaries are missing key AKI elements, even in patients with severe AKI. These gaps suggest several opportunities exist to improve discharge summary communication following AKI.

Project description:Discharge summaries are a vital tool to communicate information from Hospital to Primary Care teams; updating GPs about what happened during an admission, and handing over care detailing any follow up care required. Historically, Discharge Summaries have been posted to hospitals, increasing costs for hospitals, creating administrative work for GP practices receiving the letters, and resulting in some letters being lost or delayed in reaching the GP, with implications for patient safety if follow up requests are not received and acted upon. In an effort to improve patient care, the Clinical Commissioning Group in Surrey drew up a contract with Ashford and St Peter's Foundation Trust, aiming to increase the percentage of discharge summaries sent electronically from the rate of 9% sent within 24 hours, to over 75%. This contract set targets of 50% in May, 65% in June, and 80% in July. Financial penalties would be imposed if targets were not achieved, starting in June 2013. The Trust set up a working group comprising of doctors, IT personnel and ward PAs to devise a multi-pronged solution to achieve this target. The electronic discharge summary system was reviewed and improvements were designed and developed to make the process of signing off letters easier, and transmission of signed off letters became automated rather than requiring manual transmission by ward PAs. Presentations and leaflets to explain the importance of prompt completion and transmission of discharge summaries were given to Doctors to improve compliance using the revised IT system. Figures on transmission rates were automatically emailed to key stakeholders every day (Ward PAs, Divisional Leads) showing performance on each ward. This helped identify areas requiring more intervention. Areas (e.g. Day Surgery) that had not used electronic discharge summaries were engaged with, and persuaded to take part. As a result, transmission rates of Discharge Summaries within 24 hours of patient discharge increased from 9% on May 11th 2013, to 76% by June 29th 2013. This has improved communication with GPs, led to more reliable handover of care, and reduced costs for the Trust (both in processing and postage costs, and by avoiding fines).

Dataset Information

Leveraging artificial intelligence in neurosurgery-unveiling ChatGPT for neurosurgical discharge summaries and operative reports.

Purpose

Methods

Results

Conclusion

Publications

Leveraging artificial intelligence in neurosurgery-unveiling ChatGPT for neurosurgical discharge summaries and operative reports.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets