Dataset Information

Integration of Face-to-Face Screening With Real-time Machine Learning to Predict Risk of Suicide Among Adults.

ABSTRACT:

Importance

Understanding the differences and potential synergies between traditional clinician assessment and automated machine learning might enable more accurate and useful suicide risk detection.

Objective

To evaluate the respective and combined abilities of a real-time machine learning model and the Columbia Suicide Severity Rating Scale (C-SSRS) to predict suicide attempt (SA) and suicidal ideation (SI).

Design, setting, and participants

This cohort study included encounters with adult patients (aged ≥18 years) at a major academic medical center. The C-SSRS was administered during routine care, and a Vanderbilt Suicide Attempt and Ideation Likelihood (VSAIL) prediction was generated in the electronic health record. Encounters took place in the inpatient, ambulatory surgical, and emergency department settings. Data were collected from June 2019 to September 2020.

Main outcomes and measures

Primary outcomes were the incidence of SA and SI, encoded as International Classification of Diseases codes, occurring within various time periods after an index visit. We evaluated the retrospective validity of the C-SSRS, VSAIL, and ensemble models combining both. Discrimination metrics included area under the receiver operating curve (AUROC), area under the precision-recall curve (AUPR), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

Results

The cohort included 120 398 unique index visits for 83 394 patients (mean [SD] age, 51.2 [20.6] years; 38 107 [46%] men; 45 273 [54%] women; 13 644 [16%] Black; 63 869 [77%] White). Within 30 days of an index visit, the combined models had higher AUROC (SA: 0.874-0.887; SI: 0.869-0.879) than both the VSAIL (SA: 0.729; SI: 0.773) and C-SSRS (SA: 0.823; SI: 0.777) models. In the highest risk-decile, ensemble methods had PPV of 1.3% to 1.4% for SA and 8.3% to 8.7% for SI and sensitivity of 77.6% to 79.5% for SA and 67.4% to 70.1% for SI, outperforming VSAIL (PPV for SA: 0.4%; PPV for SI: 3.9%; sensitivity for SA: 28.8%; sensitivity for SI: 35.1%) and C-SSRS (PPV for SA: 0.5%; PPV for SI: 3.5%; sensitivity for SA: 76.6%; sensitivity for SI: 68.8%).

Conclusions and relevance

In this study, suicide risk prediction was optimal when leveraging both in-person screening (for acute measures of risk in patient-reported suicidality) and historical EHR data (for underlying clinical factors that can quantify a patient's passive risk level). To improve suicide risk classification, prediction systems could combine pretrained machine learning with structured clinician assessment without needing to retrain the original model.

SUBMITTER: Wilimitis D

PROVIDER: S-EPMC9107032 | biostudies-literature | 2022 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Integration of Face-to-Face Screening With Real-time Machine Learning to Predict Risk of Suicide Among Adults.

Wilimitis Drew D Turer Robert W RW Ripperger Michael M McCoy Allison B AB Sperry Sarah H SH Fielstein Elliot M EM Kurz Troy T Walsh Colin G CG

JAMA network open 20220502 5

<h4>Importance</h4>Understanding the differences and potential synergies between traditional clinician assessment and automated machine learning might enable more accurate and useful suicide risk detection.<h4>Objective</h4>To evaluate the respective and combined abilities of a real-time machine learning model and the Columbia Suicide Severity Rating Scale (C-SSRS) to predict suicide attempt (SA) and suicidal ideation (SI).<h4>Design, setting, and participants</h4>This cohort study included enco ...[more]

PMID: 35560048

Similar Datasets

Project description:ImportanceEvidence suggests that limiting access to firearms among individuals at high risk of suicide can be an effective means of suicide prevention, yet accurately identifying those at risk to intervene remains a key challenge. Firearm purchasing records may offer a large-scale and objective data source for the development of tools to predict firearm suicide risk.ObjectiveTo test whether a statewide database of handgun transaction records, coupled with machine learning techniques, can be used to forecast firearm suicide risk.Design, setting, and participantsThis prognostic study used the California database of 4 976 391 handgun transaction records from 1 951 006 individuals from January 1, 1996, to October 6, 2015. Transaction-level random forest classification was implemented to predict firearm suicide risk, and the relative predictive power of features in the algorithm was estimated via permutation importance. Analyses were performed from December 1, 2020, to May 19, 2022.Main outcomes and measuresThe main outcome was firearm suicide within 1 year of a firearm transaction, derived from California death records (1996-2016). With the use of California's Dealer's Records of Sale (1996-2015), 41 handgun, transaction, purchaser, and community-level predictor variables were generated.ResultsThere are a total of 4 976 391 transactions in the California's Dealer's Record of Sale database representing 1 951 006 individuals (1 525 754 men [78.2% of individuals]; mean [SD] age, 43.4 [13.9] years). Firearm suicide within 1 year occurred in 0.07% of handgun transactions (3278 transactions among 2614 individuals). A total of 38.6% of observed firearm suicides were among transactions classified in the highest-risk ventile (379 of 983 transactions), with 95% specificity. Among the small number of transactions with a random forest score above 0.95, more than two-thirds (24 of 35 [68.6%]) were associated with a purchaser who died by firearm suicide within 1 year. Important features included known risk factors, such as older age at first purchase, and previously unreported predictors, including distance to firearms dealer and month of purchase.Conclusions and relevanceThis prognostic study presented the first large-scale machine learning analysis of individual-level handgun transaction records. The results suggested the potential utility of such records in identifying high-risk individuals to aid suicide prevention efforts. It also identified handgun, individual, and community characteristics that have strong predictive relationships with firearm suicide and may warrant further study.

Project description:BackgroundCurrent depression, anxiety, and suicide screening techniques rely on retrospective patient reported symptoms to standardized scales. A qualitative approach to screening combined with the innovation of natural language processing (NLP) and machine learning (ML) methods have shown promise to enhance person-centeredness while detecting depression, anxiety, and suicide risk from in-the-moment patient language derived from an open-ended brief interview.ObjectiveTo evaluate the performance of NLP/ML models to identify depression, anxiety, and suicide risk from a single 5-10-min semi-structured interview with a large, national sample.MethodTwo thousand four hundred sixteen interviews were conducted with 1,433 participants over a teleconference platform, with 861 (35.6%), 863 (35.7%), and 838 (34.7%) sessions screening positive for depression, anxiety, and suicide risk, respectively. Participants completed an interview over a teleconference platform to collect language about the participants' feelings and emotional state. Logistic regression (LR), support vector machine (SVM), and extreme gradient boosting (XGB) models were trained for each condition using term frequency-inverse document frequency features from the participants' language. Models were primarily evaluated with the area under the receiver operating characteristic curve (AUC).ResultsThe best discriminative ability was found when identifying depression with an SVM model (AUC = 0.77; 95% CI = 0.75-0.79), followed by anxiety with an LR model (AUC = 0.74; 95% CI = 0.72-0.76), and an SVM for suicide risk (AUC = 0.70; 95% CI = 0.68-0.72). Model performance was generally best with more severe depression, anxiety, or suicide risk. Performance improved when individuals with lifetime but no suicide risk in the past 3 months were considered controls.ConclusionIt is feasible to use a virtual platform to simultaneously screen for depression, anxiety, and suicide risk using a 5-to-10-min interview. The NLP/ML models performed with good discrimination in the identification of depression, anxiety, and suicide risk. Although the utility of suicide risk classification in clinical settings is still undetermined and suicide risk classification had the lowest performance, the result taken together with the qualitative responses from the interview can better inform clinical decision-making by providing additional drivers associated with suicide risk.

Project description:ObjectiveSeveral prognostic models of suicide risk have been published; however, few have been implemented in Japan using longitudinal cohort data. The aim of this study was to identify suicide risk factors for suicidal ideation in the Japanese population and to develop a machine-learning model to predict suicide risk in Japan.Materials and methodsData was obtained from Wave1 Time 1 (November 2016) and Time 2 (March 2017) of the National Survey for Stress and Health in Japan, were incorporated into a suicide risk prediction machine-learning model, trained using 65 items related to trauma and stress. The study included 3,090 and 2,163 survey respondents >18 years old at Time 1 and Time 2, respectively. The mean (standard deviation, SD) age was 44.9 (10.9) years at Time 1 and 46.0 (10.7) years at Time 2. We analyzed the participants with increased suicide risk at Time 2 survey. Model performance, including the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, were also analyzed.ResultsThe model showed a good performance (AUC = 0.830, 95% confidence interval = 0.795-0.866). Overall, the model achieved an accuracy of 78.8%, sensitivity of 75.4%, specificity of 80.4%, positive predictive value of 63.4%, and negative predictive value of 87.9%. The most important risk factor for suicide risk was the participants' Suicidal Ideation Attributes Scale score, followed by the Sheehan Disability Scale score, Patient Health Questionnaire-9 scores, Cross-Cutting Symptom Measure (CCSM-suicidal ideation domain, Dissociation Experience Scale score, history of self-harm, Generalized Anxiety Disorder-7 score, Post-Traumatic Stress Disorder check list-5 score, CCSM-dissociation domain, and Impact of Event Scale-Revised scores at Time 1.ConclusionsThis prognostic study suggests the ability to identify patients at a high risk of suicide using an online survey method. In addition to confirming several well-known risk factors of suicide, new risk measures related to trauma and trauma-related experiences were also identified, which may help guide future clinical assessments and early intervention approaches.

Dataset Information

Integration of Face-to-Face Screening With Real-time Machine Learning to Predict Risk of Suicide Among Adults.

Importance

Objective

Design, setting, and participants

Main outcomes and measures

Results

Conclusions and relevance

Publications

Integration of Face-to-Face Screening With Real-time Machine Learning to Predict Risk of Suicide Among Adults.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets