Dataset Information

Predicting physician burnout using clinical activity logs: Model performance and lessons learned.

ABSTRACT:

Background

Burnout is a significant public health concern affecting more than half of the healthcare workforce; however, passive screening tools to detect burnout are lacking. We investigated the ability of machine learning (ML) techniques to identify burnout using passively collected electronic health record (EHR)-based audit log data.

Method

Physician trainees participated in a longitudinal study where they completed monthly burnout surveys and provided access to their EHR-based audit logs. Using the monthly burnout scores as the target outcome, we trained ML models using combinations of features derived from audit log data-aggregate measures of clinical workload, time series-based temporal measures of EHR use, and the baseline burnout score. Five ML models were constructed to predict burnout as a continuous score: penalized linear regression, support vector machine, neural network, random forest, and gradient boosting machine.

Results

88 trainee physicians participated and completed 416 surveys; greater than10 million audit log actions were collected (Mean [Standard Deviation] = 25,691 [14,331] actions per month, per physician). The workload feature set predicted burnout score with a mean absolute error (MAE) of 0.602 (95% Confidence Interval (CI), 0.412-0.826), and was able to predict burnout status with an average AUROC of 0.595 (95% CI 0.355-0.808) and average accuracy 0.567 (95% CI 0.393-0.742). The temporal feature set had a similar performance, with MAE 0.596 (95% CI 0.391-0.826), and AUROC 0.581 (95% CI 0.343-0.790). The addition of the baseline burnout score to the workload features improved the model performance to a mean AUROC of 0.829 (95% CI 0.607-0.996) and mean accuracy of 0.781 (95% CI 0.587-0.936); however, this performance was not meaningfully different than using the baseline burnout score alone.

Conclusions

Current findings illustrate the complexities of predicting burnout exclusively based on clinical work activities as captured in the EHR, highlighting its multi-factorial and individualized nature. Future prediction studies of burnout should account for individual factors (e.g., resilience, physiological measurements such as sleep) and associated system-level factors (e.g., leadership).

SUBMITTER: Lou SS

PROVIDER: S-EPMC8901565 | biostudies-literature | 2022 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Predicting physician burnout using clinical activity logs: Model performance and lessons learned.

Lou Sunny S SS Liu Hanyang H Warner Benjamin C BC Harford Derek D Lu Chenyang C Kannampallil Thomas T

Journal of biomedical informatics 20220205

<h4>Background</h4>Burnout is a significant public health concern affecting more than half of the healthcare workforce; however, passive screening tools to detect burnout are lacking. We investigated the ability of machine learning (ML) techniques to identify burnout using passively collected electronic health record (EHR)-based audit log data.<h4>Method</h4>Physician trainees participated in a longitudinal study where they completed monthly burnout surveys and provided access to their EHR-based ...[more]

PMID: 35134568

Similar Datasets

Project description:BackgroundAudit and group feedback (A&F) is an instrument used to encourage healthcare professionals to improve the quality of care. Clinical practice was audited against a set of criteria and fed back to a group by a facilitator. The aim of this study was to gain a better understanding of how physician group feedback sessions in nursing homes were conducted and to what extent they resulted in action planning.MethodsFifteen group feedback sessions of the antibiotic A&F program within a nursing home network were audio-recorded, transcribed, and analyzed via the Framework Method for thematic analysis. The coding was performed using the existing Calgary A&F Framework and Cooke's conceptual model of physician behaviors, and open inductive codes were added.ResultsElements of the conceptual model and the Calgary A&F Framework occurred within all group feedback sessions. The relationships within the group and with the facilitators were important elements when moving a group from interpreting the results to formulating action plans. Physician groups responded positively to the audit data, particularly if they were among the best performing. The data were met with doubt by physicians who did not recognize their own practice. When exploring potential reasons for lower guideline adherence, groups often considered data quality or external factors such as the choice of non-adherent treatment by locum staff. The degree of reflection on personal factors as explanations for low adherence and the extent to which groups identified learning and improvement opportunities varied: some groups were able to formulate action plans to address data problems and knowledge gaps, whereas others scheduled a follow-up meeting to develop action plans for treatment or prescribing practice changes.ConclusionsThe facilitator was crucial in supporting the group in interpreting the results, steering the conversation towards sharing change cues, and helping the physician group in developing action plans. The degree of reflection and action planning varied by group. By implementing the lessons learned from this study, group feedback sessions can be refined, supporting participants in action planning.

Project description:Although previous studies suggest an association between provider burnout and suboptimal self-reported communication, no studies relate physician burnout to observed patient-physician communication behaviors.To investigate the relationship between physician burnout and observed patient-physician communication outcomes in patient-physician encounters.Longitudinal study of enrollment data from a trial of interventions to improve patient adherence to hypertension treatment.Fifteen urban community-based clinics in Baltimore, MD.Forty physicians and 235 of their adult hypertensive patients, with oversampling of ethnic minorities and poor persons. Fifty-three percent of physicians were women, and the average practice experience was 11.2 years. Among the 235 patients, 66% were women, 60% were African-American, and 90% were insured.Audiotape analysis of communication during outpatient encounters (one per patient) using the Roter Interaction Analysis System and patients' ratings of satisfaction with and trust and confidence in the physician.The median time between the physician burnout assessment and the patient encounter was 15.1 months (range 5.6-30). Multivariate analyses revealed no significant differences in physician communication based on physician burnout. However, compared with patients of low-burnout physicians, patients of high-burnout physicians gave twice as many negative rapport-building statements (incident risk ratio 2.06, 95% CI 1.58-2.86, p < 0.001). Physician burnout was not significantly associated with physician or patient affect, patient-centeredness, verbal dominance, or length of the encounter. Physician burnout was also not significantly associated with patients' ratings of their satisfaction, confidence, or trust.Physician burnout was not associated with physician communication behaviors nor with most measures of patient-centered communication. However, patients engaged in more rapport-building behaviors. These findings suggest a complex relationship between physician burnout and patient-physician communication, which should be investigated and linked to patient outcomes in future research.

Dataset Information

Predicting physician burnout using clinical activity logs: Model performance and lessons learned.

Background

Method

Results

Conclusions

Publications

Predicting physician burnout using clinical activity logs: Model performance and lessons learned.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets