Characterizing the macrostructure of electronic health record work using raw audit logs: an unsupervised action embeddings approach.
Ontology highlight
ABSTRACT:
SUBMITTER: Lou SS
PROVIDER: S-EPMC9933072 | biostudies-literature | 2023 Feb
REPOSITORIES: biostudies-literature
Publications
Characterizing the macrostructure of electronic health record work using raw audit logs: an unsupervised action embeddings approach.
Lou Sunny S SS Liu Hanyang H Harford Derek D Lu Chenyang C Kannampallil Thomas T
Journal of the American Medical Informatics Association : JAMIA 20230201 3
Raw audit logs provide a comprehensive record of clinicians' activities on an electronic health record (EHR) and have considerable potential for studying clinician behaviors. However, research using raw audit logs is limited because they lack context for clinical tasks, leading to difficulties in interpretation. We describe a novel unsupervised approach using the comparison and visualization of EHR action embeddings to learn context and structure from raw audit log activities. Using a dataset of ...[more]
Similar Datasets
Project description:ObjectiveTo systematically review published literature and identify consistency and variation in the aims, measures, and methods of studies using electronic health record (EHR) audit logs to observe clinical activities.Materials and methodsIn July 2019, we searched PubMed for articles using EHR audit logs to study clinical activities. We coded and clustered the aims, measures, and methods of each article into recurring categories. We likewise extracted and summarized the methods used to validate measures derived from audit logs and limitations discussed of using audit logs for research.ResultsEighty-five articles met inclusion criteria. Study aims included examining EHR use, care team dynamics, and clinical workflows. Studies employed 6 key audit log measures: counts of actions captured by audit logs (eg, problem list viewed), counts of higher-level activities imputed by researchers (eg, chart review), activity durations, activity sequences, activity clusters, and EHR user networks. Methods used to preprocess audit logs varied, including how authors filtered extraneous actions, mapped actions to higher-level activities, and interpreted repeated actions or gaps in activity. Nineteen studies validated results (22%), but only 9 (11%) through direct observation, demonstrating varying levels of measure accuracy.DiscussionWhile originally designed to aid access control, EHR audit logs have been used to observe diverse clinical activities. However, most studies lack sufficient discussion of measure definition, calculation, and validation to support replication, comparison, and cross-study synthesis.ConclusionEHR audit logs have potential to scale observational research but the complexity of audit log measures necessitates greater methodological transparency and validated standards.
| S-EPMC7025338 | biostudies-literature
Project description:BackgroundElectronic health record (EHR) access and audit logs record behaviors of providers as they navigate the EHR. These data can be used to better understand provider responses to EHR-based clinical decision support (CDS), shedding light on whether and why CDS is effective.ObjectiveThis study aimed to determine the feasibility of using EHR access and audit logs to track primary care physicians' (PCPs') opening of and response to noninterruptive alerts delivered to EHR InBaskets.MethodsWe conducted a descriptive study to assess the use of EHR log data to track provider behavior. We analyzed data recorded following opening of 799 noninterruptive alerts sent to 75 PCPs' InBaskets through a prior randomized controlled trial. Three types of alerts highlighted new medication concerns for older patients' posthospital discharge: information only (n=593), medication recommendations (n=37), and test recommendations (n=169). We sought log data to identify the person opening the alert and the timing and type of PCPs' follow-up EHR actions (immediate vs by the end of the following day). We performed multivariate analyses examining associations between alert type, patient characteristics, provider characteristics, and contextual factors and likelihood of immediate or subsequent PCP action (general, medication-specific, or laboratory-specific actions). We describe challenges and strategies for log data use.ResultsWe successfully identified the required data in EHR access and audit logs. More than three-quarters of alerts (78.5%, 627/799) were opened by the PCP to whom they were directed, allowing us to assess immediate PCP action; of these, 208 alerts were followed by immediate action. Expanding on our analyses to include alerts opened by staff or covering physicians, we found that an additional 330 of the 799 alerts demonstrated PCP action by the end of the following day. The remaining 261 alerts showed no PCP action. Compared to information-only alerts, the odds ratio (OR) of immediate action was 4.03 (95% CI 1.67-9.72) for medication-recommendation and 2.14 (95% CI 1.38-3.32) for test-recommendation alerts. Compared to information-only alerts, ORs of medication-specific action by end of the following day were significantly greater for medication recommendations (5.59; 95% CI 2.42-12.94) and test recommendations (1.71; 95% CI 1.09-2.68). We found a similar pattern for OR of laboratory-specific action. We encountered 2 main challenges: (1) Capturing a historical snapshot of EHR status (number of InBasket messages at time of alert delivery) required incorporation of data generated many months prior with longitudinal follow-up. (2) Accurately interpreting data elements required iterative work by a physician/data manager team taking action within the EHR and then examining audit logs to identify corresponding documentation.ConclusionsEHR log data could inform future efforts and provide valuable information during development and refinement of CDS interventions. To address challenges, use of these data should be planned before implementing an EHR-based study.
| S-EPMC6383113 | biostudies-literature
Project description:User Identity Linkage (UIL) is the problem of matching user identities across multiple online social networks (OSNs) which belong to the same person. The solutions to UIL problem facilitate cross-platform research on OSN users and enable many useful applications such as user profiling and recommendation. As the UIL labeled data are often lacking and costly to obtain, learning user embeddings for matching user identities using an unsupervised approach is therefore highly desired. In this paper, we propose a novel unsupervised UIL framework for enhancing existing user embedding-based UIL methods. Our proposed framework incorporates two key ideas, user-discriminative features and retrofitting embedding. The user-discriminative features enable us to differentiate a specific user identity from other users in its OSN. From the user-discriminative features, we derive pairs of similar user identities across OSNs for retrofitting the base user embeddings of existing UIL methods. Through extensive experiments on three real-world OSN datasets, we show that our framework can leverage user-discriminative features to improve the accuracy of different user embedding-based UIL methods significantly. The quantum of improvement can also be surprisingly good even for existing UIL methods with very poor matching accuracy. Electronic supplementary material The online version of this chapter (10.1007/978-3-030-47426-3_30) contains supplementary material, which is available to authorized users.
| S-EPMC7206306 | biostudies-literature
Project description:SummarySkyline is a Windows application for targeted mass spectrometry method creation and quantitative data analysis. Like most graphical user interface (GUI) tools, it has a complex user interface with many ways for users to edit their files which makes the task of logging user actions challenging and is the reason why audit logging of every change is not common in GUI tools. We present an object comparison-based approach to audit logging for Skyline that is extensible to other GUI tools. The new audit logging system keeps track of all document modifications made through the GUI or the command line and displays them in an interactive grid. The audit log can also be uploaded and viewed in Panorama, a web repository for Skyline documents that can be configured to only accept documents with a valid audit log, based on embedded hashes to protect log integrity. This makes workflows involving Skyline and Panorama more reproducible.Availability and implementationSkyline is freely available at https://skyline.ms.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC7520049 | biostudies-literature
Project description:ObjectiveHospital capacity management depends on accurate real-time estimates of hospital-wide discharges. Estimation by a clinician requires an excessively large amount of effort and, even when attempted, accuracy in forecasting next-day patient-level discharge is poor. This study aims to support next-day discharge predictions with machine learning by incorporating electronic health record (EHR) audit log data, a resource that captures EHR users' granular interactions with patients' records by communicating various semantics and has been neglected in outcome predictions.Materials and methodsThis study focused on the EHR data for all adults admitted to Vanderbilt University Medical Center in 2019. We learned multiple advanced models to assess the value that EHR audit log data adds to the daily prediction of discharge likelihood within 24 h and to compare different representation strategies. We applied Shapley additive explanations to identify the most influential types of user-EHR interactions for discharge prediction.ResultsThe data include 26 283 inpatient stays, 133 398 patient-day observations, and 819 types of user-EHR interactions. The model using the count of each type of interaction in the recent 24 h and other commonly used features, including demographics and admission diagnoses, achieved the highest area under the receiver operating characteristics (AUROC) curve of 0.921 (95% CI: 0.919-0.923). By contrast, the model lacking user-EHR interactions achieved a worse AUROC of 0.862 (0.860-0.865). In addition, 10 of the 20 (50%) most influential factors were user-EHR interaction features.ConclusionEHR audit log data contain rich information such that it can improve hospital-wide discharge predictions.
| S-EPMC8633668 | biostudies-literature
Project description:Robust phenotyping of patients from electronic health records (EHRs) at scale is a challenge in clinical informatics. Here, we introduce Phe2vec, an automated framework for disease phenotyping from EHRs based on unsupervised learning and assess its effectiveness against standard rule-based algorithms from Phenotype KnowledgeBase (PheKB). Phe2vec is based on pre-computing embeddings of medical concepts and patients' clinical history. Disease phenotypes are then derived from a seed concept and its neighbors in the embedding space. Patients are linked to a disease if their embedded representation is close to the disease phenotype. Comparing Phe2vec and PheKB cohorts head-to-head using chart review, Phe2vec performed on par or better in nine out of ten diseases. Differently from other approaches, it can scale to any condition and was validated against widely adopted expert-based standards. Phe2vec aims to optimize clinical informatics research by augmenting current frameworks to characterize patients by condition and derive reliable disease cohorts.
| S-EPMC8441576 | biostudies-literature
Project description:MotivationProtein function prediction is a difficult bioinformatics problem. Many recent methods use deep neural networks to learn complex sequence representations and predict function from these. Deep supervised models require a lot of labeled training data which are not available for this task. However, a very large amount of protein sequences without functional labels is available.ResultsWe applied an existing deep sequence model that had been pretrained in an unsupervised setting on the supervised task of protein molecular function prediction. We found that this complex feature representation is effective for this task, outperforming hand-crafted features such as one-hot encoding of amino acids, k-mer counts, secondary structure and backbone angles. Also, it partly negates the need for complex prediction models, as a two-layer perceptron was enough to achieve competitive performance in the third Critical Assessment of Functional Annotation benchmark. We also show that combining this sequence representation with protein 3D structure information does not lead to performance improvement, hinting that 3D structure is also potentially learned during the unsupervised pretraining.Availability and implementationImplementations of all used models can be found at https://github.com/stamakro/GCN-for-Structure-and-Function.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC8055213 | biostudies-literature
Project description:Background In the neonatal intensive care unit (NICU), predefined acuity-based team care models are restricted to core roles and neglect interactions with providers outside of the team, such as interactions that transpire via electronic health record (EHR) systems. These unaccounted interactions may be related to the efficiency of resource allocation, information flow, communication, and thus impact patient outcomes. This study applied network analysis methods to EHR audit logs to model the interactions of providers beyond their core roles to better understand the interaction network patterns of acuity-based teams and relationships of the network structures with postsurgical length of stay (PSLOS).Methods The study used the EHR log data of surgical neonates from a large academic medical center. The study included 104 surgical neonates, for whom 9,206 unique actions were performed by 457 providers in their EHRs. We applied network analysis methods to model EHR provider interaction networks of acuity-based teams in NICU postoperative care. We partitioned each EHR network into three subnetworks based on interaction types: (1) interactions between known core providers who were documented in scheduling records (core subnetwork); (2) interactions between core and noncore providers (extended subnetwork); and (3) interactions between noncore providers (extended subnetwork). For each core subnetwork, we assessed its capability to replicate predefined core-provider relations as documented in scheduling records. We further compared each EHR network, as well as its subnetworks, using standard network measures to determine its differences in network topologies. We conducted a case study to learn provider interaction networks taking care of 15 neonates who underwent gastrostomy tube placement surgery from EHR log data and measure the effectiveness of the interaction networks on PSLOS by the proportional-odds model.Results The provider networks of four acuity-based teams (two high and two low acuity), along with their subnetworks, were discovered. We found that beyond capturing the predefined core-provider relations, EHR audit logs can also learn a large number of relations between core and noncore providers or among noncore providers. Providers in the core subnetwork exhibited a greater number of connections with each other than with providers in the extended subnetworks. Many more providers in the core subnetwork serve as a hub than those in the other types of subnetworks. We also found that high-acuity teams exhibited more complex network structures than low-acuity teams, with high-acuity team generating 6,416 interactions between 407 providers compared with 931 interactions between 124 providers, respectively. In addition, we discovered that high-acuity and low-acuity teams shared more than 33 and 25% of providers with each other, respectively, but exhibited different collaborative structures demonstrating that NICU providers shift across different acuity teams and exhibit different network characteristics. Results of case study show that providers, whose patients had lower PSLOS, tended to disperse patient-related information to more colleagues within their network than those who treated higher PSLOS patients (p = 0.03).Conclusion Network analysis can be applied to EHR log data to model acuity-based NICU teams capturing interactions between providers within the predesigned core team as well as those outside of the core team. In the NICU, dissemination of information may be linked to reduced PSLOS. EHR log data provide an efficient, accessible, and research-friendly way to study provider interaction networks. Findings should guide improvements in the EHR system design to facilitate effective interactions between providers.
| S-EPMC7356932 | biostudies-literature
Project description:Introduction: Oxford Nanopore Technologies (ONT) is a third generation sequencing approach that allows the analysis of individual, full-length nucleic acids. ONT records the alterations of an ionic current flowing across a nano-scaled pore while a DNA or RNA strand is threading through the pore. Basecalling methods are then leveraged to translate the recorded signal back to the nucleic acid sequence. However, basecall generally introduces errors that hinder the process of barcode demultiplexing, a pivotal task in single-cell RNA sequencing that allows for separating the sequenced transcripts on the basis of their cell of origin. Methods: To solve this issue, we present a novel framework, called UNPLEX, designed to tackle the barcode demultiplexing problem by operating directly on the recorded signals. UNPLEX combines two unsupervised machine learning methods: autoencoders and self-organizing maps (SOM). The autoencoders extract compact, latent representations of the recorded signals that are then clustered by the SOM. Results and Discussion: Our results, obtained on two datasets composed of in silico generated ONT-like signals, show that UNPLEX represents a promising starting point for the development of effective tools to cluster the signals corresponding to the same cell.
| S-EPMC10173771 | biostudies-literature
Project description:Time spent by physicians is a key resource in health care delivery. This study used data captured by the access time stamp functionality of an electronic health record (EHR) to examine physician work effort. This is a potentially powerful, yet unobtrusive, way to study physicians' use of time. We used data on physicians' time allocation patterns captured by over thirty-one million EHR transactions in the period 2011-14 recorded by 471 primary care physicians, who collectively worked on 765,129 patients' EHRs. Our results suggest that the physicians logged an average of 3.08 hours on office visits and 3.17 hours on desktop medicine each day. Desktop medicine consists of activities such as communicating with patients through a secure patient portal, responding to patients' online requests for prescription refills or medical advice, ordering tests, sending staff messages, and reviewing test results. Over time, log records from physicians showed a decline in the time allocated to face-to-face visits, accompanied by an increase in time allocated to desktop medicine. Staffing and scheduling in the physician's office, as well as provider payment models for primary care practice, should account for these desktop medicine efforts.
| S-EPMC5546411 | biostudies-literature