Dataset Information

Evaluation of a mandatory phishing training program for high-risk employees at a US healthcare system.

ABSTRACT:

Objective

The study sought to understand the impact of a phishing training program on phishing click rates for employees at a single, anonymous US healthcare institution.

Materials and methods

We stratified our population into 2 groups: offenders and nonoffenders. Offenders were defined as those that had clicked on at least 5 simulated phishing emails and nonoffenders were those that had not. We calculated click rates for offenders and nonoffenders, before and after a mandatory training program for offenders was implemented.

Results

A total of 5416 unique employees received all 20 campaigns during the intervention period; 772 clicked on at least 5 emails and were labeled offenders. Only 975 (17.9%) of our set clicked on 0 phishing emails over the course of the 20 campaigns; 3565 (65.3%) clicked on at least 2 emails. There was a decrease in click rates for each group over the 20 campaigns. The mandatory training program, initiated after campaign 15, did not have a substantial impact on click rates, and the offenders remained more likely to click on a phishing simulation.

Discussion

Phishing is a common threat vector against hospital employees and an important cybersecurity risk to healthcare systems. Our work suggests that, under simulation, employee click rates decrease with repeated simulation, but a mandatory training program targeted at high-risk employees did not meaningfully decrease the click rates of this population.

Conclusions

Employee phishing click rates decrease over time, but a mandatory training program for the highest-risk employees did not decrease click rates when compared with lower-risk employees.

SUBMITTER: Gordon WJ

PROVIDER: S-EPMC6515532 | biostudies-literature | 2019 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation of a mandatory phishing training program for high-risk employees at a US healthcare system.

Gordon William J WJ Wright Adam A Glynn Robert J RJ Kadakia Jigar J Mazzone Christina C Leinbach Elizabeth E Landman Adam A

Journal of the American Medical Informatics Association : JAMIA 20190601 6

<h4>Objective</h4>The study sought to understand the impact of a phishing training program on phishing click rates for employees at a single, anonymous US healthcare institution.<h4>Materials and methods</h4>We stratified our population into 2 groups: offenders and nonoffenders. Offenders were defined as those that had clicked on at least 5 simulated phishing emails and nonoffenders were those that had not. We calculated click rates for offenders and nonoffenders, before and after a mandatory tr ...[more]

PMID: 30861069

Similar Datasets

Project description:BACKGROUND:Hospitals have been one of the major targets for phishing attacks. Despite efforts to improve information security compliance, hospitals still significantly suffer from such attacks, impacting the quality of care and the safety of patients. OBJECTIVE:This study aimed to investigate why hospital employees decide to click on phishing emails by analyzing actual clicking data. METHODS:We first gauged the factors that influence clicking behavior using the theory of planned behavior (TPB) and integrating trust theories. We then conducted a survey in hospitals and used structural equation modeling to investigate the components of compliance intention. We matched employees' survey results with their actual clicking data from phishing campaigns. RESULTS:Our analysis (N=397) reveals that TPB factors (attitude, subjective norms, and perceived behavioral control), as well as collective felt trust and trust in information security technology, are positively related to compliance intention. However, compliance intention is not significantly related to compliance behavior. Only the level of employees' workload is positively associated with the likelihood of employees clicking on a phishing link. CONCLUSIONS:This is one of the few studies in information security and decision making that observed compliance behavior by analyzing clicking data rather than using self-reported data. We show that, in the context of phishing emails, intention and compliance might not be as strongly linked as previously assumed; hence, hospitals must remain vigilant with vulnerabilities that cannot be easily managed. Importantly, given the significant association between workload and noncompliance behavior (ie, clicking on phishing links), hospitals should better manage employees' workload to increase information security. Our findings can help health care organizations augment employees' compliance with their cybersecurity policies and reduce the likelihood of clicking on phishing links.

Project description:BackgroundThe Coronavirus Disease 2019 (COVID-19) has severely challenged healthcare delivery systems worldwide. Healthcare Workers were unable to assess and manage the cases due to limited knowledge of treating the virus and inadequate infrastructure. Digital interventions played a crucial role in the training of healthcare workers to get through the pandemic. Project Extension for Community Healthcare Outcomes (ECHO) initiated the COVID-ECHO telementoring program for strengthening the knowledge and skills of healthcare workers. The study aimed at assessing the effects of the ECHO telementoring model in the capacity building of healthcare workers in the context of COVID-19 in India.MethodWe adopted a mixed-method approach with a parallel combination design. A quantitative survey was used to measure changes in the knowledge and self-efficacy among doctors and nurses. In-depth Interviews were used for qualitative exploration of perceptions and experiences of all the study participants. Student t-test and ANOVA were used to assess significant differences between mean scores across participant characteristics for different themes. Statistical significance was set at p < 0.05. In-depth Interviews were analyzed using Framework Analysis. The evaluation followed the first five levels of Moore's model.ResultsThe results highlighted the strengthening of knowledge and skills of healthcare workers in the assessment and management of COVID-19 after the ECHO training. Learning and performance ratings were high as 96% reported an increase in knowledge and 98% were able to apply it in their clinical practices. The key challenges identified were technical issues like internet connectivity and lack of interaction due to limited visual connection. The hybrid sessions, use of video camera, feedback mechanism, and inclusion of Continuing Medical Education were recommended by participants to improve the model.ConclusionsThe findings of this study are an important addition to the pre-existing literature supporting the replicability of the ECHO model in the upskilling of healthcare professionals working in underserved and remote areas, not only in the context of COVID-19 but also in other public health domains. To enhance the effectiveness of this ECHO model, the study findings may be used to refine the model and improve the areas of concern.

Project description:IntroductionThe patient experience (PX) is central to improving the quality of healthcare services. We launched a PX capacity- and capability-building program at the Armed Forces Hospitals Taif Region, which is integral to our regional healthcare cluster transformation plans and is an initial step toward developing a culture of improvement in human experience in healthcare.MethodsA multidisciplinary PX committee recruited five frontline interprofessional PX heads, one from each of our regional healthcare hospitals. The Kirkpatrick model for program evaluation was used to assess the impact on four key levels: reaction, learning, behavior, and results. A pre-program competency assessment was conducted to evaluate the level of expertise across various PX competencies, and a program curriculum was developed accordingly. Participants underwent an intensive workshop-based PX capacity-building training program. A post-program competency assessment was performed along with a post-program survey. The PX-related activities led by interprofessional frontline PX heads at their respective hospitals' post-programs were tracked. The impact on the regional PX mean score across various settings, including inpatient, outpatient, and emergency settings, was measured using Press Ganey PX surveys. Our work is reported in accordance with the SQUIRE-EDU guidelines of the EQUATOR network.ResultsThe PX capacity-building program led to a significant improvement in participants' expertise across various PX competencies. Significant improvements beyond the strategic targets were observed in the PX mean score in inpatient departments pre-program (83.31) vs. post-program (86.34), with a p-value of < 0.001 across the regional healthcare system.ConclusionThe PX capacity-building program is a first step toward major cultural change amid the healthcare cluster transformation in our regional healthcare system. The Kirkpatrick model helps evaluate the impact of PX capacity- and capability-building training programs comprehensively through an organizational approach. Sustainable improvements in PX over a long period through a capacity-building program alone remain challenging.

Project description:The advent of Internet technologies has resulted in the proliferation of electronic trading and the use of the Internet for electronic transactions, leading to a rise in unauthorized access to sensitive user information and the depletion of resources for enterprises. As a consequence, there has been a marked increase in phishing, which is now considered one of the most common types of online theft. Phishing attacks are typically directed towards obtaining confidential information, such as login credentials for online banking platforms and sensitive systems. The primary objective of such attacks is to acquire specific personal information to either use for financial gain or commit identity theft. Recent studies have been conducted to combat phishing attacks by examining domain characteristics such as website addresses, content on websites, and combinations of both approaches for the website and its source code. However, businesses require more effective anti-phishing technologies to identify phishing URLs and safeguard their users. The present research aims to evaluate the effectiveness of eight machine learning (ML) and deep learning (DL) algorithms, including support vector machine (SVM), k-nearest neighbors (KNN), random forest (RF), Decision Tree (DT), Extreme Gradient Boosting (XGBoost), logistic regression (LR), convolutional neural network (CNN), and DL model and assess their performances in identifying phishing. This study utilizes two real datasets, Mendeley and UCI, employing performance metrics such as accuracy, precision, recall, false positive rate (FPR), and F-1 score. Notably, CNN exhibits superior accuracy, emphasizing its efficacy. Contributions include using purpose-specific datasets, meticulous feature engineering, introducing SMOTE for class imbalance, incorporating the novel CNN model, and rigorous hyperparameter tuning. The study demonstrates consistent model performance across both datasets, highlighting stability and reliability.

Project description:BackgroundAthletic training rooms have a high prevalence of bacteria, including multidrug-resistant organisms, increasing the risk for both local and systematic infections in athletes. There are limited data outlining formal protocols or standardized programs to reduce bacterial and viral burden in training rooms as a means of decreasing infection rate at the collegiate and high school levels.HypothesisAdaptation of a hygiene protocol would lead to a reduction in bacterial and viral pathogen counts in athletic training rooms.Study designCohort study.Level of evidenceLevel 3.MethodsTwo high school and 2 collegiate athletic training rooms were studied over the course of the 2017-2018 academic year. A 3-phase protocol, including introduction of disinfectant products followed by student-athlete and athletic trainer education, was implemented at the 4 schools. Multiple surfaces in the athletic training rooms were swabbed at 4 time points throughout the investigation. Bacterial and viral burden from swabs were analyzed for overall bacterial aerobic plate count (APC), bacterial adenosine triphosphate activity, influenza viral load, and multidrug-resistant organisms such as methicillin-resistant Staphylococcus aureus (MRSA) and vancomycin-resistant enterococcus (VRE).ResultsOverall bacterial load, as measured by APC, was reduced by 94.7% (95% CI, 72.6-99.0; P = 0.003) over the course of the investigation after protocol implementation. MRSA and VRE were found on 24% of surfaces prior to intervention and were reduced to 0% by the end of the study. Influenza was initially detected on 25% of surfaces, with no detection after intervention. No cases of athletic training room-acquired infections were reported during the study period.ConclusionA uniform infection control protocol was effective in reducing bacterial and viral burden, including multidrug-resistant organisms, when implemented in the athletic training rooms of 2 high schools and 2 colleges.Clinical relevanceA standardized infection control protocol can be utilized in athletic training rooms to reduce bacterial and viral burden.

Project description:There is an urgent need for generalized training in cardiopulmonary resuscitation (CPR) techniques, starting with secondary education. Validated instruments for assessing the efficacy of such interventions are not yet available. This study aimed to validate an evaluation questionnaire of a CPR training program for high school students, to analyze the levels of readability, difficulty, reliability, and content validity, as well as the fit the purpose for which they were designed, the trait they are intended to measure. An instrumental study was conducted in 2 phases. In the first phase, an inter-judge validation was carried out with 11 experts in CPR accredited instructors of basic and advanced CPR by the American Heart Association. In the second phase, the psychometric properties were evaluated from the perspective of Item Response Theory. During May of the 2017/18 and 2018/19 academic years, 259 4th-year secondary school students from a high school in the southeastern area of the island of Gran Canaria (mean age: 15.78 years; 50.60% male) were surveyed anonymously using the questionnaire to be validated. The questionnaire was easily readable (74.12 Flesch-Szigriszt Index); the difficulty level (Easy/Very Easy) in the context of this secondary school level of education and the ability level of the respondents overlapped sufficiently and there was no gender bias. The questionnaire was able to discriminate between respondents of slightly more than 7 levels of expertise, from low knowledge of CPR to high knowledge of CPR (Separation Index 7.53). The model fit was excellent (infit = 1/outfit = 1.01). The content validity index was adequate. The separation index and reliability exceeded what was considered adequate for guaranteed use. The level of difficulty of the items and the level of ability of the respondents is in line with the educational level of the students. The questionnaire did not produce a gender bias in response probability. The questionnaire is easily understandable and can discriminate between different levels of ability without differential gender bias, and its reliability is outstanding, as it exceeds the minimum criteria.

Dataset Information

Evaluation of a mandatory phishing training program for high-risk employees at a US healthcare system.

Objective

Materials and methods

Results

Discussion

Conclusions

Publications

Evaluation of a mandatory phishing training program for high-risk employees at a US healthcare system.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets