An analysis of the learning curve to achieve competency at colonoscopy using the JETS database.
ABSTRACT: OBJECTIVE:The number of colonoscopies required to reach competency is not well established. The primary aim of this study was to determine the number of colonoscopies trainees need to perform to attain competency, defined by a caecal intubation rate (CIR) ?90%. As competency depends on completion, we also investigated trainee factors that were associated with colonoscopy completion. DESIGN:The Joint Advisory Group on GI Endoscopy in the UK has developed a trainee e-portfolio from which colonoscopy data were retrieved. Inclusion criteria were all trainees who had performed a total of ?20 colonoscopies and had performed ?50 colonoscopies prior to submission of data to the e-portfolio. The primary outcome measure was colonoscopy completion. The number of colonoscopies required to achieve CIR ?90% was calculated by the moving average method and learning curve cumulative summation (LC-Cusum) analysis. To determine factors which determine colonoscopy completion, a mixed effect logistic regression model was developed which allowed for nesting of patients within trainees and nesting of patients within hospitals, with various patient, trainee and training factors entered as fixed effects. RESULTS:297 trainees undertook 36?730 colonoscopies. By moving average analysis, the cohort of trainees reached a CIR of 90% at 233 procedures. By LC-Cusum analysis, 41% of trainees were competent after 200 procedures. Of the trainee factors, the number of colonoscopies, intensity of training and previous flexible sigmoidoscopy experience were significant factors associated with colonoscopy completion. CONCLUSIONS:This is the largest study to date investigating the number of procedures required to achieve competency in colonoscopy. The current training certification benchmark in the UK of 200 procedures does not appear to be an inappropriate minimum requirement. The LC-Cusum chart provides real time feedback on individual learning curves for trainees. The association of training intensity and flexible sigmoidoscopy experience with colonoscopy completion could be exploited in training programmes.
Project description:Colonoscopy is the 'gold standard' assessment for large bowel mucosal pathology, but a complete examination is essential. The first national colonoscopy audit carried out in 1999 demonstrated caecal intubation rates (CIRs) of 56.9%. As a result, the Joint Advisory Group (JAG) on gastrointestinal endoscopy launched a programme of continuous quality improvement. JAG recommends that practitioners undertake 100+ procedures per annum with a target CIR of 90%. This current audit provides an assessment of performance against this quality standard.Data were collected from all procedures undertaken in 2008-2009 from six hospitals across three English regions.16064 colonoscopies performed: CIR = 90.57% (95% CI 90.11% to 91.01%). Operators doing 100+ procedures per annum, CIR=91.76% (91.24% to 92.25%). Operators doing <100 procedures per annum, CIR=87.77% (86.82% to 88.67%). Gastroenterologists, CIR=91.01% (90.32% to 91.70%). Surgeons, CIR=91.03% (90.27% to 91.79%). Other practitioners, CIR=81.51% (78.79% to 84.22%). Bowel cancer screening programme (BCSP) colonoscopies, CIR=97.71% (97.07% to 98.34%). Non-screening colonoscopies, CIR=88.31% (95% CI 87.68% to 88.94%).This audit of 16064 colonoscopies across three regions demonstrates aggregated achievement of the CIR quality standard. However, there is a significant performance gap when comparing BCSP colonoscopists with non-screening colonoscopists and the overall CIR of >90% is supported by the volume of BCSP colonoscopy. Endoscopists performing low volume colonoscopy (<100 per annum), have CIR of <90%. Endoscopists with low volume practice who do not meet the quality standards should engage in skills augmentation plus further training and increase volume of colonoscopy with local mentorship, or stop performing colonoscopy.
Project description:Introduction In the UK, endoscopy certification is administered by the Joint Advisory Group on Gastrointestinal Endoscopy (JAG). Since 2011, certification for upper and lower gastrointestinal endoscopy has been awarded via a national (JETS) e-portfolio to the main training specialties of: gastroenterology, gastrointestinal surgeons (GS) and non-medical endoscopists (NME). Trends in endoscopy certification and differences between trainee specialties were analyzed. Methods This prospective UK-wide observational study identified trainees awarded gastroscopy, sigmoidoscopy, colonoscopy (provisional and full) certification between June 2011 - 2017. Trends in certification, procedures and time-to-certification, and key performance indicators (KPIs) in the 3-month pre- and post-certification period were compared between the three main training specialties. Results Three thousand one hundred fifty-seven endoscopy-related certifications were awarded to 1928 trainees from gastroenterology (52.3 %), GS (28.4 %) and NME (16.5 %) specialties. During the study period, certification numbers increased for all modalities and specialties, particularly NME trainees. For gastroscopy and colonoscopy, procedures-to-certification were lowest for GS ( P < 0.001), whereas time-to-certification was consistently shortest in NMEs ( P < 0.001). A post-certification reduction in mean cecal intubation rate (95.2 % to 93.8 %, P < 0.001) was observed in colonoscopy, and D2 intubation (97.6 % to 96.2 %, P < 0.001) and J-maneuver (97.3 % to 95.8 %, P < 0.001) in gastroscopy. Overall, average pre- and post-certification KPIs still exceeded national minimum standards. There was an increase in PDR for NMEs after provisional colonoscopy certification but a decrease in PDR for GS trainees after sigmoidoscopy and full colonoscopy certification. Conclusion Despite variations among trainee specialties, average pre- and post-certification KPIs for certified trainees met national standards, suggesting that JAG certification is a transparent benchmark which adequately safeguards competency in endoscopy training.
Project description:OBJECTIVES:To determine the operative experience of UK general surgery trainees and assess the changing procedural supervision and acquisition of competency assessments through the course of training. BACKGROUND:Competency assessment is changing with concepts of trainee autonomy decisions (termed entrustment decisions) being introduced to surgical training. METHODS:Data from the Intercollegiate Surgical Curriculum Programme and the eLogbook databases for all UK General Surgery trainees registered from August 1, 2007 who had completed training were used. Total and index procedures (IP) were counted and variation by year of training assessed. Recorded supervision codes and competency assessment outcomes for IPs were assessed by year of training. RESULTS:We identified 311 trainees with complete data. Appendicectomy was the most frequently undertaken IP during first year of training [mean procedures (mp) = 26] and emergency laparotomy during final year of training (mp = 27). The proportion of all IPs recorded as unsupervised increased through training (P < 0.05) and varied between IPs with 91.2% of appendicectomies (mp = 20), 40.6% of emergency laparotomies (mp = 27), and 17.4% of segmental colectomies (mp = 15) recorded as unsupervised during the final year of training. Acquisition of competency assessments increased through training and varied by IP. CONCLUSIONS:The changing autonomy of trainees through the course of an entire training scheme, alongside formal competency assessments, may provide evidence of changing entrustment decisions made by trainers for different key procedures. Other countries utilizing electronic logbooks could adopt similar techniques to further understanding of competency attainment amongst their surgical trainees.
Project description:The measurement of the quality of colonoscopy has been in the vanguard of quality improvement. The Joint Advisory Group on Gastrointestinal endoscopy (JAG) has issued guidance for practitioners to achieve caecal intubation rates (CIR) of ≥90% and to undertake ≥100 colonoscopies per annum. The British Society of Gastroenterology National Colonoscopy Audit published in 2012-2013 demonstrated a combined CIR of 92.3%. In 2012, we published data from 16 064 colonoscopies showing a combined CIR of 90.57%-both meeting JAG's standard. Analysis of our audit looked at the relationship of volume and outcome. CIR of operators performing ≥100 procedures per annum was 91.76%; those performing <100 was 87.77%. The 2-year data we collected involved 120+ operators. This provided an opportunity to study the correlation between volume and CIR in detail.We analysed 129 operator records who had undertaken 20-399 procedures per annum (total 12 594). Each operator's volume was plotted against CIR as individuals and groups of operators undertaking a similar annual volume. 9859 procedures (78.3%) were performed by operators undertaking 20-199 procedures per annum (120 operators); this subgroup was further analysed.When plotting individuals and groups of individuals who have undertaken a similar annual volume against CIR, the trend-lines cross a 90% CIR at a volume of 120-125 procedures. The subgroup analysis showed the trend-line crossing at 110-120 procedures.This detailed analysis of 12 594 colonoscopies over 2 years suggests that JAG should advise operators to undertake ≥120 procedures per annum to support the quality standard for CIR of ≥90%.
Project description:The aim of this study was to evaluate UK trainee experience in endoscopy for acute upper gastrointestinal bleeding (AUGIB).Data was prospectively collected from all patients presenting to South Yorkshire Hospitals with AUGIB from September 2011 to December 2011 and compared with data from 1996. Concurrently, all gastroenterology trainees registered with the British Society of Gastroenterology were invited to respond to a web-based questionnaire regarding their experience in AUGIB management.77% (589/766) of the patient cohort underwent endoscopy for AUGIB; 15% (90/589) were performed by trainees. 7.2% (9/125) of the out of hours endoscopy case load was performed by trainees; all were low-risk or medium-risk cases (pre-endoscopy Rockall score ≤4). During the study period, dual therapy was delivered by a trainee on only four occasions. Comparison with the 1996 cohort demonstrated a marked reduction in the number of trainee performed endoscopies (76% vs 15%; p<0.001). Questionnaires were returned by 51% (245/478) of British Society of Gastroenterology trainees. 81% (198/245) thought that <10% of the gastroscopies they had performed involved therapeutic intervention. 23% (57/245) felt they would not be competent in AUGIB endoscopy by completion of specialty training.This study demonstrates the decline over time in trainee experience in AUGIB endoscopy. It also highlights a lack of trainee exposure to more challenging cases, out of hours endoscopy and therapeutic procedures. Furthermore, trainees are concerned that a level of competency may not be attained during specialty training. We advocate reviewing UK endoscopic training provision for AUGIB to ensure that experienced endoscopists are produced to meet future service needs.
Project description:BACKGROUND:Validated competency assessment tools and the data supporting milestone development during gastroscopy training are lacking. We aimed to assess the validity of the formative direct observation of procedural skills (DOPS) assessment tool in diagnostic gastroscopy and study competency development using DOPS. METHODS:This was a prospective multicentre (N?=?275) analysis of formative gastroscopy DOPS assessments. Internal structure validity was tested using exploratory factor analysis and reliability estimated using generalisability theory. Item and global DOPS scores were stratified by lifetime procedure count to define learning curves, using a threshold determined from receiver operator characteristics (ROC) analysis. Multivariable binary logistic regression analysis was performed to identify independent predictors of DOPS competence. RESULTS:In total, 10086 DOPS were submitted for 987 trainees. Exploratory factor analysis identified three distinct item groupings, representing 'pre-procedure', 'technical', and 'post-procedure non-technical' skills. From generalisability analyses, sources of variance in overall DOPS scores included trainee ability (31%), assessor stringency (8%), assessor subjectivity (18%), and trainee case-to-case variation (43%). The combination of three assessments from three assessors was sufficient to achieve the reliability threshold of 0.70. On ROC analysis, a mean score of 3.9 provided optimal sensitivity and specificity for determining competency. This threshold was attained in the order of 'pre-procedure' (100-124 procedures), 'technical' (150-174 procedures), 'post-procedure non-technical' skills (200-224 procedures), and global competency (225-249 procedures). Higher lifetime procedure count, DOPS count, surgical trainees and assessors, higher trainee seniority, and lower case difficulty were significant multivariable predictors of DOPS competence. CONCLUSION:This study establishes milestones for competency acquisition during gastroscopy training and provides validity and reliability evidence to support gastroscopy DOPS as a competency assessment tool.
Project description:The detection of adenomatous lesions is a major indicator for quality and competence in colonoscopy. Little is known about adenoma detection rates (ADR) of endoscopy trainees. The aim of our study was to investigate the performance of trainee colonoscopists in detecting adenomas and to depict the shape of adenoma detection learning curves during apprenticeship.We retrospectively investigated a prospectively maintained database of a single tertiary referral center to reveal colonoscopies performed by trainee endoscopists during 2001 and 2013. Colonoscopy reports were chronologically retrieved and separately analyzed for each trainee. Using cumulative curves, courses of trainee's Adenoma detection rates (ADR) during apprenticeship were displayed. Additionally, procedural data including cecal intubation rate and occurrence of complications were assessed.We retrospectively analyzed 4354 colonoscopies conducted by 10 trainee endoscopists (TE). A median number of 371 investigations were performed by each apprentice. Group ADR was 23%. No significant difference between aggregated ADRs at the beginning (23%) and at the end (22%) of apprenticeship could be determined (p?=?0.70). However, individual learning curves showed considerable different slopes. Personal ADR values ranged between 17% and 31%. Overall cecum intubation rate was 99.0 %. Complication rates were low and fulfilled quality requirements recommended in guidelines.From the beginning of education, trainee colonoscopists are capable to provide high-quality investigations considering the detection of adenomas as a benchmark quality indicator. Nevertheless, performance differs markedly between investigators. Therefore, individual detection rates should be reviewed regularly to reveal further need for training.
Project description:Background:Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) has a high diagnostic yield and low complication rate. Whilst it has been included in international guidelines for the diagnosis and staging of lung cancer, current results are mostly based on EBUS experts performing EBUS-TBNA in centres of excellence. The impact of simulation training on diagnostic yield, complications, scope damage and repair cost in a real-world teaching hospital is unclear. Methods:A review of our hospital EBUS-TBNA registry from August 2008 to December 2016 was performed. A positive diagnosis was defined as a confirmed histological or microbiological diagnosis based on EBUS sampling. Complications were classified as major or minor according to the British Thoracic Society guidelines. In addition, we assessed the cost of repairs for scope damage before and after simulation training was implemented. Using CUSUM analysis, the learning curves of individual trainees and the institution were plotted. Results:There were 608 EBUS-TBNA procedures included in the study. The number of procedures performed by trainees who underwent conventional training was 331 and those who underwent simulation training performed 277 procedures. Diagnostic yield for trainees without simulation training was 88.2% vs. 84.5% for trainees with simulation training (P=0.179). There was no statistical difference in the diagnostic yield between the groups of trainees (OR: 0.781, 95% CI: 0.418-1.460, P=0.438) after adjusting for risk factors. There was an increase in overall complications from 13.6% to 16.6% (OR: 2.247, 95% CI: 1.297-3.891, P=0.004) after introduction of the simulation training, but a trend to decrease in major complications 3.6% to 0.7% (P=0.112). The cost for scope repairs for the trainees without simulation training was SGD 413.88 per procedure vs. SGD 182.79 per procedure for the trainees with simulation training, with the mean difference being SGD 231.09 per procedure (95% CI: 178.40-640.60, P=0.268). CUSUM analysis showed an increasing learning curve for the trainees with simulation training after an initial competency period. Conclusions:There was no statistical difference in diagnostic yield from EBUS-TBNA and cost of scope damage after simulation training was introduced into our training program. Interestingly, there was an increase in minor complications. CUSUM analysis can provide additional information on institutional learning curves. The value of simulation training in EBUS-TBNA remains uncertain.
Project description:BACKGROUND & AIMS:Studies have reported substantial variation in the competency of advanced endoscopy trainees, indicating a need for more supervised training in endoscopic ultrasound (EUS). We used a standardized, validated, data collection tool to evaluate learning curves and measure competency in EUS among trainees at multiple centers. METHODS:In a prospective study performed at 15 centers, 17 trainees with no prior EUS experience were evaluated by experienced attending endosonographers at the 25th and then every 10th upper EUS examination, over a 12-month training period. A standardized data collection form was used (using a 5-point scoring system) to grade the EUS examination. Cumulative sum analysis was applied to produce a learning curve for each trainee; it tracked the overall performance based on median scores at different stations and also at each station. Competency was defined by a median score of 1, with acceptable and unacceptable failure rates of 10% and 20%, respectively. RESULTS:Twelve trainees were included in the final analysis. Each of the trainees performed 265 to 540 EUS examinations (total, 4257 examinations). There was a large amount of variation in their learning curves: 2 trainees crossed the threshold for acceptable performance (at cases 225 and 245), 2 trainees had a trend toward acceptable performance (after 289 and 355 cases) but required continued observation, and 8 trainees needed additional training and observation. Similar results were observed at individual stations. CONCLUSIONS:A specific case load does not ensure competency in EUS; 225 cases should be considered the minimum caseload for training because we found that no trainee achieved competency before this point. Ongoing training should be provided for trainees until competency is confirmed using objective measures.
Project description:BACKGROUND AND AIMS:Although coronavirus disease 2019 (COVID-19) has affected endoscopy services globally, the impact on trainees has not been evaluated. We aimed to assess the impact of COVID-19 on procedural volumes and on the emotional well-being of endoscopy trainees worldwide. METHODS:An international survey was disseminated over a 3-week period in April 2020. The primary outcome was the percentage reduction in monthly procedure volume before and during COVID-19. Secondary outcomes included potential variation of COVID-19 impact between different continents and rates and predictors of anxiety and burnout among trainees. RESULTS:Across 770 trainees from 63 countries, 93.8% reported a reduction in endoscopy case volume. The median percentage reduction in total procedures was 99% (interquartile range, 85%-100%), which varied internationally (P < .001) and was greatest for colonoscopy procedures. Restrictions in case volume and trainee activity were common barriers. A total of 71.9% were concerned that the COVID-19 pandemic could prolonged training. Anxiety was reported in 52.4% of respondents and burnout in 18.8%. Anxiety was independently associated with female gender (odds ratio [OR], 2.15; P < .001), adequacy of personal protective equipment (OR, 1.75; P = .005), lack of institutional support for emotional health (OR, 1.67; P = .008), and concerns regarding prolongation of training (OR, 1.60; P = .013). Modifying existing national guidelines to support adequate endoscopy training during the pandemic was supported by 68.9%. CONCLUSIONS:The COVID-19 pandemic has led to restrictions in endoscopic volumes and endoscopy training, with high rates of anxiety and burnout among endoscopy trainees worldwide. Targeted measures by training programs to address these key issues are warranted to improve trainee well-being and support trainee education.