Utility of eye-tracking technology for preparing medical students in Spain for the summative objective structured clinical examination.
Ontology highlight
ABSTRACT:
SUBMITTER: Sanchez-Ferrer F
PROVIDER: S-EPMC5729213 | biostudies-literature | 2017
REPOSITORIES: biostudies-literature
Publications
Utility of eye-tracking technology for preparing medical students in Spain for the summative objective structured clinical examination.
Sánchez-Ferrer Francisco F Ramos-Rincón J M JM Grima-Murcia M D MD Luisa Sánchez-Ferrer María M Sánchez-Del Campo Francisco F Compañ-Rosique Antonio F AF Fernández-Jover Eduardo E
Journal of educational evaluation for health professions 20171112
Similar Datasets
Project description:BackgroundThe objective structured clinical examination (OSCE) is a test used throughout Spain to evaluate the clinical competencies, decision making, problem solving, and other skills of sixth-year medical students.ObjectiveThe main goal of this study is to explore the possible applications and utility of portable eye-tracking systems in the setting of the OSCE, particularly questions associated with attention and engagement.MethodsWe used a portable Tobii Glasses 2 eye tracker, which allows real-time monitoring of where the students were looking and records the voice and ambient sounds. We then performed a qualitative and a quantitative analysis of the fields of vision and gaze points attracting attention as well as the visual itinerary.ResultsEye-tracking technology was used in the OSCE with no major issues. This portable system was of the greatest value in the patient simulators and mannequin stations, where interaction with the simulated patient or areas of interest in the mannequin can be quantified. This technology proved useful to better identify the areas of interest in the medical images provided.ConclusionsPortable eye trackers offer the opportunity to improve the objective evaluation of candidates and the self-evaluation of the stations used as well as medical simulations by examiners. We suggest that this technology has enough resolution to identify where a student is looking at and could be useful for developing new approaches for evaluating specific aspects of clinical competencies.
| S-EPMC7500558 | biostudies-literature
Project description:Background/purposeObjective structured clinical examination (OSCE) has found increasingly wide use in the education of clinical dentistry in recent years. The objectives of this study were to find the predictability of OSCE in representing a student's actual ability in a clinical environment and to provide instructors with better ideas to guide an individual student in a certain field.Materials and methodsThirty-one dentistry students' qualification OSCE scores were obtained after the conclusion of their dental clerkship in December 2014. The OSCE scores were compared with the various scores given by instructors from different departments during their internship a year later. The correlation was investigated between these two scores.ResultsThe scores of the first station of OSCE-explaining a treatment plan for the restoration of a missing tooth-were correlated with the scores of prosthodontics in the internship. The scores of the fourth station of OSCE-explaining to a mother regarding the obtainment of a radiograph of her child's tooth-were correlated with the scores of orthodontics. In addition, the total score of the six OSCE stations was correlated with the scores of orthodontics, oral surgery, and pediatric dentistry during dental internship.ConclusionThe results of clerkship qualification OSCE could provide instructors with insights into the needs of the students before their entering internship.
| S-EPMC6388860 | biostudies-literature
Project description:ObjectiveTo investigate the influence of providing online procedural videos to postgraduate chiropractic students preparing for an objective structured clinical examination (OSCE).MethodsEighty-three postgraduate chiropractic students enrolled in a diagnostic unit during 2017 received supplemental video resources prior to their final OSCE. Ninety students enrolled in the 2016 offering of the unit acted as the control group. Two-sample t tests were used to compare OSCE results between groups and paired t tests were used for within-group comparisons. Regression analysis was used to examine the association of age, undergraduate grade point average, and gender with the final OSCE scores. Students were also surveyed regarding their perceptions of the video resources using a purpose-built questionnaire.ResultsA paired t test comparing initial and final OSCE scores found a small but significant increase in scores for the 2017 (mean change 3.6 points; p = .001) but not the 2016 (mean change -1.1 scores; p = .09) cohort. The 2017 cohort had significantly more change than the 2016 cohort (mean difference 4.7 points; p < .001). Analysis of responses to the questionnaire highlighted overall positive feedback for the procedural videos.ConclusionOnline procedural videos as learning resources had a small but positive effect on OSCE performance for a group of postgraduate chiropractic students. Students perceived the resource as being helpful for OSCE preparation.
| S-EPMC7682643 | biostudies-literature
Project description:ObjectivesThis study examines factors affecting reliability, or consistency of assessment scores, from an objective structured clinical examination (OSCE) in neurology through generalizability theory (G theory).MethodsData include assessments from a multistation OSCE taken by 194 medical students at the completion of a neurology clerkship. Facets evaluated in this study include cases, domains, and items. Domains refer to areas of skill (or constructs) that the OSCE measures. G theory is used to estimate variance components associated with each facet, derive reliability, and project the number of cases required to obtain a reliable (consistent, precise) score.ResultsReliability using G theory is moderate (Φ coefficient = 0.61, G coefficient = 0.64). Performance is similar across cases but differs by the particular domain, such that the majority of variance is attributed to the domain. Projections in reliability estimates reveal that students need to participate in 3 OSCE cases in order to increase reliability beyond the 0.70 threshold.ConclusionsThis novel use of G theory in evaluating an OSCE in neurology provides meaningful measurement characteristics of the assessment. Differing from prior work in other medical specialties, the cases students were randomly assigned did not influence their OSCE score; rather, scores varied in expected fashion by domain assessed.
| S-EPMC4642144 | biostudies-literature
Project description:BackgroundObjective structured clinical examination (OSCE) has been used in evaluating clinical competence in health professions education around the world. Despite its implementation in Iraq for around a decade, limited studies investigated the challenges and opportunities to improve the standard and quality of this examination from student's perspective.MethodsThis qualitative study was based on an online open-ended questionnaire survey that was carried out in the College of Medicine, Hawler Medical University, Iraq at the beginning of the 2018-2019 academic year. A convenience sample of 180 students in the clinical phase (4th, 5th, and 6th) year of study were invited to participate.ResultsA total of 141 students responded to the online questionnaire. The participants were generally happy with the OSCE, and they recognized many positive aspects, including the role of the OSCE in increasing confidence, engagement and motivating learning, the role of the OSCE in achieving a higher level of learning, the content validity of the OSCE, and the quality of the OSCE. The main weak points of the OSCE identified by the students included unfairness, gender discrimination, duration of the OSCE, and the behavior of the examiners. Suggestions to improve the OSCE examination included improving the examiners' behavior, with the focus on the training of the examiners, and avoiding discrimination among students.ConclusionsMost of the students were generally satisfied with the current OSCE examination. The main concern of the students was related to the organization of the OSCE. Valuable suggestions were raised to improve the OSCE quality including examiners' and simulated patients' training.
| S-EPMC9439190 | biostudies-literature
Project description:ObjectivesWe evaluated the reliability and predictive ability of an objective structured clinical examination (OSCE) in the assessment of medical students at the completion of a neurology clerkship.MethodsWe analyzed data from 195 third-year medical students who took the OSCE. For each student, the OSCE consisted of 2 standardized patient encounters. The scores obtained from each encounter were compared. Faculty clinical evaluations of each student for 2 clinical inpatient rotations were also compared. Hierarchical regression analysis was applied to test the ability of the averaged OSCE scores to predict standardized written examination scores and composite clinical scores.ResultsStudents' OSCE scores from the 2 standardized patient encounters were significantly correlated with each other (r = 0.347, p < 0.001), and the scores for all students were normally distributed. In contrast, students' faculty clinical evaluation scores from 2 different clinical inpatient rotations were uncorrelated, and scores were skewed toward the highest ratings. After accounting for clerkship order, better OSCE scores were predictive of better National Board of Medical Examiners standardized examination scores (R(2)Δ = 0.131, p < 0.001) and of better faculty clinical scores (R(2)Δ = 0.078, p < 0.001).ConclusionsStudent assessment by an OSCE provides a reliable and predictive objective assessment of clinical performance in a neurology clerkship.
| S-EPMC3414663 | biostudies-literature
Project description:Introduction: The aim of the Objective Structured Clinical Examination (OSCE) is a standardized and fair assessment of clinical skills. Observing second clinical year medical students during a summative OSCE assessing a General Practice clerkship, we noticed that information exchange with peers led to a progressively faster and overly focused management of simulations. Therefore, we established a Multiple Scenario-OSCE (MS-OSCE) where all students had to manage the same chief complaint at a station but it's underlying scenarios being randomly changed during students' rotation through their parcours. We wanted to ensure they fully explore differential diagnosis instead of managing their task influenced by shared information. We wanted to assess if a MS-OSCE violates the assumption of objectivity and fairness given that students are not tested with the same scenarios. Methods: We developed and piloted five OSCE stations (chest pain, abdominal pain, back pain, fatigue and acute cough) with two or three different underlying scenarios each. At each station these scenarios randomly changed from student to student. Performance was assessed with a checklist and global rating. The effect of scenarios and raters on students' grades was assessed calculating the intraclass correlation coefficient with a fixed effect two level linear model. Results: A total of 169 students and 23 raters participated in the MS-OSCE. The internal consistency over all stations was 0.65 by Cronbach's alpha. The difference of the mean grades between the scenarios of a given chief complaint ranged from 0.03 to 0.4 on a 1 to 5 grading scale. The effect of scenarios on the variance of the final grades at each station ranged from 4% to 9% and of raters from 20% to 50% when adjusted for students' skills. Conclusions: The effect of different scenarios on the grades was relevant but small compared to the effect of raters on grades. Improving rater training is more important to ensure objectivity and fairness of MS-OSCE than providing the same scenario to all students.
| S-EPMC6545613 | biostudies-literature
Project description:BACKGROUND:Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students' performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. OBJECTIVE:This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students' radiology knowledge and the reproducibility of this assessment tool. METHODS:We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students' clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. RESULTS:A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (-0.02; P=.48), adjusting for the medical school year. CONCLUSIONS:Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested.
| S-EPMC7240440 | biostudies-literature
Project description:BackgroundIt is not known if construct-related validity (progression of scores with different levels of training) and generalizability of Objective Structured Clinical Examination (OSCE) scenarios previously used with non-US graduating anesthesiology residents translate to a US training program.ObjectiveWe assessed for progression of scores with training for a validated high-stakes simulation-based anesthesiology examination.MethodsFifty US anesthesiology residents in postgraduate years (PGYs) 2 to 4 were evaluated in operating room, trauma, and resuscitation scenarios developed for and used in a high-stakes Israeli Anesthesiology Board examination, requiring a score of 70% on the checklist for passing (including all critical items).ResultsThe OSCE error rate was lower for PGY-4 than PGY-2 residents in each field, and for most scenarios within each field. The critical item error rate was significantly lower for PGY-4 than PGY-3 residents in operating room scenarios, and for PGY-4 than PGY-2 residents in resuscitation scenarios. The final pass rate was significantly higher for PGY-3 and PGY-4 than PGY-2 residents in operating room scenarios, and also was significantly higher for PGY-4 than PGY-2 residents overall. PGY-4 residents had a better error rate, total scenarios score, general evaluation score, critical items error rate, and final pass rate than PGY-2 residents.ConclusionsThe comparable error rates, performance grades, and pass rates for US PGY-4 and non-US (Israeli) graduating (PGY-4 equivalent) residents, and the progression of scores among US residents with training level, demonstrate the construct-related validity and generalizability of these high-stakes OSCE scenarios.
| S-EPMC4535213 | biostudies-literature
Project description:PurposeIn a sequential objective structured clinical examination (OSCE), all students initially take a short screening OSCE. Examinees who pass are excused from further testing, but an additional OSCE is administered to the remaining examinees. Previous investigations of sequential OSCE were based on classical test theory. We aimed to design and evaluate screening OSCEs based on item response theory (IRT).MethodsWe carried out a retrospective observational study. At each station of a 10-station OSCE, the students' performance was graded on a Likert-type scale. Since the data were polytomous, the difficulty parameters, discrimination parameters, and students' ability were calculated using a graded response model. To design several screening OSCEs, we identified the 5 most difficult stations and the 5 most discriminative ones. For each test, 5, 4, or 3 stations were selected. Normal and stringent cut-scores were defined for each test. We compared the results of each of the 12 screening OSCEs to the main OSCE and calculated the positive and negative predictive values (PPV and NPV), as well as the exam cost.ResultsA total of 253 students (95.1%) passed the main OSCE, while 72.6% to 94.4% of examinees passed the screening tests. The PPV values ranged from 0.98 to 1.00, and the NPV values ranged from 0.18 to 0.59. Two tests effectively predicted the results of the main exam, resulting in financial savings of 34% to 40%.ConclusionIf stations with the highest IRT-based discrimination values and stringent cut-scores are utilized in the screening test, sequential OSCE can be an efficient and convenient way to conduct an OSCE.
| S-EPMC5676019 | biostudies-literature