Dataset Information

Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool.

ABSTRACT:

Background

The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.

Methods

Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups' method was used to set a pass/fail standard.

Results

Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach's alpha =0.94) and high inter-rater reliability (Cronbach's alpha =0.91). The total item score significantly correlated with the global score (r_s=0.86, P<0.001). Participants' first performance correlated to second performance (test-retest reliability) with a Pearson's r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points.

Conclusions

The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.

SUBMITTER: Nayahangan LJ

PROVIDER: S-EPMC8339737 | biostudies-literature | 2021 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool.

Nayahangan Leizl Joy LJ Svendsen Morten Bo Søndergaard MBS Bodtger Uffe U Rahman Najib N Maskell Nick N Sidhu Jatinder Singh JS Lawaetz Jonathan J Clementsen Paul Frost PF Konge Lars L

Journal of thoracic disease 20210701 7

<h4>Background</h4>The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.<h4>Methods</h4>Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a ne ...[more]

PMID: 34422330

Similar Datasets

Project description:PURPOSE:Develop and establish the content validity of the Behavioral Assessment Screening Tool (BASTβ), a self-reported measure of behavioral and emotional symptoms after traumatic brain injury. METHODS:This was an assessment development study, including two focus groups of individuals with traumatic brain injury (n = 11) and their family members (n = 10) and an expert panel evaluation of content validity by experts in traumatic brain injury rehabilitation (n = 7). We developed and assessed the Content Validity Index of the BASTβ. RESULTS:The BASTβ initial items (n = 77) corresponded with an established conceptual model of behavioral dysregulation after traumatic brain injury. After expert panel evaluation and focus group feedback, the final BASTβ included 66 items (60 primary, 6 branching logic) rated on a three-level ordinal scale (Never, Sometimes, Always) with reference to the past two weeks, and an Environmental Context checklist including recent major life events (n = 23) and four open-ended questions about environmental factors. The BASTβ had a high Content Validity Index of 89.3%. CONCLUSION:The BASTβ is a theoretically grounded, multidimensional self-reported assessment of behavioral dysregulation after traumatic brain injury, with good content validity. Future translation into mobile health modalities could improve effectiveness and efficiency of long-term symptom monitoring post-traumatic brain injury. Future work will establish and validate the factor structure, internal consistency reliabilities and other validities of the BAST. Implications for Rehabilitation Behavioral problems after traumatic brain injury is one of the strongest contributing factors to poor mood and community integration outcomes after injury. Behavior is complex and multidimensional, making it a challenge to measure and to monitor long term. The Behavioral Assessment Screening Tool (BAST) is a patient-oriented outcome assessment developed in collaboration with individuals with traumatic brain injury, their care partners, and experts in the field of traumatic brain injury rehabilitation to be relevant and accessible for adults with traumatic brain injuries. The BAST is a long-term monitoring and screening tool for community-dwelling adults with traumatic brain injuries, to improve identification and management of behavioral and emotional sequelae.

Project description:BackgroundMentorship plays an essential role in enhancing the success of junior faculty. Previous evaluation tools focused on specific types of mentors or mentees. The main objective was to develop and provide validity evidence for a Mentor Evaluation Tool (MET) to assess the effectiveness of one-on-one mentoring for faculty in the academic health sciences.MethodsEvidence was collected for the validity domains of content, internal structure and relationship to other variables. The 13 item MET was tested for internal structure evidence with 185 junior faculty from Schools of Dentistry, Medicine, Nursing, and Pharmacy. Finally, the MET was studied for additional validity evidence by prospectively enrolling mentees of three different groups of faculty (faculty nominated for, or winners of, a lifetime achievement in mentoring award; faculty graduates of a mentor training program; and faculty mentors not in either of the other two groups) at the University of California San Francisco (UCSF) and asking them to rate their mentors using the MET. Mentors and mentees were clinicians, educators and/or researchers.ResultsThe 13 MET items mapped well to the five mentoring domains and six competencies described in the literature. The standardized Cronbach's coefficient alpha was 0.96. Confirmatory factor analysis supported a single factor (CFI = 0.89, SRMR = 0.05). The three mentor groups did not differ in the single overall assessment item (P = 0.054) or mean MET score (P = 0.288), before or after adjusting for years of mentoring. The mentorship score means were relatively high for all three groups.ConclusionsThe Mentor Evaluation Tool demonstrates evidence of validity for research, clinical, educational or career mentors in academic health science careers. However, MET did not distinguish individuals nominated as outstanding mentors from other mentors. MET validity evidence can be studied further with mentor-mentee pairs and to follow prospectively the rating of mentors before and after a mentorship training program.

Dataset Information

Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool.

Background

Methods

Results

Conclusions

Publications

Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets