Dataset Information

Simulation-based assessment of robotic cardiac surgery skills: An international multicenter, cross-specialty trial.

ABSTRACT:

Objective

This study aimed to investigate the validity of simulation-based assessment of robotic-assisted cardiac surgery skills using a wet lab model, focusing on the use of a time-based score (TBS) and modified Global Evaluative Assessment of Robotic Skills (mGEARS) score.

Methods

We tested 3 wet lab tasks (atrial closure, mitral annular stitches, and internal thoracic artery [ITA] dissection) with both experienced robotic cardiac surgeons and novices from multiple European centers. The tasks were assessed using 2 tools: TBS and mGEARS score. Reliability, internal consistency, and the ability to discriminate between different levels of competence were evaluated.

Results

The results demonstrated a high internal consistency for all 3 tasks using mGEARS assessment tool. The mGEARS score and TBS could reliably discriminate between different levels of competence for the atrial closure and mitral stitches tasks but not for the ITA harvesting task. A generalizability study also revealed that it was feasible to assess competency of the atrial closure and mitral stitches tasks using mGEARS but not the ITA dissection task. Pass/fail scores were established for each task using both TBS and mGEARS assessment tools.

Conclusions

The study provides sufficient evidence for using TBS and mGEARS scores in evaluating robotic-assisted cardiac surgery skills in wet lab settings for intracardiac tasks. Combining both assessment tools enhances the evaluation of proficiency in robotic cardiac surgery, paving the way for standardized, evidence-based preclinical training and credentialing.

Clinical trial registry number

NCT05043064.

SUBMITTER: Atroshchenko GV

PROVIDER: S-EPMC10775167 | biostudies-literature | 2023 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Simulation-based assessment of robotic cardiac surgery skills: An international multicenter, cross-specialty trial.

Atroshchenko Gennady V GV Navarra Emiliano E Valdis Matthew M Sandoval Elena E Hashemi Nasseh N Cerny Stepan S Pereda Daniel D Palmen Meindert M Bjerrum Flemming F Bruun Niels Henrik NH Tolsgaard Martin G MG

JTCVS open 20231102

<h4>Objective</h4>This study aimed to investigate the validity of simulation-based assessment of robotic-assisted cardiac surgery skills using a wet lab model, focusing on the use of a time-based score (TBS) and modified Global Evaluative Assessment of Robotic Skills (mGEARS) score.<h4>Methods</h4>We tested 3 wet lab tasks (atrial closure, mitral annular stitches, and internal thoracic artery [ITA] dissection) with both experienced robotic cardiac surgeons and novices from multiple European cent ...[more]

PMID: 38204726

Similar Datasets

Project description:IntroductionA surgeon's technical skills are an important factor in delivering optimal patient care. Most existing methods to estimate technical skills remain subjective and resource intensive. Robotic-assisted surgery (RAS) provides a unique opportunity to develop objective metrics using key elements of intraoperative surgeon behavior which can be captured unobtrusively, such as instrument positions and button presses. Recent studies have shown that objective metrics based on these data (referred to as objective performance indicators [OPIs]) correlate to select clinical outcomes during robotic-assisted radical prostatectomy. However, the current OPIs remain difficult to interpret directly and, therefore, to use within structured feedback to improve surgical efficiencies.MethodsWe analyzed kinematic and event data from da Vinci surgical systems (Intuitive Surgical, Inc., Sunnyvale, CA, USA) to calculate values that can summarize the use of robotic instruments, referred to as OPIs. These indicators were mapped to broader technical skill categories of established training protocols. A data-driven approach was then applied to further sub-select OPIs that distinguish skill for each technical skill category within each training task. This subset of OPIs was used to build a set of logistic regression classifiers that predict the probability of expertise in that skill to identify targeted improvement and practice. The final, proposed feedback using OPIs was based on the coefficients of the logistic regression model to highlight specific actions that can be taken to improve.ResultsWe determine that for the majority of skills, only a small subset of OPIs (2-10) are required to achieve the highest model accuracies (80-95%) for estimating technical skills within clinical-like tasks on a porcine model. The majority of the skill models have similar accuracy as models predicting overall expertise for a task (80-98%). Skill models can divide a prediction into interpretable categories for simpler, targeted feedback.ConclusionWe define and validate a methodology to create interpretable metrics for key technical skills during clinical-like tasks when performing RAS. Using this framework for evaluating technical skills, we believe that surgical trainees can better understand both what can be improved and how to improve.

Project description:IntroductionAlthough robotic-assisted surgery is increasingly performed, objective assessment of technical skills is lacking. The aim of this study is to provide validity evidence for objective assessment of technical skills for robotic-assisted surgery.MethodsAn international multicenter study was conducted with participants from the academic hospitals Heidelberg University Hospital (Germany, Heidelberg) and the Amsterdam University Medical Centers (The Netherlands, Amsterdam). Trainees with distinctly different levels of robotic surgery experience were divided into three groups (novice, intermediate, expert) and enrolled in a training curriculum. Each trainee performed six trials of a standardized suturing task using the da Vinci Surgical System. Using the ForceSense system, five force-based parameters were analyzed, for objective assessment of tissue handling skills. Mann-Whitney U test and linear regression were used to analyze performance differences and the Wilcoxon signed-rank test to analyze skills progression.ResultsA total of 360 trials, performed by 60 participants, were analyzed. Significant differences between the novices, intermediates and experts were observed regarding the total completion time (41 s vs 29 s vs 22 s p = 0.003), mean non zero force (29 N vs 33 N vs 19 N p = 0.032), maximum impulse (40 Ns vs 31 Ns vs 20 Ns p = 0.001) and force volume (38 N3 vs 32 N3 vs 22 N3 p = 0.018). Furthermore, the experts showed better results in mean non-zero force (22 N vs 13 N p = 0.015), maximum impulse (24 Ns vs 17 Ns p = 0.043) and force volume (25 N3 vs 16 N3 p = 0.025) compared to the intermediates (p ≤ 0.05). Lastly, learning curve improvement was observed for the total task completion time, mean non-zero force, maximum impulse and force volume (p ≤ 0.05).ConclusionConstruct validity for force-based assessment of tissue handling skills in robot-assisted surgery is established. It is advised to incorporate objective assessment and feedback in robot-assisted surgery training programs to determine technical proficiency and, potentially, to prevent tissue trauma.

Project description:BackgroundWhile bedside assistants play a critical role in many robotic operations, substantial heterogeneity remains in bedside assistant training pathways. As such, this study aimed to develop consensus guidelines for bedside assistant skills required for team members in robotic operations.MethodsWe designed a study using the Delphi process to develop consensus guidelines around bedside assistant skills. We generated an initial list of bedside assistant skills from the literature, training materials, and expert input. We selected experts for the Delphi process based on prior scholarship in the area of robotic bedside assistant education and experience facilitating robotic bedside assistant training. For each item, respondents specified which robotic team members should have the skill from a list of "basic" bedside assistants, "advanced" bedside assistants, surgeons, surgical technologists, and circulating nurses. We conducted two rounds of the Delphi process and defined 80% agreement as sufficient for consensus.ResultsFourteen experts participated in two rounds of the Delphi process. By the end of the second round, the group had reached consensus on 253 of 305 items (83%). The group determined that "basic" bedside assistants should have 52 skills and that "advanced" bedside assistants should have 60 skills. The group also determined that surgeons should have 54 skills, surgical technologists should have 25 skills, and circulating nurses should have 17 skills. Experts agreed that all participants should have certain communication skills and basic knowledge of aspects of the robotic system.ConclusionsWe developed consensus guidelines on the skills required during robotic surgery by bedside assistants and other team members using the Delphi process. These findings can be used to design training around bedside assistant skills and assess team members to ensure that each team member has the appropriate skills. Hospitals can also use these guidelines to standardize expectations for robotic team members.

Dataset Information

Simulation-based assessment of robotic cardiac surgery skills: An international multicenter, cross-specialty trial.

Objective

Methods

Results

Conclusions

Clinical trial registry number

Publications

Simulation-based assessment of robotic cardiac surgery skills: An international multicenter, cross-specialty trial.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets