Dataset Information

Endoscope Automation Framework with Hierarchical Control and Interactive Perception for Multi-Tool Tracking in Minimally Invasive Surgery.

ABSTRACT: In the context of Minimally Invasive Surgery, surgeons mainly rely on visual feedback during medical operations. In common procedures such as tissue resection, the automation of endoscopic control is crucial yet challenging, particularly due to the interactive dynamics of multi-agent operations and the necessity for real-time adaptation. This paper introduces a novel framework that unites a Hierarchical Quadratic Programming controller with an advanced interactive perception module. This integration addresses the need for adaptive visual field control and robust tool tracking in the operating scene, ensuring that surgeons and assistants have optimal viewpoint throughout the surgical task. The proposed framework handles multiple objectives within predefined thresholds, ensuring efficient tracking even amidst changes in operating backgrounds, varying lighting conditions, and partial occlusions. Empirical validations in scenarios involving single, double, and quadruple tool tracking during tissue resection tasks have underscored the system's robustness and adaptability. The positive feedback from user studies, coupled with the low cognitive and physical strain reported by surgeons and assistants, highlight the system's potential for real-world application.

SUBMITTER: Fozilov K

PROVIDER: S-EPMC10748016 | biostudies-literature | 2023 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Endoscope Automation Framework with Hierarchical Control and Interactive Perception for Multi-Tool Tracking in Minimally Invasive Surgery.

Fozilov Khusniddin K Colan Jacinto J Davila Ana A Misawa Kazunari K Qiu Jie J Hayashi Yuichiro Y Mori Kensaku K Hasegawa Yasuhisa Y

Sensors (Basel, Switzerland) 20231216 24

In the context of Minimally Invasive Surgery, surgeons mainly rely on visual feedback during medical operations. In common procedures such as tissue resection, the automation of endoscopic control is crucial yet challenging, particularly due to the interactive dynamics of multi-agent operations and the necessity for real-time adaptation. This paper introduces a novel framework that unites a Hierarchical Quadratic Programming controller with an advanced interactive perception module. This integra ...[more]

PMID: 38139711

Similar Datasets

Project description:SignificanceAs an example of a minimally invasive arthroscopic surgical procedure, arthroscopic osteochondral autograft transplantation (OAT) is a common option for repairing focal cartilage defects in the knee joints. Arthroscopic OAT offers considerable benefits to patients, such as less post-operative pain and shorter hospital stays. However, performing OAT arthroscopically is an extremely demanding task because the osteochondral graft harvester must remain perpendicular to the cartilage surface to avoid differences in angulation.AimWe present a practical ArthroNavi framework for instrument pose localization by combining a self-developed stereo endoscopy with electromagnetic computation, which equips surgeons with surgical navigation assistance that eases the operational constraints of arthroscopic OAT surgery.ApproachA prototype of a stereo endoscope specifically fit for a texture-less scene is introduced extensively. Then, the proposed framework employs the semi-global matching algorithm integrating the matching cubes method for real-time processing of the 3D point cloud. To address issues regarding initialization and occlusion, a displaying method based on patient tracking coordinates is proposed for intra-operative robust navigation. A geometrical constraint method that utilizes the 3D point cloud is used to compute a pose for the instrument. Finally, a hemisphere tabulation method is presented for pose accuracy evaluation.ResultsExperimental results show that our endoscope achieves 3D shape measurement with an accuracy of <730 μm. The mean error of pose localization is 15.4 deg (range of 10.3 deg to 21.3 deg; standard deviation of 3.08 deg) in our ArthroNavi method, which is within the same order of magnitude as that achieved by experienced surgeons using a freehand technique.ConclusionsThe effectiveness of the proposed ArthroNavi has been validated on a phantom femur. The potential contribution of this framework may provide a new computer-aided option for arthroscopic OAT surgery.

Project description:A better understanding of automation disengagements can lead to improved safety and efficiency of automated systems. This study investigates the factors contributing to automation disengagements initiated by human operators and the automation itself by analyzing semi-structured interviews with 103 users of Tesla's Autopilot and FSD Beta. The factors leading to automation disengagements are represented by categories. In total, we identified five main categories, and thirty-five subcategories. The main categories include human operator states (5), human operator's perception of the automation (17), human operator's perception of other humans (3), the automation's perception of the human operator (3), and the automation incapability in the environment (7). Human operators disengaged the automation when they anticipated failure, observed unnatural or unwanted automation behavior (e.g., erratic steering, running red lights), or believed the automation is not capable to operate safely in certain environments (e.g., inclement weather, non-standard roads). Negative experiences of human operators, such as frustration, unsafe feelings, and distrust represent some of the adverse human operate states leading to automation disengagements initiated by human operators. The automation, in turn, monitored human operators and disengaged itself if it detected insufficient vigilance or speed rule violations by human operators. Moreover, human operators can be influenced by the reactions of passengers and other road users, leading them to disengage the automation if they sensed discomfort, anger, or embarrassment due to the automation's actions. The results of the analysis are synthesized into a conceptual framework for automation disengagements, borrowing ideas from the human factor's literature and control theory. This research offers insights into the factors contributing to automation disengagements, and highlights not only the concerns of human operators but also the social aspects of this phenomenon. The findings provide information on potential edge cases of automated vehicle technology, which may help to enhance the safety and efficiency of such systems.

Dataset Information

Endoscope Automation Framework with Hierarchical Control and Interactive Perception for Multi-Tool Tracking in Minimally Invasive Surgery.

Publications

Endoscope Automation Framework with Hierarchical Control and Interactive Perception for Multi-Tool Tracking in Minimally Invasive Surgery.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets