MUTUAL: Towards Holistic Sensing and Inference in the Operating Room.

ABSTRACT: Embodied AI (E-AI) in the form of intelligent surgical robotics and other agents is calling for data platforms to facilitate its development and deployment. In this work, we present a cross-platform multimodal data recording and streaming software, MUTUAL, successfully deployed on two clinical studies, along with its ROS 2 distributed adaptation, MUTUAL-ROS 2. We describe and compare the two implementations of MUTUAL through their recording performance under different settings. MUTUAL offers robust recording performance at target configurations for multiple modalities, including video, audio, and live expert commentary. While this recording performance is not matched by MUTUAL-ROS 2, we demonstrate its advantages related to real-time streaming capabilities for AI inference and more horizon

SUBMITTER: Quarez J 

PROVIDER: S-EPMC7617325 | biostudies-literature | 2025 Jan

REPOSITORIES: biostudies-literature