A vision transformer for decoding surgeon activity from surgical videos.

ABSTRACT: The intraoperative activity of a surgeon has substantial impact on postoperative outcomes. However, for most surgical procedures, the details of intraoperative surgical actions, which can vary widely, are not well understood. Here we report a machine learning system leveraging a vision transformer and supervised contrastive learning for the decoding of elements of intraoperative surgical activity from videos commonly collected during robotic surgeries. The system accurately identified surgical steps, actions performed by the surgeon, the quality of these actions and the relative contribution of individual video frames to the decoding of the actions. Through extensive testing on data from three different hospitals located in two different continents, we show that the system generalizes acro

SUBMITTER: Kiyasseh D 

PROVIDER: S-EPMC10307635 | biostudies-literature | 2023 Jun

REPOSITORIES: biostudies-literature