CPC H04N 7/183 (2013.01) [A61B 1/0004 (2022.02); A61B 1/00009 (2013.01); A61B 1/00045 (2013.01); A61B 1/00055 (2013.01); A61B 1/000096 (2022.02); A61B 1/045 (2013.01); A61B 1/3132 (2013.01); A61B 34/20 (2016.02); A61B 34/25 (2016.02); A61B 34/30 (2016.02); A61B 90/37 (2016.02); G06N 20/00 (2019.01); G06V 20/41 (2022.01); H04N 7/0117 (2013.01); H04N 19/59 (2014.11); A61B 2034/2055 (2016.02); A61B 2034/2057 (2016.02); A61B 2090/373 (2016.02); G06V 20/44 (2022.01)] | 19 Claims |
1. A computer-implemented method comprising:
receiving an endoscope video of a surgical procedure corresponding to an endoscope view;
displaying a first portion of video images of the endoscope video on a display to assist a surgeon performing the surgical procedure, wherein the first portion of the video images represents an on-screen portion of the endoscope view;
while displaying the first portion of the video images, using one or more deep-learning models to monitor a second portion of the video images of the endoscope video not being displayed on the display, wherein the second portion of the video images represents an off-screen portion of the endoscope view, wherein each deep-learning model in the one or more deep-learning models is constructed to detect an adverse event among a set of known adverse events, and wherein the one or more deep-learning models include:
a first deep-learning model trained to detect a first tool off-screen risk event in a set of surgical-tool off-screen risk events, wherein the first tool off-screen event is when two jaws of a surgical tool are unintentionally engaged on a tissue; or
a second deep-learning model trained to detect a second tool off-screen risk event in the set of surgical-tool off-screen risk events, wherein the second tool off-screen event is when a sharp tip of a surgical tool is approaching a critical anatomy; and
in response to detecting the detected adverse event by the one or more deep-learning models in the second portion of the video images, notifying the surgeon of the detected adverse event to prompt an appropriate action to the detected adverse event.
|