	US 11,836,917 B2
	Surgical video consumption by identifying useful segments in surgical videos
Joëlle Barral, Mountain View, CA (US); Martin Habbecke, Palo Alto, CA (US); Lin Yang, Sunnyvale, CA (US); and Xing Jin, San Jose, CA (US)
Assigned to Verily Life Sciences LLC, South San Francisco, CA (US)
Filed by Verily Life Sciences LLC, South San Francisco, CA (US)
Filed on Apr. 14, 2022, as Appl. No. 17/720,414.
Application 17/720,414 is a continuation of application No. 16/807,629, filed on Mar. 3, 2020, granted, now 11,348,235.
Claims priority of provisional application 62/822,106, filed on Mar. 22, 2019.
Prior Publication US 2022/0237792 A1, Jul. 28, 2022
This patent is subject to a terminal disclaimer.
Int. Cl. G06T 7/136 (2017.01); G06V 20/40 (2022.01); G06T 7/00 (2017.01); G06N 20/00 (2019.01)

CPC G06T 7/0012 (2013.01) [G06N 20/00 (2019.01); G06T 7/136 (2017.01); G06V 20/49 (2022.01)]

20 Claims

1. A method comprising:

dividing a video of a surgical procedure into a plurality of segments based, at least in part, upon (a) log data obtained from a robotic system where the video of the surgical procedure is captured or (b) detected surgical phases or events by applying a machine learning model on the video;

pre-processing the video to include annotations specifying phases or events of the surgical procedure;

accessing user activities of a plurality of users who have watched the video of the surgical procedure, the user activities comprising operations performed during playback of the video;

determining a popularity score for each of the plurality of segments based on assigning weights to the operations according to experience levels of the plurality of users or insertion or use of a surgical tool;

identifying one or more useful segments from the plurality of segments based on the popularity scores; and

associating an indication of the identified one or more useful segments with the video of the surgical procedure.

10. A method comprising:

normalizing a plurality of videos of a surgical procedure to identify corresponding segments of the plurality of videos;

determining, for each video of the plurality of videos, a plurality of video segments based, at least in part, upon (a) log data obtained from a corresponding robotic system where the video is captured or (b) detected surgical phases or events by applying a machine learning model on the video;

pre-processing each video of the plurality of videos to include annotations specifying phases or events of the surgical procedure;

identifying one or more useful segments in the plurality of videos based on different characteristics between corresponding video segments of the plurality of videos; and

associating an indication of the identified one or more useful segments with the plurality of videos of the surgical procedure.

16. A method comprising:

identifying one or more useful segments from a video based on operations performed by a plurality of users during playback of the video;

receiving a request to watch the video, the request comprising a watch time limit specifying the maximum amount of time that a user associated with the request will spend watching the video;

accessing a user preference associated with the user, the user preference specifying a subject that the user is interested in watching; and

generating an adapted video by selecting one or more useful segments of the video that contain the subject that the user is interested in watching and combining the one or more useful segments to create the adaptive video having a time duration no greater than the watch time limit.