US 11,810,360 B2
Methods for arthroscopic surgery video segmentation and devices therefor
Brian Quist, Salem, NH (US); and Carlos A. Rodriguez, Wakefield, MA (US)
Assigned to SMITH & NEPHEW, INC., Memphis, TN (US); SMITH & NEPHEW ORTHOPAEDICS AG, Zug (CH); and SMITH & NEPHEW ASIA PACIFIC PTE. LIMITED, Singapore (SG)
Filed by SMITH & NEPHEW, INC., Memphis, TN (US); SMITH & NEPHEW ORTHOPAEDICS AG, Zug (CH); and SMITH & NEPHEW ASIA PACIFIC PTE. LIMITED, Singapore (SG)
Filed on Sep. 13, 2022, as Appl. No. 17/931,600.
Application 17/931,600 is a continuation of application No. PCT/US2021/025704, filed on Apr. 5, 2021.
Claims priority of provisional application 63/004,585, filed on Apr. 3, 2020.
Prior Publication US 2023/0005266 A1, Jan. 5, 2023
Int. Cl. G06V 20/40 (2022.01); G06T 7/00 (2017.01)
CPC G06V 20/49 (2022.01) [G06T 7/0012 (2013.01); G06T 2207/10016 (2013.01); G06T 2207/10068 (2013.01); G06T 2207/30204 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A method for automatically segmenting surgical video data, the method comprising:
obtaining, by a computing device, video data for a surgical procedure from a camera configured to capture the video data, wherein the video data comprises a field of view of an anatomical region of a patient during the surgical procedure;
generating, by the computing device, a first video data stream and a second video data stream;
applying, by an AI processing unit associated with the computing device, one or more machine learning models to the second video data stream;
identifying, by the AI processing unit, one or more segments of the second video data stream based on the machine learning models; and
generating, by the AI processing unit, a set of tags associated with the one or more segments of the second video data stream;
wherein identifying the one or more segments of the second video data stream further comprises:
analyzing, by the AI processing unit, each of a plurality of frames in the second video data stream;
determining, by the AI processing unit, one or more features present within each of the plurality of frames in the second video data stream based on the machine learning models; and
identifying, by the AI processing unit, the one or more segments of the second video data stream by:
comparing, by the AI processing unit, each of the plurality of frames based on the one or more features present with a previous one of the frames in the plurality of frames,
identifying, by the AI processing unit, a state change in one of the one or more features based on the comparing, and
identifying, by the AI processing unit, a new segment in the second video data stream based on the state change.