US 11,816,423 B2
Automated clinical documentation system and method
Donald E. Owen, Orlando, FL (US); Uwe Helmut Jost, Groton, MA (US); Daniel Paulino Almendro Barreda, London (GB); and Dushyant Sharma, Woburn, MA (US)
Assigned to Nuance Communications, Inc., Burlington, MA (US)
Filed by Nuance Communications, Inc., Burlington, MA (US)
Filed on Mar. 23, 2021, as Appl. No. 17/210,052.
Application 17/210,052 is a continuation of application No. 16/058,912, filed on Aug. 8, 2018, granted, now 10,957,427.
Claims priority of provisional application 62/638,809, filed on Mar. 5, 2018.
Claims priority of provisional application 62/543,762, filed on Aug. 10, 2017.
Prior Publication US 2021/0210180 A1, Jul. 8, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. G16H 10/60 (2018.01); A61B 5/00 (2006.01); G06F 3/16 (2006.01); G06F 16/635 (2019.01); G06F 16/683 (2019.01); G06F 16/904 (2019.01); G06F 21/62 (2013.01); G06F 40/174 (2020.01); G06F 40/30 (2020.01); G06F 40/40 (2020.01); G06K 9/62 (2022.01); G06K 19/077 (2006.01); G06N 3/00 (2006.01); G06T 7/00 (2017.01); G06V 20/10 (2022.01); H04R 3/12 (2006.01); G16H 40/20 (2018.01); G10L 17/00 (2013.01); G16Y 20/00 (2020.01); G06V 20/52 (2022.01); G06V 40/16 (2022.01); G06V 40/20 (2022.01); G06V 40/10 (2022.01); H04L 51/222 (2022.01); G06F 18/25 (2023.01); G16H 50/30 (2018.01); G16H 80/00 (2018.01); G16H 30/00 (2018.01); G16H 40/63 (2018.01); H04N 7/18 (2006.01); H04R 3/00 (2006.01); G16H 10/20 (2018.01); H04R 1/32 (2006.01); G16H 15/00 (2018.01); H04L 51/02 (2022.01); G16H 40/60 (2018.01); G16H 30/20 (2018.01); G16H 30/40 (2018.01); G10L 21/0232 (2013.01); G16H 50/20 (2018.01); G06N 3/006 (2023.01); G16B 50/00 (2019.01); G10L 15/08 (2006.01); G11B 27/10 (2006.01); H04R 3/02 (2006.01); G10L 15/26 (2006.01); H04R 1/40 (2006.01); G10L 15/22 (2006.01); G10L 21/0208 (2013.01); G10L 15/18 (2013.01)
CPC G16H 10/60 (2018.01) [A61B 5/7405 (2013.01); G06F 3/16 (2013.01); G06F 16/637 (2019.01); G06F 16/685 (2019.01); G06F 16/904 (2019.01); G06F 18/25 (2023.01); G06F 21/6245 (2013.01); G06F 40/174 (2020.01); G06F 40/30 (2020.01); G06F 40/40 (2020.01); G06K 19/07762 (2013.01); G06N 3/006 (2013.01); G06T 7/00 (2013.01); G06V 20/10 (2022.01); G06V 20/52 (2022.01); G06V 40/103 (2022.01); G06V 40/16 (2022.01); G06V 40/172 (2022.01); G06V 40/23 (2022.01); G10L 15/08 (2013.01); G10L 17/00 (2013.01); G10L 21/0232 (2013.01); G11B 27/10 (2013.01); G16B 50/00 (2019.02); G16H 10/20 (2018.01); G16H 15/00 (2018.01); G16H 30/00 (2018.01); G16H 30/20 (2018.01); G16H 30/40 (2018.01); G16H 40/20 (2018.01); G16H 40/60 (2018.01); G16H 40/63 (2018.01); G16H 50/20 (2018.01); G16H 50/30 (2018.01); G16H 80/00 (2018.01); G16Y 20/00 (2020.01); H04L 51/02 (2013.01); H04L 51/222 (2022.05); H04N 7/183 (2013.01); H04R 1/326 (2013.01); H04R 3/005 (2013.01); H04R 3/12 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/10044 (2013.01); G06T 2207/10048 (2013.01); G06T 2207/10116 (2013.01); G06T 2207/10132 (2013.01); G10L 15/1815 (2013.01); G10L 15/22 (2013.01); G10L 15/26 (2013.01); G10L 2021/02082 (2013.01); H04N 7/181 (2013.01); H04R 1/406 (2013.01); H04R 3/02 (2013.01); H04R 2420/07 (2013.01); H04S 2400/15 (2013.01)] 18 Claims
OG exemplary drawing
 
1. A computer-implemented method for synchronizing machine vision and audio, executed on a computing device, comprising:
obtaining encounter information of a patient encounter, wherein the encounter information includes machine vision encounter information and audio encounter information, wherein obtaining encounter information includes utilizing a virtual assistant to prompt an encounter participant to provide at least one of a pre-visit portion of the patient encounter and a post-visit portion of the patient encounter, wherein the virtual assistant verbally prompts the encounter participant to provide at least one of the pre-visit portion of the patient encounter and the post-visit portion of the patient encounter;
temporally aligning the machine vision encounter information and the audio encounter information to produce a temporally-aligned encounter recording;
receiving a user selection of at least a portion of the temporally-aligned encounter information to be rendered, wherein at least the portion of the temporally-aligned encounter recording selected by the user is the at least one of the pre-visit portion of the patient encounter obtained by the virtual assistant and the post-visit portion of the patient encounter obtained by the virtual assistant; and
rendering, visually, only the user selection of at least the portion of the temporally-aligned encounter recording selected by the user to include an indication of each of the portions of the audio encounter information during which the encounter participant of the patient encounter speaks in the pre-visit portion of the patient encounter when the pre-visit portion of the patient encounter is selected by the user, and rendering, visually, only the user selection of at least the portion of the temporally-aligned encounter recording selected by the user to include an indication of each of the portions of the audio encounter information during which the encounter participant of the patient encounter speaks in the post-visit portion of the patient encounter when the post-visit portion of the patient encounter is selected by the user.