US 11,809,883 B2
Image pattern matching to robotic process automations
Brandon Nott, Bellevue, WA (US); Christian Berg, Seattle, WA (US); and Vaclav Skarda, Domazlice (CZ)
Filed by UiPath, Inc., New York, NY (US)
Filed on Jul. 8, 2021, as Appl. No. 17/370,145.
Prior Publication US 2023/0009201 A1, Jan. 12, 2023
Int. Cl. G06F 9/451 (2018.01); G06N 20/00 (2019.01); G06F 18/22 (2023.01)
CPC G06F 9/451 (2018.02) [G06F 18/22 (2023.01); G06N 20/00 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A computing system comprising:
a memory configured to store processor executable instructions for a workflow recommendation assistant engine; and
at least one processor coupled to the memory,
wherein the at least one processor is configured to execute the workflow recommendation assistant engine to cause the computing system to perform:
pre-processing one or more images of a user interface captured during user activity to generate one or more corresponding hashes comprising action characteristics, the action characteristics comprising a type of action, a number of actions, a proximity of actions, and interactions with the user interface;
holding, based on a sliding window, a fixed number of the one or more corresponding hashes, the workflow recommendation assistant engine determining a size of the sliding window and filling the sliding window in a first-in-first-out fashion with a sequence of the one or more corresponding hashes;
pattern matching the one or more corresponding hashes in the sliding window by comparing the action characteristics in and out of sequence to target characteristics of each of a plurality of robotic process automations to determine a match set of robotic process automations from the plurality of robotic process automations;
dynamically sizing the sliding window by automatic reconfiguration; and
presenting via a prompt the match set of robotic process automations in the user interface.