US 11,808,857 B2
Multi-sensor superresolution scanning and capture system
Schuyler Alexander Cullen, Santa Clara, CA (US); Gerard Dirk Smits, Los Gatos, CA (US); and Steven Dean Gottke, Concord, CA (US)
Assigned to Summer Robotics, Inc., Campbell, CA (US)
Filed by Summer Robotics, Inc., Campbell, CA (US)
Filed on Aug. 25, 2022, as Appl. No. 17/895,489.
Claims priority of provisional application 63/260,681, filed on Aug. 27, 2021.
Prior Publication US 2023/0060421 A1, Mar. 2, 2023
Int. Cl. G02B 26/08 (2006.01); G02B 26/10 (2006.01); G01S 7/48 (2006.01); G01S 17/04 (2020.01); G01S 17/10 (2020.01); G01S 17/42 (2006.01); G01S 17/48 (2006.01); G01S 17/66 (2006.01); G01S 17/894 (2020.01); G02B 26/12 (2006.01); G02B 27/00 (2006.01); H04N 23/698 (2023.01)
CPC G01S 17/894 (2020.01) [G01S 7/4802 (2013.01); G01S 7/4804 (2013.01); G01S 7/4808 (2013.01); G01S 17/04 (2020.01); G01S 17/10 (2013.01); G01S 17/42 (2013.01); G01S 17/48 (2013.01); G01S 17/66 (2013.01); G02B 26/0833 (2013.01); G02B 26/101 (2013.01); G02B 26/105 (2013.01); G02B 26/123 (2013.01); G02B 27/0031 (2013.01); H04N 23/698 (2023.01)] 28 Claims
OG exemplary drawing
 
1. A method for perceiving surfaces and objects using one or more processors that are configured to execute instructions, wherein the executed instructions enable performance of actions, comprising:
employing a sensing system to scan a plurality of paths across one or more objects using one or more beams, wherein the sensing system includes one or more event sensors and one or more image sensors, and wherein the one or more image sensors are a higher resolution than the one or more event sensors;
employing the one or more event sensors to provide one or more events that detect one or more of a movement, a shape, or a position for the one or more objects with one or more points for each cell location for each event sensor that is triggered based on detection of one or more portions of the one or more beams that are reflected by the one or more objects;
employing the one or more image sensors to provide one or more images based on the one or more reflected portions of the one or more beams;
generating a plurality of first trajectories based on the one or more events and one or more event paths, wherein the one or more points and the one or more event paths are employed to determine one or more event time parameterized curves for the one or more events;
generating a plurality of second trajectories based on the one or more images and one or more image paths that match the one or more event paths, wherein the one or more matched image paths are employed to determine one or more image time parameterized curves for the one or more images;
generating one or more enhanced trajectories based on a combination of the plurality of first trajectories and the plurality of second trajectories by matching the one or more event time parameterized curves to the one or more image time parameterized curves; and
providing the one or more enhanced trajectories to a modeling engine, wherein the one or more enhanced trajectories provide super resolution of an output of the sensing system, and wherein the modeling engine executes one or more actions based on the one or more enhanced trajectories and the one or more objects.