US 9,811,880 B2
Backfilling points in a point cloud
Terrell Nathan Mundhenk, Calabasas, CA (US); Yuri Owechko, Newbury Park, CA (US); and Kyungnam Kim, Oak Park, CA (US)
Assigned to THE BOEING COMPANY, Chicago, IL (US)
Filed by The Boeing Company, Chicago, IL (US)
Filed on Nov. 9, 2012, as Appl. No. 13/673,429.
Prior Publication US 2014/0132733 A1, May 15, 2014
Int. Cl. H04N 7/18 (2006.01); G06T 3/40 (2006.01); H04N 13/00 (2006.01); G06T 5/00 (2006.01); H04N 13/02 (2006.01); G06T 17/00 (2006.01)
CPC G06T 3/4007 (2013.01) [G06T 3/4038 (2013.01); G06T 5/002 (2013.01); G06T 17/00 (2013.01); H04N 13/0022 (2013.01); H04N 13/025 (2013.01); G06T 2210/56 (2013.01)] 17 Claims
OG exemplary drawing
 
1. An apparatus for improved resolution and accuracy of an object in a scene in a point cloud, comprising:
an image processing system comprising a processor configured to:
receive a two-dimensional image of a scene from a camera system and a point cloud of the scene having a first resolution from a light detection and ranging system;
create transformed points by mapping point locations for a portion of points in the point cloud describing a three-dimensional representation of the scene to corresponding pixel locations in the two-dimensional image of the scene to form the transformed points;
create a fused data array using the two-dimensional image and the transformed points, wherein each transformed point corresponds to a pixel in the two-dimensional image and to a matched element in a number of matched elements in the fused data array;
calculate a goodness score using a combination of a distance of a transformed point mapped to a pixel corresponding to the matched element from the camera system, a dissimilarity of the matched element to other matched elements wherein the goodness score increases as the dissimilarity increases, and a depth value in a data vector associated with the matched element, wherein the goodness score is generated based on a relationship where:

OG Complex Work Unit Drawing
 and
Δi=di o·γ,
wherein:
G i is the goodness score of an ith matched element;
Mi is a dissimilarity score for the ith matched element;
Δi is a distance measurement for the ith matched element;
j is an index for an n matched elements;
F is a response to a feature;
m is a number of features;
dio is the depth value in a data vector associated with the ith matched element; and
γ is a normalizing constant;
use the goodness score to identify a number of support elements; use the fused data array and the number of support elements to identify new points for the point cloud;
add the new points to the point cloud to form a new point cloud, wherein the new point cloud has a second resolution greater than the first resolution; and
identify the object in the scene using the new point cloud.