US 9,811,946 B1
High resolution (HR) panorama generation without ghosting artifacts using multiple HR images mapped to a low resolution 360-degree image
Kwok Wai Hung, Hong Kong (HK); Luhong Liang, Hong Kong (HK); and Xuejiao Liu, Guangdong (CN)
Assigned to Hong Kong Applied Science and Technology Research Institute Company, Limited, Hong Kong (HK)
Filed by Kwok Wai Hung, Hong Kong (HK); Luhong Liang, Hong Kong (HK); and Xuejiao Liu, Guangdong (CN)
Filed on May 30, 2016, as Appl. No. 15/168,156.
Int. Cl. G06T 3/40 (2006.01); G06T 19/00 (2011.01); H04N 5/232 (2006.01)
CPC G06T 19/006 (2013.01) [G06T 3/4038 (2013.01); H04N 5/23238 (2013.01); G06T 2200/04 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A hybrid-resolution panoramic processor comprising:
a panorama input for receiving a 360-degree image of a 360-degree scene;
a detailed-view input for receiving a plurality of High-Resolution (HR) images taken by a plurality of HR cameras, each HR image covering only a limited-arc portion of the 360-degree scene;
wherein the plurality of HR images have a higher pixel density of visible objects in the 360-degree scene than does the 360-degree image;
an upscaler that generates additional pixels for insertion into the 360-degree image to generate an upscaled 360-degree image;
a patch searcher for scanning the upscaled 360-degree image to locate a plurality of query patches, and for each query patch, searching a search window within an HR image to locate a best-matching HR patch in the HR image, and for a query patch near an interface between adjacent HR images, also searching a second search window within a second HR image to locate a second best-matching HR patch in the second HR image;
a patch collector that collects the best-matching HR patch and the second best-matching HR patch;
a patch evaluator that generates a similarity parameter for each best-matching HR patch collected by the patch collector, the similarity parameter quantifying a visual similarity of a best-matching HR patch with a query patch;
a reconstructed patch generator that blends the best-matching HR patch and the second best-matching HR patch using the similarity parameters as blending weights to generate a reconstructed patch; and
a joint-view reconstructor that places the reconstructed patch onto a reconstructed HR panorama image at a location of the query patch within the upscaled 360-degree image.