US 9,811,575 B2
Dynamic distributed generation of data representations from highly condensed data
Stacey Michelle Christian, Pittsboro, NC (US); Michael Stephen Whitcher, Apex, NC (US); Donald Kent McAlister, Apex, NC (US); and Phillip Elliot Hanna, Raleigh, NC (US)
Assigned to SAS Institute Inc., Cary, NC (US)
Filed by SAS Institute Inc., Cary, NC (US)
Filed on Mar. 30, 2017, as Appl. No. 15/474,832.
Application 15/474,832 is a continuation of application No. 15/474,636, filed on Mar. 30, 2017.
Claims priority of provisional application 62/315,235, filed on Mar. 30, 2016.
Claims priority of provisional application 62/328,248, filed on Apr. 27, 2016.
Prior Publication US 2017/0286509 A1, Oct. 5, 2017
Int. Cl. G06F 17/30 (2006.01)
CPC G06F 17/30569 (2013.01) [G06F 17/30333 (2013.01); G06F 17/30592 (2013.01)] 30 Claims
OG exemplary drawing
 
1. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:
store, at a node device of a grid of node devices, a data block of a plurality of data blocks of a data blob, wherein:
the plurality of data blocks of the data blob are distributed among the grid of node devices to be stored by the grid of node devices;
the data blob comprises multiple data items; and
each data block of the plurality of data blocks comprises a subset of the multiple data items;
receive, at the node device, data blob metadata that comprises an indication of organization of the multiple data items within the data blob;
receive, at the node device, a command that originates from a requesting device to generate at least one row of a representation portion of a plurality of representation portions from the data block, wherein:
a representation data that provides a two-dimensional (2D) representation of data items of the data blob is to be assembled from the plurality of representation portions;
the plurality of representation portions is to be generated from a subset of data blocks of the plurality of data blocks by a subset of node devices of the grid of node devices that store the subset of data blocks; and
the subset of node devices comprises the node device;
use the data blob metadata and a node block map of a plurality of node block maps to derive one or more transforms to be performed at the node device with the data block to generate the at least one row of the representation portion, wherein:
the plurality of node block maps is provided to the subset of node devices to generate the plurality of representation portions from the subset of data blocks; and
the node block map comprises an indication of which data items of the data block are required to generate the representation portion, what data values are to be generated from the required data items, and an organization of the generated data values within the representation portion;
perform, at the node device, the one or more transforms with the data block to generate the at least one row of the representation portion;
transmit, from the node device to a coordinating device of the grid of node devices, node data comprising an indication of availability of at least storage resources of the node device to enable the coordinating device to coordinate provision of the plurality of data blocks by at least one source device to the grid of node devices, and to coordinate storage of the plurality of data blocks by the grid of node devices;
receive, at the node device, a command to store the data block from the coordinating device;
receive, at the node device, the data block from the at least one source device;
receive, at the node device, the command to generate at least one row of the representation portion from the coordinating device; and
transmit, from the node device, the at least one row of the representation portion to the coordinating device to enable generation, at the coordinating device, of at least one row of the representation data from at least one row of each representation portion of the plurality of representation portions.