US 9,813,502 B1
Data transfers in columnar data systems
Xing Chen, Beijing (CN); Paul C. Huffman, Lively (CA); David C. Kalmuk, Toronto (CA); Sam S. Lightstone, Toronto (CA); and Shaorong Liu, Wilmette, IL (US)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on May 2, 2017, as Appl. No. 15/584,320.
Application 15/584,320 is a continuation of application No. 15/169,774, filed on Jun. 1, 2016.
This patent is subject to a terminal disclaimer.
Int. Cl. G06F 17/00 (2006.01); H04L 29/08 (2006.01); G06F 17/30 (2006.01); H04L 12/803 (2013.01)
CPC H04L 67/1097 (2013.01) [G06F 17/30315 (2013.01); G06F 17/30339 (2013.01); H04L 47/125 (2013.01)] 1 Claim
OG exemplary drawing
 
1. A method, executed by one or more processors, the method comprising:
receiving a request to transmit column group data to a target node, the column group data comprising C selected columns within a column-oriented data table where C is greater than 1 and less than the number of columns in the column-oriented data table;
determining an optimal packet size or packet size range for transmitting data to the target node;
determining a transmission row count R, corresponding to the optimal packet size or packet size range, for transmitting the column group data to the target node where R is greater than 1;
repeatedly transmitting transmission packets comprising R sequentially-ordered data elements for each of the C selected columns until all of the column group data is transmitted to the target node;
wherein the R sequentially-ordered data elements for each of the C selected columns reside sequentially within memory;
wherein R is selected to achieve one or more of preventing stalling, supporting serialization, increasing transmission throughput, and decreasing transmission packet latency;
wherein transmitting a transmission packet to the target node comprises sequentially accessing R data elements for each of the C selected columns and assembling the transmission packet within a transmission buffer;
wherein repeatedly transmitting transmission packets comprises transmitting multiple windowed transmission packets before a packet acknowledgement is received for a previous transmission packet of the multiple windowed transmission packets;
wherein a maximum windowed packet count for windowed transmission packets corresponds to a transmission buffer size or a receiving buffer size; and
wherein the data elements within the column-oriented data table are compressed data elements.