US 11,681,569 B2
Streaming data pipeline with batch performance
Michael Pippin, Sunnyvale, CA (US); David Willcox, Urbana, IL (US); Allie K. Watfa, Urbana, IL (US); George Aleksandrovich, Hoffman Estates, IL (US); and Jacob Tolar, Champaign, IL (US)
Assigned to YAHOO ASSETS LLC, New York, NY (US)
Filed by YAHOO ASSETS LLC, Dulles, VA (US)
Filed on May 22, 2020, as Appl. No. 16/881,302.
Prior Publication US 2021/0365307 A1, Nov. 25, 2021
Int. Cl. G06F 9/54 (2006.01); G06F 16/2455 (2019.01); G06F 16/242 (2019.01); G06F 16/182 (2019.01)
CPC G06F 9/542 (2013.01) [G06F 9/544 (2013.01); G06F 9/546 (2013.01); G06F 16/182 (2019.01); G06F 16/244 (2019.01); G06F 16/2456 (2019.01); G06F 16/24568 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
a stream processor configured to process events emitted by an event stream and output processed events;
a queue configured to receive the processed events from the stream processor;
a spooler configured to read the processed events from the queue, write the processed events to a buffer, and remove duplicated events in the buffer, the duplicated events including a composite event and an unjoined event processed by the stream processor before the composite event, the composite event including data associated with the unjoined event and data associated with a secondary event received after the unjoined event, the composite event created by the stream processor after receiving the secondary event; and
an auditor process configured to extract a written event from the buffer and confirm that the written event matches an event present in the event stream.