US 11,741,078 B1
Witness service for ensuring data consistency in a distributed storage system
Vishwas Narendra, Evanston, IL (US); James Zuber, Seattle, WA (US); Arvinth Ravi, Seattle, WA (US); Sameer Choudhary, Seattle, WA (US); Rajesh Sampath, Redmond, WA (US); Tyler Michael Jung, Seattle, WA (US); Nicholas Jacob Essenburg, Seattle, WA (US); Phillip H. Pruett, Woodinville, WA (US); Prathiban Mohanasundaram, Redmond, WA (US); Nishant Jain, Bothell, WA (US); Suchindra Yogendra Agarwal, Kirkland, WA (US); David Grant Emeny-Smith, Seattle, WA (US); Hardik Navin Nagda, Seattle, WA (US); Bryan T. Wilkinson, Seattle, WA (US); Liming Ye, Bothell, WA (US); Janko Jerinic, Vancouver (CA); Vijayasarathy Kannan, Vancouver (CA); Pierre Vigneras, Richmond (CA); and Nikhil Shah, Redmond, WA (US)
Assigned to Amazon Technologies, Inc., Seattle, WA (US)
Filed by Amazon Technologies, Inc., Seattle, WA (US)
Filed on Sep. 30, 2021, as Appl. No. 17/449,597.
Int. Cl. G06F 15/16 (2006.01); G06F 16/23 (2019.01); G06F 16/22 (2019.01); G06F 16/27 (2019.01); G06F 16/2455 (2019.01)
CPC G06F 16/2365 (2019.01) [G06F 16/2255 (2019.01); G06F 16/2358 (2019.01); G06F 16/24552 (2019.01); G06F 16/27 (2019.01)] 16 Claims
OG exemplary drawing
 
1. A distributed storage system comprising:
an object storage service comprising one or more object storage nodes, wherein the object storage service stores a plurality of data objects;
a witness service comprising a plurality of witness nodes, wherein a witness node of the plurality of witness nodes stores a hash table comprising freshness data that associates a first key of a first data object of the plurality of data objects with a first sequence number, and wherein the first sequence number represents a most recent change of location data associated with the first data object; and
an index service comprising a plurality of cache nodes and a plurality of persistent storage nodes, wherein a cache node of the plurality of cache nodes is configured to at least:
receive a request for the location data, wherein the request comprises the first key, and wherein the location data represents a location in the object storage service at which the first data object is stored;
access first cached key map data that maps the first key to cached location data, wherein the first cached key map data is associated with a second sequence number;
obtain a plurality of sequence numbers associated with the first key from the witness service, wherein each sequence number of the plurality of sequence numbers is obtained from a different witness node of the plurality of witness nodes;
determine that the second sequence number is less than one or more of the plurality of sequence numbers including the first sequence number;
obtain, from a persistent storage node of the plurality of persistent storage nodes, updated key map data that maps the first key to updated location data, wherein the updated key map data is obtained based at least partly on the first sequence number;
generate a response to the request comprising the updated location data; and
replace the first cached key map data with second cached key map data based at least partly on the updated key map data.