US 11,721,335 B2
Hierarchical self-attention for machine comprehension
Tao Li, Cambridge, MA (US); Sheikh Sadid Al Hasan, Cambridge, MA (US); Vivek Varma Datla, Cambridge, MA (US); and Oladimeji Feyisetan Farri, Yorktown Heights, NY (US)
Assigned to Koninklijke Philips N.V., Eindhoven (NL)
Filed by KONINKLIJKE PHILIPS N.V., Eindhoven (NL)
Filed on Jun. 30, 2020, as Appl. No. 16/916,697.
Claims priority of provisional application 62/869,075, filed on Jul. 1, 2019.
Prior Publication US 2021/0005195 A1, Jan. 7, 2021
Int. Cl. G10L 15/00 (2013.01); G10L 15/22 (2006.01); G10L 15/18 (2013.01); G06F 40/216 (2020.01); G06N 3/02 (2006.01); G06F 17/16 (2006.01); G06F 3/16 (2006.01)
CPC G10L 15/22 (2013.01) [G06F 3/167 (2013.01); G06F 17/16 (2013.01); G06F 40/216 (2020.01); G06N 3/02 (2013.01); G10L 15/18 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method for determining the answer to a query in a document, comprising:
encoding, by an encoder, the query and the document;
generating a query-aware context encodings G by a bidirectional attention system using the encoded query and the encoded document;
performing a hierarchical self-attention on the query aware document by a hierarchical self-attention system by applying a word to word attention and a word to sentence attention mechanism resulting in a matrix M by:
applying a bidirectional recurrent neural network (BiRNN) on the query-aware context encoding G to produce a matrix G′;
extracting sentence-level encodings S′ from G′;
producing a word-word self-attention matrix A_w by comparing each word in G′ with each other word in G′
producing a word-sentence self-attention matrix A_S by comparing each word in G′ to each sentence in the extracted sentence-level encodings S′, wherein the matrix M is based upon A_w and A_s; and
determining the starting word and the ending word of the answer in the document by a span detector based upon the matrix M.