CPC G10L 15/22 (2013.01) [G06F 3/167 (2013.01); G06F 17/16 (2013.01); G06F 40/216 (2020.01); G06N 3/02 (2013.01); G10L 15/18 (2013.01)] | 20 Claims |
1. A method for determining the answer to a query in a document, comprising:
encoding, by an encoder, the query and the document;
generating a query-aware context encodings G by a bidirectional attention system using the encoded query and the encoded document;
performing a hierarchical self-attention on the query aware document by a hierarchical self-attention system by applying a word to word attention and a word to sentence attention mechanism resulting in a matrix M by:
applying a bidirectional recurrent neural network (BiRNN) on the query-aware context encoding G to produce a matrix G′;
extracting sentence-level encodings S′ from G′;
producing a word-word self-attention matrix A_w by comparing each word in G′ with each other word in G′
producing a word-sentence self-attention matrix A_S by comparing each word in G′ to each sentence in the extracted sentence-level encodings S′, wherein the matrix M is based upon A_w and A_s; and
determining the starting word and the ending word of the answer in the document by a span detector based upon the matrix M.
|