Memory Networks for Question Answering

1. Memory Networks for Question Answering

2. Question Answering • Return the correct answer to a question expressed as a natural language question • Different from IR (search), which returns the most relevant documents, to a query expressed with keywords. • In principle, most AI/NLP tasks can be reduced to QA

3. Examples I: Mary walked to the bathroom. I: Sandra went to the garden. I: Daniel went back to the garden. I: Sandra took the milk there. Q: Where is the milk? A: garden I: Everybody is happy. Q: What’s the sentiment? A: positive I: Jane has a baby in London. Q: What are the named entities? A: Jane-person, London-location I: Jane has a baby in London. A: NNP VBZ DT NN IN NNP I: I think this book is fun. Q: What is the French translation? A: Je croisque celivre estamusant.

4. Difficulty • No single model architecture for NLP is consistently best on all tasks

5. Tasks Reading Comprehension Inference QA • Story T • Question q • Answer q performing inference over T • Story T • Cloze sentence s • Fill the gaps in s using information from T Sentence in which key words have been deleted

6. bAbI Dataset

7. The Facebook bAbI dataset • A collection of 20 tasks • Eachtaskchecksoneskillthatareasoningsystemshouldhave. • Aims atsystemsabletosolvealltasks:notaskspecificengineering.

8. (T2) Two supporting facts (T1) Single supporting fact • Questionswhereasinglesupportingfact,previouslygiven,providestheanswer. • Simplestcaseofthis:askingforthelocationofaperson. • Hardertask:questionswheretwosupportingstatementshavetobe chainedtoanswerthequestion. John is in the playground. Bob is in the office. Where is John? A: playground John is in the playground. Bob is in the office. John picked up the football. Bob went to the kitchen. Where is the football? A: playground Where was Bob before the kitchen? A: office

9. (T4) Two argument relations (T3) Three supporting facts • Similarly, one can make a task with three supporting facts • The first three statements are all required to answer this • To answer questions the ability to differentiate and recognize subjects and objects is crucial. • We consider the extreme case: sentences feature re-ordered words: John picked up the apple. John went to the office. John went to the kitchen. John dropped the apple. Where was the apple before the kitchen? A:office The office is north of the bedroom. The bedroom is north of the bathroom. What is north of the bedroom? A:office What is the bedroom north of? A:bathroom

10. (T7) Counting (T6) Yes/No questions • This task tests, in the simplest case possible (with a single supporting fact) the ability of a model to answer true/false type questions: • This task tests the ability of the QA system to perform simple counting operations, by asking about the number of objects with a certain property Daniel picked up the football. Daniel dropped the football. Daniel got the milk. Daniel took the apple. How many objects is Daniel holding? A:two John is in the playground. Daniel picks up the milk. Is John in the classroom? A:no Does Daniel have the milk? A:yes

11. (T18) Reasoning about size (T17) Positional reasoning • This task tests spatial reasoning, one of many components of the classical block world • The Yes/No task (6) is a prerequisite. • This task requires reasoning about relative size of objects • Tasks 3 (three supporting facts) and 6 (Yes/No) are prerequisites The triangle is to the right of the blue square The red square is on top of the blue square The red sphere is to the right of the blue square. Is the red sphere to the right of the blue square? A:yes Is the red square to the left of the triangle? A:yes The football fits in the suitcase. The suitcase fits in the cupboard. The box of chocolates is smaller than the football. Will the box of chocolates fit in the suitcase? A:yes

12. (T20) Agent’s motivation (T19) Path finding • thegoalistofindthepathbetweenlocations • Thistaskisdifficultbecauseiteffectivelyinvolvessearch The kitchen is north of the hallway. The den is east of the hallway. How do you go from den to kitchen? A: west,north John is hungry. John goes to the kitchen. John grapples the apple there. Daniel is hungry. Why did John go to the kitchen? A: hungry Where does Daniel go?A: kitchen

13. Difficulty

14. Dynamic Memory Networks

15. Intuition • Imagine having to read an article, memorize it, then get asked questions about it -> Hard! • You can't store everything in working memory • Optimal: give youthe input data, give you the question, allow as many glances as possible

16. Dynamic Memory Networks Semantic Memory Module (Embeddings) Use RNNs, specifically GRUs for every module Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

17. DMN: The Input Module Final GRU Output for sentence Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

18. DMN: Question Module Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

19. DMN: Episodic Memory, 1st hop Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

20. DMN: Episodic Memory, 2nd hop Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

21. Inspiration from Neuroscience • Episodic memory is the memory of autobiographical events (times, places, etc). A collection of past personal experiences that occurred at a particular time and place. • The hippocampus, the seat of episodic memory in humans, is active during transitive inference • In the DMN repeated passes over the input are needed for transitive inference Slide from Richard Socher

22. DMN • When the end of the input is reached, the relevant facts are summarized in another GRU or simple NNet Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

23. DMN: Answer Module Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

24. DMN How many GRUs were used with 2 hops? Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

25. Experiments: QA on bAbI (1k)

26. Focus during processing a query Ask Me Anything: Dynamic Memory Networks for Natural Language Processing, Kumar et. al. ICML 2016

27. Comparison: MemNets vs DMN Differences Similarities • MemNets and DMNs have input, scoring, attention and response mechanisms • For input representations MemNets use bag of word, nonlinear or linear embeddings that explicitly encode position • MemNets iteratively run functions for attention and response • DMNs shows that neural sequence models can be used for input representation, attention and response mechanisms  naturally captures position and temporality • Enables broader range of applications

28. Question Dependent Recurrent Entity Network for Question Answering Andrea Madotto Giuseppe Attardi

29. Examples RQA: bAbI RC: CNNnews articles Story Story Robert Downey Jr. may be Iron Man in the popular Marvel superhero films, but he recently dealt in some advanced bionic ... John picked up the apple John went to the office John went to the kitchen John dropped the apple Question Question "@placeholder" star Robert Downey Jr presents a young child with a bionic arm Where was the apple before the kitchen? Answer Answer office Iron Man

30. Related Work Many models have been proposed to solve the RQA and RC. Pointer Networks Memory Networks • Attentive Sum Reader • Attention Over Attention • EpiReader • Stanford Attentive Reader • Dynamic EntityRepresentation • Memory Network • Recurrent Entity Network • Dynamic Memory Network • Neural Touring Machine • End To End Memory Network (MemN2N)

31. Question Dependent Recurrent Entity Network • Based on the Memory Network framework as a variant of Recurrent Entity Network (Henaff et a., 2017) • Input Encoder: creates an internal representation • Dynamic Memory: stores relevant information about entities (Person, Location, etc.) • Output Module: generates the output • Idea: include the question (q) in the memorization process

32. Input Encoder • Set of sentences{s1,…,st}and a questionq. • ER|V |×dembedding matrix→E(w) = eRd • {e1(s), … ,em(s)} vectors for the words inst • {e1(q), … ,ek(q)}} vectors for the words inq • f (s/q) = { f1 , … , fm} multiplicative masks withfi Rd

33. Dynamic Memory Consists ina set of blocks meant torepresent entities in the story. A block is made by a key kithat identifies the entity and a hidden state hithat stores information about it.

34. Output Module • Scoresmemories hiusing q • embedding matrix E R|V|×dto generate the outputŷ where ŷR|V| representsthe model answer Training Given{(xi,yi)}ni=1,we used cross entropy loss function H(y, ŷ). End-To-End training with standard Backpropagation Through Time (BPTT) algorithm.

35. Overall Architecture Overallpicture

36. Experiments and Results • QDREN implementation1withTensorFlow • Tests on the RQA and RC tasks using: RC - CNNnewsarticles RQA - bAbI1K • CNN news articles • split into: • 380298 training • 3924 validation • 3198 test • 20 tasks • each of which has: • 900 training • 100 validation • 1000 test 1 Available athttps://github.com/andreamad8/QDREN

37. RQA - bAbI1K Error rates on the test set.

38. RC - CNN news articles • Four variants, using either plain text or windows of text as input: • REN • REN + WIND • QDREN • QDREN + WIND window ..

39. Analysis: REN gating activation where ismary? 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 marywenttothekitchen john movedto the hallway sandrawenttothekitchen johnwentbacktotheoffice mary travelled to thehallway marymovedtothebathroom john journeyed to thekitchen daniel journeyed to thegarden sandrawentbacktothegarden sandrawentbacktothebathroom

40. Analysis: QDREN gating activation where ismary? 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 marywenttothekitchen john movedto the hallway sandrawenttothekitchen johnwentbacktotheoffice mary travelled to thehallway marymovedtothebathroom john journeyed to thekitchen daniel journeyed to thegarden sandrawentbacktothegarden sandrawentbacktothebathroom

41. References • Ask Me Anything: Dynamic Memory Networks for Natural Language Processing (Kumar et al., 2015) • Dynamic Memory Networks for Visual and Textual Question Answering (Xiong et al., 2016) • Sequence to Sequence (Sutskeveret al., 2014) • Neural Turing Machines (Graves et al., 2014) • Teaching Machines to Read and Comprehend (Hermann et al., 2015) • Learning to Transduce with Unbounded Memory (Grefenstette 2015) • Structured Memory for Neural Turing Machines (Wei Zhang 2015) • End to end memory networks (Sukhbaataret et al., 2015) • A. Moschitti, A. Severyn. 2015. UNITN: Training Deep Convolutional Network for Twitter Sentiment Classification. SemEeval 2015. • M. Henaff, J. Weston, A. Szlam, A. Bordes, Y. LeCun. Tracking the World State with Recurrent Entity Network. arXiv preprint arXiv:1612.03969, 2017. • J. Weston, S. Chopra, A.Bordes. Memory Networks. arXiv preprint arXiv:1410.3916, 2014. • A. Madotto, G. Attardi. Question Dependent Recurrent Entity Network for Question Answering. 2017.