The Role of Prior Knowledge in Human Reconstructive Memory

1. The Role of Prior Knowledge in Human Reconstructive Memory Mark Steyvers Pernille Hemmer University of California, Irvine

2. Interaction between Prior Knowledge & Episodic Memory reconstructive memory Prior Knowledge, Semantic Memory, “Schemas” Episodic Memory “What objects do you remember from your hotel room?” “What coffee did you order from Starbucks last week?”

3. Research questions • Machine learning • How can we formalize prior knowledge (“schemas”)? • Psychology • How do humans integrate memory and prior knowledge? • What errors do people make? • Interface between human & machine learning • Can we explain human error with rational inference principles? • Can we build better information retrieval systems by taking cognitive processes into account?

4. Remembering Objects from a Graduate Office chair desk skull books (30% of subjects) Brewer & Treyens (1981)

5. Experiments • perception: list objects you see in a scene • memory: list objects you remember from a scene • guessing: list objects you might see in a kitchen/office/… Kitchen Dining Hotel

6. Human Data: Precision vs. Output Position 10 Secs. Precision 2 Secs guessing

7. Bayesian Analysis of Reconstructive Memory Posterior What objects were studied given noisy memory contents? Likelihood How likely is this memory content given these objects were studied? Prior How likely are these objects a priori? 7

8. Problem 1: Infer prior knowledge Prior Knowledge ? kitchen2 kitchen3 kitchen1 Hierarchical Beta processes (Thibaux & Jordan, 2007); related to Indian Buffet process

9. Hierarchical Beta Process(Thibaux & Jordan, 2007) “Kitchen” Generalknowledge Object probs. for specific image y Reported objects participants objects Kitchen65 Kitchen45

10. Problem 2: Recall using Prior + Episodic M. Prior Knowledge semantic memory ? Actual objects in image episodic memory Noisy Memory Content

11. Data Model prior only

12. Limitations of Model • Independence assumption between objects • Recall often clusters correlated objects • “computer”  “mouse”  “keyboard”

13. Reconstructing the Style of Written Digits Study Digit space from “Earth Mover” distances personal prior Reconstructed reconstructed studied digit

14. Reconstructing Drawings from Memory Reconstruction Studied Drawing

15. Reconstructing Lists of Words Study this list: PEAS, CARROTS, BEANS, SPINACH, LETTUCE, HAMMER, TOMATOES, CORN, CABBAGE, SQUASH HAMMER, PEAS, CARROTS, ...

16. Conclusion • Prior knowledge has strong effect on memory • Guessing based on prior knowledge leads to quite good performance • Machine learning methods help formalize “schemas” • Useful to understand human memory when designing information retrieval systems

17. Thanks!