Human Frontal Lobes & AI Planning Memex and it’s Decendants

1. Human Frontal Lobes & AI PlanningMemex and it’s Decendants 18 February 2008 Machines of Loving Grace

2. Subcortex: reactive • Sensory cortex: receiving & storing information • Frontal lobes: planning, execution, regulation

4. PEAT • Planning and Execution assistant • Goal management: associate goals with rewards • Plan management: simulate (future execution of) scripts • Execution management: start / stop script execution • Does not handle regulation (yet) • “Scripts”: canned routine plans • Defined by user (or caretaker) • Contain actions, choice points, and calls to other scripts • Plan management = planning = simulating the different ways a script may be performed, in order to find the choices that maximize goal achievement

6. PEAT Status • Small user base • Small clinical trial planned • One problem: user must constantly tell device when each action in a script begins or ends • Recently won DARPA grant to work with UR to use RFID, GPS, and cell phone use sensors to automate execution monitoring • E.g.: no need to remind user to use bathroom if tag on flush handle sensed; no need to remind to go to store if GPS says user is at the store; etc.

7. As We May Think • Vannevar Bush, 1945 • Led Office of Scientific Research & Development during WW II • Precursor to both NSF and DARPA • Science & engineering, including computing had been dedicated to fighting the war • What is the great challenge now? • Expand our minds’ ability to retrieve & organize information, in order to deal with the complexity of the modern world

8. Forces • Growing scientific specialization • Cheap & reliable electronics • Electronic imaging • television had been delayed during WW II

9. Vision • Scientist captures data by simply looking • Rapid & automatic data analysis • Computing for logical reasoning • Electronic commerce – idea of a bank card • Associative memory • Idea behind hypertext & internet • But emphasized personal memory • Memex display

11. Cyborgs • In the outside world, all forms of intelligence, whether of sound or sight, have been reduced to the form of varying currents in an electric circuit in order that they may be transmitted. Inside the human frame exactly the same sort of process occurs. Must we always transform to mechanical movements in order to proceed from one electrical phenomenon to another? It is a suggestive thought, but it hardly warrants prediction without losing touch with reality and immediateness.

12. Discussion Questions • What parts of Bush’s vision are here now? • What might be here in the foreseeable future? • What did Bush get wrong? • Compare PEAT to Memex. • What different kinds of functionality do they provide? • How does each relate to a model of how the mind operates?

13. MyLifeBits • Project led by Gordon Bell at Microsoft Research to achieve the Memex vision • Bell’s data collection 2001-2005:

15. Issues • Data capture • Date organization • Browsing • Targeted retrieval • Why?

18. Claim Suppose you have a damaged memory…

19. SenseCam • Wearable camera developed at Microsoft Research Cambridge as part of MyLifeBits project • Ambient light sensor & accelerometer control rate of picture taking

20. Study

21. Procedure • Conditions • SenseCam / Active (user takes pictures) • SenseCam / Passive (automatic pictures) • Control / Active (user’s partner had SenseCam/Active) • Control / Passive (user’s partner had SenseCam/Passive) • Testing • Initial free recall • View & order images • Final free recall • Repeat for Short (3 days) versus Long (10 days)

27. Summary • Patient with severe memory impairment wore SenseCam • Compared ability to form memories • With no aids • By reviewing SenseCam photos • By reviewing a written diary • Many more memories formed and retained longer using SenseCam