Context Machine

1. Context Machine Perry McDowell Michael Capps Mike Zyda Andrzej Kapolka

2. Outline • The Context Machine • Literary Inspirations • Payoff and Impact • Goals and Objectives Technical Approach • Accomplishments • Demo • Future Work

3. Literary Inspirations • Diamond Age, Neal Stephenson • Futuristic novel of a young girl who discovers an interactive book which plays scenes demonstrating proper behavior of a young lady • She is able to drastically change her life by adhering to lessons learned in the book

4. Literary Inspirations • Starship Troopers, Robert A. Heinlein • Futuristic novel of a young man who joins the Mobile Infantry and fights in a intergalatic war • The soldiers wear an automated suit which greatly enhances their lethality and survivability • Goal is to give soldiers the same improvement mentally that the suits gave physically

5. The Context Machine • We are building a Context Machine that determines the “context” of a situation for the user and provides advice/guidance. • We anticipate that this augmentation will become essential to future soldiers.

6. How the Context Machine Helps Users • The context machine is envisioned to improve flow of information to users in several ways: • Informing users of things they can not observe • Informing and filtering for users things they don’t have the time to observe • Explaining situations and devices users can’t understand • Providing an perceptive interface for the user

7. Visual Display Aural Display Actions People Determine Context and COA Outer World Abstraction Output to User Goal 3 Goal 2 Goal 4 Goal 1 Goal n Location Objects Preferences User’s Commands Where Does the Context Machine Fit in AugCog?

8. Payoff/Impact • A mechanism for accepting symbolic information from a sensory stream and inserting it into a robust dynamic architecture for determining and representing context • Provides expert advice in limited domains • Capable of learning new and novel situations • Monitors user’s goals and assists where appropriate • Adaptable to individual users

9. Game Engine World Context Machine Game Engine Display How to Build the Middle Without the Ends • We used a game engine to create a world where we could test the Context Machine’s ability to determine context • The engine simulates the system that extracts data from the real world and puts it into the symbolic information

10. And Now, a Word From My Sponsor… • MOVES Institute has a new War Games Lab • Uses game technology to rapidly build high fidelity simulations • 20 artists and programmers from the game industry • Plus plenty of cheap graduate student labor • Work closely with numerous game companies • Unlimited license for Epic Games’ UNREAL engine • Currently building products or demonstrations for US Army, SOCOM, DTRA, DOE, ONR, NRO, and others • Vague website at http://www.wargameslab.com/

11. Technical Approach • We are pursuing a concurrent and intertwined two-tracked approach: • Track 1 involves developing hypothesis of context • Literature search of current research into context • Develop a taxonomy of context elements • Define “necessary and sufficient” for creating context-aware applications • Track 2 involves creating context-aware applications to test our hypotheses • These applications begin simply in specific domains and increase in complexity and generality

12. What We Have Accomplished • Wrote a technical report which consists of an in-depth literature review of the research previously done on context-aware systems. • Summarizes the major efforts in context-aware computing to date • Presents both the definitions of context and the attributes used to determine context • Determines a taxonomy to be used in future investigations into context

13. What We Have Accomplished • Created a simulation demonstrating assistance furnished by a Context Machine • Acts as a base for our later investigations • Demonstrated rudimentary context sensing capability • Demonstrated ways the Context Machine can increase cognitive throughput • Serves as a test platform to test techniques to improve the user’s cognition

14. Specific Accomplishments • Created a pathing system which can determine the best path for a user to follow • Created an imperfect mental model to reflect the facts known by the Context Machine • Use that information to determine context and advise user of appropriate course of action

15. Demo • The goal of the simulation is to transit a treeline to the left of the camp covertly, go to the roof of a building, and observe and record a convoy passing down the road to the right.

16. What the Demo Shows • Directional guidance • Arrows provide directional guidance • Intend to add other information to arrows • Improving recognition of important objects • Augmented reality cursor points out important information that might otherwise not be noticed • Displays information based upon Mental Model • Bar indicates degree of certainty as to the identification of an object/person in the Context Machine’s mental model

17. What the Demo Shows (cont) • Remembers information • After identifying objects, the Context Machine remembers them so user can concentrate on other things • Finds paths based upon current situation • Pathing can be based on a wide variety of criteria

18. Contact Information Perry McDowell The MOVES Institute Naval Postgraduate School 833 Dyer Road, Code CS/mp Monterey, CA 93943-5118 (831) 656-4075 mcdowell@nps.navy.mil http://www.movesinstitute.org/~augcog/