Structured Probabilistic Inference in an Embodied Construction Grammar

1. Structured Probabilistic Inference in an Embodied Construction Grammar Jerome Feldman International Computer Science Institute U. California at Berkeley Berkeley, CA jfeldman@icsi.berkeley.edu Srini Narayanan International Computer Science Institute Berkeley, CA snarayan@icsi.berkeley.edu and

2. Language, Learning and Neural Modeling • Scientific Goal Understand how people learn and use language • Practical Goal Build systems that analyze and produce language • Approach Embodied linguistic theories with advanced biologically-based computational methods

3. State of the Art • Limited Commercial Speech Applications transcription, simple response systems • Statistical NLP for Restricted Tasks tagging, parsing, information retrieval • Template-based Understanding programs expensive, brittle, inflexible, unnatural • Essentially no NLU in QA Systems

4. Hypothesis • NLU is essential to large, open domain QA. • One line dismissals (lack of world knowledge) are out of date (by at least 5 yrs.) • Substantial Progress in Enabling Technologies • Knowledge Representation/Inference Techniques • Active Knowledge • Dealing With Uncertainty. • Simulation Semantics • Scaling Up • CYC, Wordnet, Term-bases • FrameNet, Semantic Web, MetaNet • Extraction of Semantic Relations • Empirical • Linguistic • The goal of NLU can be realized

5. Hypothesis • NLU is essential to large, open domain QA. • One line dismissals (lack of world knowledge) are out of date (by at least 5 yrs.) • Substantial Progress in Enabling Technologies • Knowledge Representation/Inference Techniques • Active Knowledge • Dealing With Uncertainty. • Simulation Semantics • Scaling Up • CYC, Wordnet, Term-bases • FrameNet, Semantic Web, MetaNet • Extraction of Semantic Relations • Empirical • Linguistic • The goal of NLU can be realized, perhaps! Anyway, it is time to try again.

6. Complex questions andsemantic information (SH, UTD) • Complex questions are not characterized only by a question class (e.g. manner questions) • Example: How can a biological weapons program be detected ? • Associated with the pattern “How can X be detected?” • And the topic X = “biological weapons program” • Processing complex questions is also based on access to the semantics of the question topic • The topic is modeled by a set of discriminating relations, e.g. Develop(program); Produce(biological weapons); Acquire(biological weapons) or stockpile(biological weapons) • Such relations are extracted from topic-relevant texts

7. The need for Semantic Inference in QA • Some questions are complex! • Example (from UTD CNS QA database): • What is the evidence that IRAQ has WMD? • Answer: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors.

8. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

9. Answer Structure (continued) ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: possess(Iraq, delivery systems(type : scud missiles; launchers; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium possess(Iraq, fuel stock(purpose: power launchers)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium hide(Iraq, Seeker: UN Inspectors; Hidden: CBW stockpiles & guided missiles) Justification: DETECTION SchemaInspection status: Past; Likelihood: Medium

10. Embodiment Of all of these fields, the learning of languages would be the most impressive, since it is the most human of these activities. This field, however, seems to depend rather too much on the sense organs and locomotion to be feasible. Alan Turing (Intelligent Machines,1948)

11. NTL Manifesto • Basic concepts and words derive their meaning from embodied experience. • Abstract and Theoretical concepts derive their meaning from metaphorical maps to more basic embodied concepts. • Structured Connectionist Models can capture both of these processes nicely.

12. General and Domain Knowledge • Conceptual Knowledge and Inference • Embodied • Language and Domain Independent • Powerful General Inferences • Ubiquitous in Language • Domain Specific Frames and Ontologies • Framenet • Metaphor links domain specific to general • E.g., France slipped into recession.

13. What are Image schemas? • Regularities in our perceptual, motor and cognitive systems • Structure our experiences and interactions with the world. • May be grounded in a specific cognitive system, but are not situation-specific in their application (can apply to many domains of experience)

14. Basis of Image schemas • Perceptual systems • Motor routines • Social Cognition • Image Schema properties depend on • Neural circuits • Interactions with the world

15. Representing image schemas semantic schemaSource-Path-Goal roles: source path goal trajector semantic schemaContainer roles: interior exterior portal boundary Boundary Interior Trajector Portal Source Goal Path Exterior These are abstractions over sensorimotor experiences.

16. Schema Formalism SCHEMA <name> SUBCASE OF <schema> EVOKES <schema> AS <local name> ROLES < self role name>: <role restriction> < self role name> <-> <role name> CONSTRAINTS <role name> <- <value> <role name> <-> <role name> <setting name> :: <role name> <-> <role name> <setting name> :: <predicate> | <predicate>

17. A Simple Example SCHEMA hypotenuse SUBCASE OF line-segment EVOKES right-triangle AS rt ROLES Comment inherited from line-segment CONSTRAINTS SELF <-> rt.long-side

18. Source-Path-Goal SCHEMA: spg ROLES: source: Place path: Directed Curve goal: Place trajector: Entity

19. Translational Motion SCHEMA translational motion SUBCASE OF motion EVOKES spg AS s ROLES mover <-> s.trajector source <-> s.source goal <-> s.goal CONSTRAINTS before:: mover.location <-> source after:: mover.location <-> goal

20. Extending Inferential Capabilities • Given the formalization of the conceptual schemas • How to use them for inferencing? • Earlier pilot systems • Used metaphor and Bayesian belief networks • Successfully construed certain inferences • But don’t scale to large open domains • New approach • Probabilistic relational models • Support an open ontology

21. Frames • Frames are conceptual structures that may be culture specific • Words evoke frames • The word “talk” evokes the Communication frame • The word buy (sell, pay) evoke the Commercial Transaction (CT) frame. • The words journey, set out, schedule, reach etc. evoke the Journey frame. • Frames have roles and constraints like schemas. • CT has roles vendor, goods, money, customer. • Words bind to frames by specifying binding patterns • Buyer binds to Customer, Vendor binds to Seller.

22. The FrameNet Project C Fillmore PI (ICSI) Co-PI’s: S Narayanan (ICSI, SRI)D Jurafsky (U Colorado) J M Gawron (San Diego State U) Staff: C Baker Project Manager B Cronin Programmer C Wooters Database Designer

23. FrameNet in the Larger Context • The long-term goal is to reason about the world in a way that humans understand and agree with. • Such a system requires a knowledge representation that includes the level of frames. • FrameNet can provide such knowledge for a number of domains. • FrameNet representations complement ontologies and lexicons.

24. The core work of FrameNet • characterize frames • find words that fit the frames • develop descriptive terminology • extract sample sentences • annotate selected examples • derive "valence" descriptions

25. The Core Data The basic data on which FrameNet descriptions are based take the form of a collection of annotated sentences, each coded for the combinatorial properties of one word in it. The annotation is done manually, but several steps are computer-assisted.

26. Types of Words / Frames • events • artifacts, built objects • natural kinds, parts and aggregates • terrain features • institutions, belief systems, practices • space, time, location, motion • etc.

27. FrameNet Product • For every target word, • describe the frames or conceptual structures which underlie them, • and annotate example sentences that cover the ways in which information from the associated frames are expressed in these sentences.

28. Q: What kind of materials were stolen from the Russian navy? FS(Q): What [GOODS: kind of nuclear materials] were [Target-Predicate:stolen] [VICTIM: from the Russian Navy]? Applying Frame Structures to QA • Parsing Questions • Parsing Answers • Result: exact answer= “approximately 7 kg of HEU” A(Q): Russia’s Pacific Fleet has also fallen prey to nuclear theft; in 1/96, approximately 7 kg of HEU was reportedly stolen from a naval base in Sovetskaya Gavan. FS(A(Q)): [VICTIM(P1): Russia’s Pacific Fleet] has also fallen prey to [Goods(P1): nuclear ] [Target-Predicate(P1): theft]; in 1/96, [GOODS(P2): approximately 7 kg of HEU] was reportedly [Target-Predicate (P2): stolen] [VICTIM (P2): from a naval base] [SOURCE(P2): in Sovetskawa Gavan]

29. Hypothesis: Linguistic input is converted into a mental simulation based on bodily grounded structures Semantic schemas, including image schemas (Johnson 1987) and executing schemas (Bailey, Narayanan 1997) are abstractions over neurally grounded perceptual and motor representations Linguistic units make reference to these structures in their semantic pole Analysis produces a simulation specification linking these structures and providing parameters for a simulation engine Model requires: schema representations (image, motor, frame, social, etc.) lexicalandphrasal construction representations that invoke those schemas and other cultural frames (FrameNet+ frames) Language understanding via simulation

30. Cafe Simulation-based language understanding Utterance “Harry walked to the cafe.” Constructions Analysis Process General Knowledge Simulation Specification Schema Trajector Goal walk Harry cafe Belief State Simulation

31. Simulation specification • The analysis process produces a simulation specification that • includes image-schematic, motor control and conceptual structures • provides parameters for a mental simulation

32. Simulation Semantics • BASIC ASSUMPTION: SAME REPRESENTATION FOR PLANNING AND SIMULATIVE INFERENCE • Evidence for common mechanisms for recognition and action (mirror neurons) in the F5 area (Rizzolatti et al (1996), Gallese 96, Buccino 2002, Tettamanti 2004) and from motor imagery (Jeannerod 1996) • IMPLEMENTATION: • x-schemas affect each other by enabling, disabling or modifying execution trajectories. Whenever the CONTROLLERschema makes a transition it may set, get, or modify stateleading to triggering or modificationof other x-schemas. State is completely distributed (a graph marking) over the network. • RESULT: INTERPRETATION IS IMAGINATIVE SIMULATION!

33. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reports add up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

34. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reportsadd up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

35. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reportsadd up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. Temporal Reference/Grounding A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

36. Answer Structure (continued) ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reportsadd up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Present Progressive Perfect A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: possess(Iraq, delivery systems(type : scud missiles; launchers; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium Present Progressive Continuing possess(Iraq, fuel stock(purpose: power launchers)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium hide(Iraq, Seeker: UN Inspectors; Hidden: CBW stockpiles & guided missiles) Justification: DETECTION SchemaInspection status: Past; Likelihood: Medium

37. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reportsadd up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. Uncertainty and Belief A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

38. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reportsadd up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. Uncertainty and Belief A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Multiple partly reliable sources Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

39. ANSWER: Evidence-Combined:Pointer to Text Source: A1: In recent months, Milton Leitenberg, an expert on biological weapons, has been looking at this murkiest and most dangerous corner of Saddam Hussein's armory. A2: He says a series of reportsadd up to indications that Iraq may be trying to develop a new viral agent, possibly in underground laboratories at a military complex near Baghdad where Iraqis first chased away inspectors six years ago. Answer Structure A3: A new assessment by the United Nations suggests Iraq still has chemical and biological weapons - as well as the rockets to deliver them to targets in other countries. Event Structure Metaphor A4:The UN document says Iraq may have hidden a number of Scud missiles, as well as launchers and stocks of fuel. A5: US intelligence believes Iraq still has stockpiles of chemical and biological weapons and guided missiles, which it hid from the UN inspectors Content: Biological Weapons Program: develop(Iraq, Viral_Agent(instance_of:new)) Justification: POSSESSION Schema Previous (Intent and Ability): Prevent(ability, Inspection); Inspection terminated; Status: Attempt ongoing Likelihood: Medium Confirmability: difficult, obtuse, hidden possess(Iraq, Chemical and Biological Weapons) Justification: POSSESSION SchemaPrevious (Intent and Ability): Prevent(ability, Inspection); Status: Hidden from Inspectors Likelihood: Medium possess(Iraq, delivery systems(type : rockets; target: other countries)) Justification: POSSESSION SchemaPrevious (Intent and Ability): Hidden from Inspectors; Status: Ongoing Likelihood: Medium

40. Temporal relations in QA • Results of the workshop are accessible from http://www.cs.brandeis.edu/~jamesp/arda/time/documentation/TimeML-use-in-qa-v1.0.pdf • A set of questions that require the extraction of temporal relations was created (TimeML question corpus) • E.g.: • “When did the war between Iran and Iraq end?” • “Who was Secretary of Defense during the Golf War?” • A number of features of these questions were identified and annotated • E.g.: • Number of TEMPEX relations in the question • Volatility of the question (how often does the answer change) • Reference to repetitive events • Number of events mentioned in the question

41. Event Structure for semantically based QA • Reasoning about dynamics • Complex event structure • Multiple stages, interruptions, resources, framing • Evolving events • Conditional events, presuppositions. • Nested temporal and aspectual references • Past, future event references • Metaphoric references • Use of motion domain to describe complex events. • Reasoning with Uncertainty • Combining evidence from multiple, unreliable sources • Non-monotonic inference • Retracting previous assertions • Conditioning on partial evidence • Linguistic Ambiguity • Figurative inference

42. Relevant Previous Work Event Structure Aspect (VDT, TimeML), Situation Calculus (Steedman), Frame Semantics (Fillmore), Cognitive Linguistics (Langacker, Talmy, Lakoff, Sweetser), Metaphor and Aspect (Narayanan) Reasoning about Uncertainty Bayes Nets (Pearl), Probabilistic Relational Models (Pfeffer, Koller), Graphical Models (Jordan), First Order Probabilistic Inference (Poole, Braz et al) Reasoning about Dynamics Dynamic Bayes Nets (Murphy, Friedman), Distributed Systems (Alur, Meseguer), Control Theory (Ramadge and Wonham), Causality (Pearl)

43. walker at goal energy walker=Harry goal=home Active representations • Many inferences about actions derive from what we know about executing them • Representation based on stochastic Petri nets captures dynamic, parameterized nature of actions • Walking: • bound to a specific walker with a direction or goal • consumes resources (e.g., energy) • may have termination condition(e.g., walker at goal) • ongoing, iterative action

44. X-Schema Extensions to Petri Nets • Parameterization • x-schemas take parameter values (speed, force) • Walk(speed = slow, dest = store1) • Dynamic Binding • X-schemas allow run-time binding to different objects/entities • Grasp(cup1), push(cart1) • Hierarchical control and durative transitions • Walk is composed of steps which are composed of stance and swing phases • Stochasticity and Inhibition • Uncertainties in world evolution and in action selection

45. Event Structure in Language • Fine-grained • Rich Notion of Contingency Relationships. • Phenomena: Aspect, Tense, Force-dynamics, Modals, Counterfactuals • Event Structure Metaphor: • Phenomena: Abstract Actions are conceptualized in Motion and Manipulation terms. • Schematic Inferences are preserved.

46. Aspect • Aspect is the name given to the ways languages describe the structure of events using a variety of lexical and grammatical devices. • Viewpoints • is walking, walk • Phases of events • Starting to walk, walking, finish walking • Inherent Aspect • run vs cough vs. rub • Composition with • Temporal modifiers, tense.. • Noun Phrases (count vs. mass) etc..

47. Phases, Viewpoints, and Aspects • John is walking to the store. • John is about to walk to the store. • John walked to the store. • John started walking to the store. • John is starting to walk to the store. • John has walked to the store. • John has started to walk to the store. • John is about to start walking to the store. • John resumed walking to the store. • John has been walking to the store. • John has finished walking to the store. • John almost walked to the store.

49. Phasal Aspect Maps to the Controller Iterative (repeat) Inceptive (start, begin) Iterate Ready Start Process Finish Done interrupt resume Cancel Suspend Completive (finish, end) Resumptive(resume)

50. Embedding: The end of the beginning Ready Start Process Finish Done resume interrupt Suspend R S P F D r i S C X-Schema for X with bindings