Linguistic Knowledge Representation

1. Linguistic Knowledge Representation Scott Farrar Department of Linguistics farrar@u.arizona.edu

2. Problems to Overcome • Specifying the relationship between linguistic and other forms of knowledge.

3. Inference from Knowledge L-Linguistic CS-Commonsense V-Visual John’s hand is in his pocket. L John owns the hand. CS, V The hand is physically attached to John. L, CS The hand is physically contained in the pocket, not the other way around. V A hand is smaller than a pocket. CS John’s hand is not in Bill’s pocket. CS John’s wants his hand to be in his pocket. L This event is occurring now. CS Hand is a body-part, not a person. CS A pocket is a container in clothing.

4. Problems to Overcome • Specifying the relationship between linguistic and other forms of knowledge. • Dealing with ambiguity and other issues of natural language processing (NLP).

5. What is Meaning? • Symbols, Representation, Extensions [010011101] moon

6. What is Meaning? • Conceptual structure hypothesis: The meaning of a word is the corresponding mental representation in the mind of an agent.

7. The Lexicon links Linguistic Knowledge to the Rest of Cognition Auditory input Vision input Conceptual Structure Syntactic Structure Spatial Structure Haptic input Motor output Jackendoff’s Model

8. The Computational Lexicon • form—what data structures comprise the lexicon? • organization—how are the data structures organized? • content—what information is contained in the data structures?

9. Form of the Lexicon • Features—Katz and Fodor (1963): knowledge is a conjunction of features (monadic predicates) bachelor (x)→ unmarried (x) & male (x) & young (x)

10. Form of the Lexicon • Frames—Minsky (1975): knowledge is organized around concepts give: <agent Person> <recipient Person> <theme PhysicalObject> slots values

11. Form of the Lexicon • Attribute-value matrices (feature structures) From knowledge engineering (AI) community e.g., Head-Driven Phrase Structure Grammar <see example>

12. Organization of the Lexicon • Semantic network—Quillian (1966): knowledge is interconnected has-part animal lungs Sub has-part bird feathers Inst tweety

13. Hierarchy of Concepts Artifact Machine Tool Motorized- Machine Non-motorized- Machine … … automobile drill loom windmill

14. Problems to Overcome • Specifying the relationship between linguistic and other forms of knowledge. • Dealing with ambiguity and other issues of natural language processing (NLP). • Determining what role visual knowledge of objects and events has in the disambiguation process.

15. Formalism for Ambiguity Resolution(9/13/02) Scott Farrar Department of Linguistics farrar@u.arizona.edu

16. Goals of the Present Research • Build a theoretical system that can construct a visual scene from English text input. • focus on the problem of lexical ambiguity • access and use the visual knowledge linked to lexical items • argue for a knowledge-rich approach to natural language processing (lexicon)

17. Lexical Ambiguity When one linguistic form has multiple meanings: The book is on the edge of the table. [area] The edge of the table is sharp. [line] The park is five blocks away. [large dimension] Kids like to play with blocks. [small dimension] The middle (of the bench) is wet. [center-part] Put the pan in the middle (between the bowls). [space-between] To vote “YES” check the upper box. [2d] Put your hand in the box. [3d]

18. Natural Language Processing lexicon “The king gave the people bread.” Grammar semantics syntax The king gave the people bread DT N VB DT N N (the king) (gave) (the people) (bread) other knowledge give: tense: past agent: the king recipient: the people theme: bread The people have bread. The people ate the bread.

19. How much visual knowledge does the lexicon have access to? Type of conceptExample • a-spatial fear, hour, duration • extrinsically animal, robot, spatial instrument • intrinsically horse, man, violin spatial my leg • strictly spatial square, margin, height (Bierwisch 1996: 52)

20. Content of the Lexicon Purely linguistic: dog “dog”, [da:g] dog collar not *collar dog dog+PL = dogs not *doges

21. Content of the Lexicon Purely visual: dog shape: size: color: texture:

22. Content of the Lexicon Commonsense/other/visual: dog has-part (dog, tail) makes-noise (dog, “bark”) disjoint (dog cat) likes (Scott, terrier)

23. Knowledge Components knowledge is distributed yet interoperable L Linguistic CS Commonsense V Visual

24. Formalization of the Problem • input: a list of well-formed English utterances U, where U={u1,u2,u3,…,un}, |U|≥1, and U can be interpreted as a complete visual scene.

25. Examples of U {John is standing on the bridge. John has his hands in his pockets. John is wearing a cap…} {The table is in the middle of the room. There is a ball on the edge of the table…} not {Mary loves John. She has known him for four years…}

26. Formalization of the Problem (cont.) • output: VU, such that VUis a visual scene based on U consisting of a 3-tuple <I, O, R> where I is a set of icons, O is a set of orientations for the icons, and R is a set of relations among the icons.

27. A Solution Approach • Represent all knowledge in pure first-order logic • a knowledge base KB consists of axioms and facts about the domain with no distinctions made between types of knowledge • a forward-chaining algorithm is used to generate a visual scene

28. Formalization of the Lexicon A lexicon L is at least the 4-tuple <F, R, G, C>, where: F is the set of linguistic forms. R is the set of formal relations among members of F. G is the grammatical information relevant to F. C is the conceptual content (the meaning).

29. Commonsense Knowledge • A ball will not remain stationary on an inclined surface. • A jar can be a container. • Unsupported objects fall.

30. Formalization of Commonsense Knowledge • KB is the tuple <C, R, I, O>, where: C is the (possibly infinite) set of concepts. R is set of relations over C. I is the set of individuals. O is an ontology specifying the precise formalization of C, R, and I.

31. Visual Knowledge Concepts: AbstractShapes={Circle, Line, Sphere,…} Relations: SpatialRelations={In, On, Contains,…} Axioms: Peas are smaller than landmines. If object A contacts object B, then A is near B.

32. Formalization of Visual Knowledge Visual knowledge V is a subset of KB.

33. So far so Good • lexicon L = <F, R, G, C> • knowledge base KB = <C, R, I, O> • visual knowledge V ∈ KB * Well understood inferencing procedures for FOL knowledge bases: theorem proving: Prolog, forward-chaining: CLIPS

34. So Far so Intractable * If represented in pure first-order (or higher order) logic, then the problem will eventually become computationally intractable depending on scope of domain and, for NLP, ambiguity of the word in question (compare ‘edge’ to ‘hand’).

35. Alternatives • Use a logical system that is well-understood and known to be complete and tractable Description Logic (Brachman 1979)

36. Description Logic • A KR formalism (like frames, semantic nets, prod. syst.) • A way to build a conceptualization of the domain • Basic structure is the concept (a structured entity) • Intuitively appealing • Incorporates a subset of FOL • Expressive syntax and decidable inference procedures • e.g., KL-ONE (Brachman 1979) KRYPTON (Brachman, Fikes, and Levesque (1983) LOOM, CLASSIC…

37. Description Logic Components • Syntax • The KB • Semantics • Reasoning Procedures • Reasoning Tasks

38. Syntax: Atoms • concepts (unary predicates) • roles (binary predicates) • individuals (constants)

39. Syntax: Concepts • Round • Flat • Long • Hole • Person • Event

40. Syntax: Roles • On • In • Strike • Touch • Neighbor • Father

41. Syntax: Individuals • TABLE-1 • JOHN • MY-HAND • ROLLING-EVENT-3

42. The Knowledge Base of a Description Logic • Terminology (TBox) – hierarchy of concepts and roles • Assertions (ABox) – axioms for individual objects

43. Benefits of Dividing the KB • Reasoning is tractable due to sacrifice of expressiveness (Brachman and Levesque 1985) • Philosophically ‘clean’: TBox (intensional knowledge, always true, doesn’t change, a priori) ABox (extensional knowledge, can change, a posteriori) • Satisfiability of conceptualization (domain) is determined easily when only TBox is considered • Conceptual modeling appears more intuitive

44. Syntax: Constructors • intersection (C  D) : Round  Flat Round  Flat  Light • value restriction (∀ R.C): ∀hasHole.Container • limited existential quantification (∃R.T): ∃hasHole.T

45. Syntax: TBox Axioms • Definitions Ball  Sphere  Toy Box  Container  Cube Biped  Animal  =2hasLegs Definitions are basic operation in TBox for deriving new concepts (other concepts are primitive).

46. Syntax: TBox Axioms (cont.) • subsumption operator: ‘’ • Table  Furniture Human  Animal Sphere  3DShape • Provide structure for TBox

47. Syntax: ABox Assertions • Concept assertions C(a): Person (JOHN) Table (TABLE-1) Tool (HAMMER-23) • Role assertions R(a,b): Likes (JOHN, HAMMER-23) On (HAMMER-23, TABLE-1) • Serves to link individuals in ABox to concepts in TBox

48. Note on T/ABox Relation • TBox imposes selection restrictions on ABox assertions, e.g., Eat(JOHN, TABLE-1) is not possible, because • Edible(Table) is not in TBox

49. Informal Semantics • A concept C is a set of individuals {a,b,…} • A role R is a relation between a pair of individuals or concepts. • A conjoined concept C ⊔ D is both C and D.

50. Reasoning Tasks • TBox categorization, satisfiability • ABox consistency checking w.r.t. TBox