Qualitative spatial reasoning in interpreting text and narrative
Download
1 / 52

Qualitative Spatial Reasoning in Interpreting Text and Narrative - PowerPoint PPT Presentation


  • 143 Views
  • Uploaded on

Qualitative Spatial Reasoning in Interpreting Text and Narrative. Ernest Davis New York University COSIT 2011. Thesis. Simple natural language texts and narratives often raise problems in commonsense spatial knowledge and reasoning of surprising logical complexity and geometric richness.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Qualitative Spatial Reasoning in Interpreting Text and Narrative' - fola


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Qualitative spatial reasoning in interpreting text and narrative

Qualitative Spatial Reasoning in Interpreting Text and Narrative

Ernest Davis

New York University

COSIT 2011


Thesis
Thesis Narrative

Simple natural language texts and narratives often raise problems in commonsense spatial knowledge and reasoning of surprising logical complexity and geometric richness.


Outline
Outline Narrative

Examples: 5 literary texts, 6 (or 9) contrived texts

  • Fitting things into containers

  • Blocking, pursuing, escaping, and hiding

  • Household objects

    Summary

    Morals for research


Let stalk strine afferbeck lauder
Let Stalk Strine NarrativeAfferbeck Lauder

Usage of “aorta”

“Aorta have more busses. An aorta mikem smaller so they don’t take up half the road. An aorta put more seats innem so you doan tefter stann all the time. An aorta have more room innem --- you carn tardly move innem air so crowded. Aorta do something about it.”


Unusual logical form
Unusual logical form Narrative

Forward inference to notice contradiction.

Infer: The speaker is a fool.

(Common in actual narratives, but unusual in inference systems).


Simple representation
Simple representation Narrative

AreaOf(Bus) =

NumberOfSeatsIn(Bus) * AreaOf(Seat)

Infer:

Area(Seat)↑ ∧ NumberOfSeatsIn(Bus)↑ ⇒ Area(Bus)↑


More general and deeper argument
More general and deeper argument Narrative

The region occupied by a bus is (to simplify) the union of the regions occupied by the seats.

Two seats on a bus occupy disjoint regions.

The area of a collection of disjoint regions is the sum of the area of each region.


Infer
Infer Narrative

If

  • each seat in bus b1 corresponds to a seat in bus b2;

  • each seat in b1 is smaller than the corresponding seat in b2; and

  • there are more seats in b2 than b1

    then b2 must be bigger than b1.


Expressive representation given
Expressive Representation. NarrativeGiven:

Bus(b) ⇒

PlaceOf(b) = ∪sSeatsOf(b)PlaceOf(s).

Bus(b) ∧ s1≠ s2 ∧ s1,s2  SeatsOf(b) ⇒ DR(PlaceOf(s1),PlaceOf(s2)).

CollectionOfRegions(c) ∧

[∀r1,r2c r1 ≠ r2 ⇒ DR(r1,r2)] ⇒

AreaOf(∪r  c r) = ∑ r  c AreaOf(r).


Infer: Narrative

Bus(b1) ∧ Bus(b2) ∧ Injection(m,SeatsOf(b1),SeatsOf(b2)) ∧ [∀sSeatsOf(b1) AreaOf(PlaceOf(s)) <

AreaOf(PlaceOf(Apply(m,s)))]

AreaOf(PlaceOf(b2)) > AreaOf(PlaceOf(b1)).


Disclaimer
Disclaimer Narrative

The logic is just a convenient notation.

I am not saying that a reasoning program has to use a logical representation.

I am saying that the reasoning here will pretty much have to carry out this inference.


Infer: Narrative

Bus(b1) ∧ Bus(b2) ∧ Injection(m,SeatsOf(b1),SeatsOf(b2)) ∧ [∀sSeatsOf(b1) AreaOf(PlaceOf(s)) <

AreaOf(PlaceOf(Apply(m,s)))]

AreaOf(PlaceOf(b2)) > AreaOf(PlaceOf(b1)).

Representational issues:

Sets of regions.

Interaction of measure (Area) and mereology (DR, Union).


The winograd schema challenge hector levesque commonsense 2011
The Winograd Schema Challenge NarrativeHector LevesqueCommonsense 2011

Winograd schema: A pair of sentences, differing in one or two words, with an ambiguity that is resolved oppositely.

Winograd (1972):

“The city councilmen refused the demonstrators a permit because they feared violence”

vs.“… because they advocated violence.”


Winograd schema challenge
Winograd Schema Challenge Narrative

Collecting corpus of schemas that

  • are effortlessly disambiguated by human readers

  • sound natural.

  • can’t be solved using selectional restrictions

  • are not easily Googlable.

    Challenge for AI. Less far-reaching than the Turing Test, but less problematic.


The winograd schema challenge hector levesque commonsense 20111
The Winograd Schema Challenge NarrativeHector LevesqueCommonsense 2011

“The trophy does not fit in the suitcase because it is too large.”

“The trophy does not fit in the suitcase because it is too small.”

Gloss “The trophy is too large to fit’’ as “The trophy does not fit and no larger object fits, but some smaller object fits’’


Rule out incorrect interpretation for disambiguation
Rule out incorrect interpretation for disambiguation Narrative

Incorrect reading: The suitcase is too large.

∼FitsIn(Trophy,Suitcase) ∧

[∀s1s1⊃ Suitcase ⇒∼FitsIn(Trophy,s1)] ∧

[∃s2 s2⊂Suitcase ∧ FitsIn(Trophy,s2)].

Infer that this is impossible.


Representational issues
Representational issues Narrative

∼FitsIn(Trophy,Suitcase) ∧

[∀s1s1⊃ Suitcase ⇒∼FitsIn(Trophy,s1)] ∧

[∃s2 s2⊂Suitcase ∧ FitsIn(Trophy,s2)].

Interaction of subset with fit.

Existential quantification over regions.


Books in boxes
Books in boxes Narrative

“Each of my books fits in this box, but they do not all fit in the box. If I had more boxes like this, I could fit them all.”

∀a,cArrangement(a,c) ≡

[∀xcFeasibleShape(PlaceOf(x,a),x)] ∧

[∀x1,x2cx1≠x2 ⇒

DR(PlaceOf(x1,a),PlaceOf(x2,a))]


M=My books. X = the inside of the box. Narrative

∀bM∃aArrangement(a,{b}) ∧

PlaceOf(b,a)⊂X.

∼∃aArrangement(a, M) ∧

∀b M PlaceOf(b,a)⊂X

∃c,a [∀y c Congruent(y,X)] ∧

Arrangement(a,M) ∧

∀bM ∃ y c PlaceOf(b,a)⊂y.


Representational issues1
Representational issues Narrative

∃c,a [∀y c Congruent(y,X)] ∧

RigidArrangement(a,M) ∧

∀bM ∃ y c PlaceOf(b,a)⊂y.

  • Arrangement of a collection of objects.

  • 3-level quantifier alternation.

    (A natural formulation in terms of the actual boxes has a 5-level quantifier alternation.)


A representational challenge
A Representational Challenge Narrative

“The equipment came out of the box, but now I can’t fit it into the box.”

A common experience.

Geometrically impossible.

Geometric aspects.

A reasoner that knows more about geometry than people might interpret “it” as the box.


Dog out of the kitchen
Dog out of the Kitchen Narrative

“I tried to keep the dog out of the kitchen by putting a chair in the middle of the doorway, but it was too wide.”

What was too wide?

Why in the middle?

How did the dog get into the kitchen?

Spatio-temporal reasoning.


The chair blocks the dog
The chair blocks the dog Narrative

∀h Fixed(h,Frame,PosF) ∧

Fixed(h,Chair,PosC) ∧

Feasible(h) ∧

DR(PlaceOf(Dog,Start(h)),Kitchen) ⇒

DR(PlaceOf(Dog,End(h)),Kitchen)

Feasible(h) ≡

∀x,y,s Solid(x) ∧ Solid(y) ∧ x≠y ∧ State(s,h) ⇒

DR(PlaceOf(x,s),PlaceOf(y,s))


No placement of the chair blocks the dog
No placement of the chair blocks the dog Narrative

∼[∃pc

∀h Fixed(h,Frame,PosF) ∧

Fixed(h,Chair,pc) ∧

Feasible(h) ∧

DR(PlaceOf(Dog,Start(h)),Kitchen) ⇒

DR(PlaceOf(Dog,End(h)),Kitchen)]


Central placement
Central placement Narrative

I put the chair in the center of the doorframe, because I know that, for any chair, if any placement blocks the dog, then there is a central placement that blocks the dog.


Representational issues2
Representational issues Narrative

Spatio-temporal reasoning.

Alternating quantification over histories and object positions.


The tale of benjamin bunny beatrix potter
The Tale of Benjamin Bunny NarrativeBeatrix Potter


Spatial inferences for narrative coherence
Spatial Inferences for Narrative Coherence Narrative

The cat does not initially see the rabbits, because they are behind her head.

The cat cannot see the rabbits under the basket.

No light can penetrate under the basket.

The rabbits cannot get out without lifting the basket, which would disturb the cat.


A man and his dog thomas mann
“A Man and his Dog” NarrativeThomas Mann

Many dogs, they say, are the death of a hare, a single dog cannot achieve it, even one much speedier and more enduring than Bashan. The hare can “double” and Bashan cannot --- and that is all there is to it. … The hare gives a quick, easy, almost malicious twitch at right angles to the course and Bashan shoots past from his rear … Before he can stop, turn around, and get going in the other direction, the hare has gained so much ground that it is out of sight.


Inferences
Inferences Narrative

  • Before the double, Bashan and the hare are running in the same direction.

  • Bashan is close behind the hare (a second or two)

  • The distance of the “double” is at least comparable to the width of Bashan, but not large enough that jumping forward that distance would allow the hare to escape.


  • The dog is faster than the hare, so could catch it if the hare didn’t double.

  • The hare can change velocity discontinuously.

  • The dog has a limit on the rate of deceleration.

  • A pack of dogs can catch the hare by surrounding it, so doubling does not help.

    The reader can visualize the scenario.


Representational issues3
Representational issues hare didn’t double.

  • Differential games

  • Properties of the second derivative.


Household objects buttons and cord
Household Objects hare didn’t double.Buttons and Cord

  • No end of hard problems.

  • Hard to separate physics from geometry

  • The physics is often not easy.


Buttons
Buttons hare didn’t double.

“I tried to put the button through the hole, but it was too [large/small].”

“I tried to put the button through the hole, but it had been sewn too tightly to the coat.”

“I forgot that the top button was fastened, so when I took off the coat, it tore off.”

* “I forgot that the top button was unfastened …”


David copperfield charles dickens
David Copperfield, hare didn’t double.Charles Dickens

“[Peggotty] gave [my head] a good squeeze. I know it was a good squeeze because, being very plump, whenever she made any little exertion after she was dressed, some of the buttons on the back of her gown flew off. And I recollect two bursting to the opposite side of the parlour while she was hugging me.”


Inferences1
Inferences hare didn’t double.

  • The gown is too tight.

  • The burst buttons are behind Peggotty and in front of David.

  • The gown will now gap in back until the buttons are fixed.


Difficulties
Difficulties hare didn’t double.

  • Characterize the position of a fastened button – doable.

  • Characterizing the motion involved in fastening a button – difficult. (“Too small” refers to the action, not to the static state.)

  • Sufficient characterization of the physics of fabric (“The gown is too tight”) – difficult.


Cord hare didn’t double.

“The power cord on the laptop would not reach from the desk to the outlet, so I got an extension cord. Then my wife objected to having an electric cord across the center of the living room (‘Men!’), so I laid it around the edge of the room, and hid it behind the furniture, but of course I had to get a much longer cord.”

Note the disambiguation of “it”, and the time sequence of “laid” and “had to get.”


Representational issues4
Representational Issues hare didn’t double.

Cord(c) ∧ FeasibleShape(r,c) ⇒

LengthOf(AxisOf(r)) = CordLength(c)

∃p FeasibleShape(p,C1) ^ Through(p,LivingRoom) ^ At(End1(p),Table) ^ At(End2(p),Outlet)

  • Reasoning about arc lengths

  • Quantifying over paths


Death of bill sykes oliver twist charles dickens
Death of Bill Sykes hare didn’t double.Oliver Twist, Charles Dickens

[H]e set his foot against the stack of chimneys, fastened one end of the rope tightly and firmly around it, and with the other made a strong running noose … He could let himself down by the cord to within a less distance of the ground than his own height, and had his knife ready in his hand to cut it then and drop.

At the very instant when he brought the loop over his head previous to slipping it beneath his arm-pits … he lost his balance …


Inferences2
Inferences hare didn’t double.

What is the cause of Sykes’ death?

What would have happened if he had fallen

  • Before attaching the rope to the chimneys?

  • Before slipping the rope over his head?

  • After getting the rope under his armpits?

    If the rope had been much shorter? 10 feet longer?

    If things had gone right, what would have been the final state of the rope?


Summary text as a source for qualitative reasoning problems
Summary:Text as a source for qualitative reasoning problems hare didn’t double.

Advantages:

  • Natural, interesting

  • Undeniable qualitative information

  • Wide range of

    • Spatial relations

    • Logical forms

    • Directions of inference

      Less susceptible to “Use simulation!” and “Use ML!” than other applications


Disadvantages
Disadvantages hare didn’t double.

  • Finding or contriving texts takes work.

  • Does not yield a well-defined class of problems.

  • Some important spatial characteristics e.g. shape are entirely implicit.

  • Hard to separate spatial from other issues

  • Often hard to be sure of logical form

  • Difficult to connect logical form to text


Morals for research
Morals for research hare didn’t double.

WARNING!

The following should not be taken as career advice, particularly if you are pursuing:

  • A degree

  • Tenure

  • Funding (at least US)


Morals for research1
Morals for research hare didn’t double.

  • QSR research based on close reading of one or few texts.

  • Look at small, logically complex, heterogeneous problems.

  • We don’t know how to evaluate these, empirically or theoretically. Table the issue of evaluation.


More morals
More morals hare didn’t double.

  • Study common, geometrically complex, relations e.g. Blocks(c,s,x,k)

    The collection of objects c as placed in state s blocks object x from entering region k.

  • Problem: These appear in many different variants.

  • Problem: What inferences?

  • Study common higher-level operators: “Too φ’’.


More morals1
More morals hare didn’t double.

  • Get help from linguists and cognitive psychologists.

  • Perhaps attempt a taxonomy of the QSR problems in text.


Final remark
Final remark hare didn’t double.

At this point, automated commonsense reasoning is high-risk, high-payoff, like fusion reactors or SETI.

More fun than either of those, though.


ad