- 106 Views
- Uploaded on
- Presentation posted in: General

Qualitative Spatial Reasoning in Interpreting Text and Narrative

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Qualitative Spatial Reasoning in Interpreting Text and Narrative

Ernest Davis

New York University

COSIT 2011

Simple natural language texts and narratives often raise problems in commonsense spatial knowledge and reasoning of surprising logical complexity and geometric richness.

Examples: 5 literary texts, 6 (or 9) contrived texts

- Fitting things into containers
- Blocking, pursuing, escaping, and hiding
- Household objects
Summary

Morals for research

Usage of “aorta”

“Aorta have more busses. An aorta mikem smaller so they don’t take up half the road. An aorta put more seats innem so you doan tefter stann all the time. An aorta have more room innem --- you carn tardly move innem air so crowded. Aorta do something about it.”

Forward inference to notice contradiction.

Infer: The speaker is a fool.

(Common in actual narratives, but unusual in inference systems).

AreaOf(Bus) =

NumberOfSeatsIn(Bus) * AreaOf(Seat)

Infer:

Area(Seat)↑ ∧ NumberOfSeatsIn(Bus)↑ ⇒ Area(Bus)↑

The region occupied by a bus is (to simplify) the union of the regions occupied by the seats.

Two seats on a bus occupy disjoint regions.

The area of a collection of disjoint regions is the sum of the area of each region.

If

- each seat in bus b1 corresponds to a seat in bus b2;
- each seat in b1 is smaller than the corresponding seat in b2; and
- there are more seats in b2 than b1
then b2 must be bigger than b1.

Bus(b) ⇒

PlaceOf(b) = ∪sSeatsOf(b)PlaceOf(s).

Bus(b) ∧ s1≠ s2 ∧ s1,s2 SeatsOf(b) ⇒ DR(PlaceOf(s1),PlaceOf(s2)).

CollectionOfRegions(c) ∧

[∀r1,r2c r1 ≠ r2 ⇒ DR(r1,r2)] ⇒

AreaOf(∪r c r) = ∑ r c AreaOf(r).

Infer:

Bus(b1) ∧ Bus(b2) ∧ Injection(m,SeatsOf(b1),SeatsOf(b2)) ∧ [∀sSeatsOf(b1) AreaOf(PlaceOf(s)) <

AreaOf(PlaceOf(Apply(m,s)))]

⇒

AreaOf(PlaceOf(b2)) > AreaOf(PlaceOf(b1)).

The logic is just a convenient notation.

I am not saying that a reasoning program has to use a logical representation.

I am saying that the reasoning here will pretty much have to carry out this inference.

Infer:

Bus(b1) ∧ Bus(b2) ∧ Injection(m,SeatsOf(b1),SeatsOf(b2)) ∧ [∀sSeatsOf(b1) AreaOf(PlaceOf(s)) <

AreaOf(PlaceOf(Apply(m,s)))]

⇒

AreaOf(PlaceOf(b2)) > AreaOf(PlaceOf(b1)).

Representational issues:

Sets of regions.

Interaction of measure (Area) and mereology (DR, Union).

Winograd schema: A pair of sentences, differing in one or two words, with an ambiguity that is resolved oppositely.

Winograd (1972):

“The city councilmen refused the demonstrators a permit because they feared violence”

vs.“… because they advocated violence.”

Collecting corpus of schemas that

- are effortlessly disambiguated by human readers
- sound natural.
- can’t be solved using selectional restrictions
- are not easily Googlable.
Challenge for AI. Less far-reaching than the Turing Test, but less problematic.

“The trophy does not fit in the suitcase because it is too large.”

“The trophy does not fit in the suitcase because it is too small.”

Gloss “The trophy is too large to fit’’ as “The trophy does not fit and no larger object fits, but some smaller object fits’’

Incorrect reading: The suitcase is too large.

∼FitsIn(Trophy,Suitcase) ∧

[∀s1s1⊃ Suitcase ⇒∼FitsIn(Trophy,s1)] ∧

[∃s2 s2⊂Suitcase ∧ FitsIn(Trophy,s2)].

Infer that this is impossible.

∼FitsIn(Trophy,Suitcase) ∧

[∀s1s1⊃ Suitcase ⇒∼FitsIn(Trophy,s1)] ∧

[∃s2 s2⊂Suitcase ∧ FitsIn(Trophy,s2)].

Interaction of subset with fit.

Existential quantification over regions.

“Each of my books fits in this box, but they do not all fit in the box. If I had more boxes like this, I could fit them all.”

∀a,cArrangement(a,c) ≡

[∀xcFeasibleShape(PlaceOf(x,a),x)] ∧

[∀x1,x2cx1≠x2 ⇒

DR(PlaceOf(x1,a),PlaceOf(x2,a))]

M=My books. X = the inside of the box.

∀bM∃aArrangement(a,{b}) ∧

PlaceOf(b,a)⊂X.

∼∃aArrangement(a, M) ∧

∀b M PlaceOf(b,a)⊂X

∃c,a [∀y c Congruent(y,X)] ∧

Arrangement(a,M) ∧

∀bM ∃ y c PlaceOf(b,a)⊂y.

∃c,a [∀y c Congruent(y,X)] ∧

RigidArrangement(a,M) ∧

∀bM ∃ y c PlaceOf(b,a)⊂y.

- Arrangement of a collection of objects.
- 3-level quantifier alternation.
(A natural formulation in terms of the actual boxes has a 5-level quantifier alternation.)

“The equipment came out of the box, but now I can’t fit it into the box.”

A common experience.

Geometrically impossible.

Geometric aspects.

A reasoner that knows more about geometry than people might interpret “it” as the box.

“I tried to keep the dog out of the kitchen by putting a chair in the middle of the doorway, but it was too wide.”

What was too wide?

Why in the middle?

How did the dog get into the kitchen?

Spatio-temporal reasoning.

∀h Fixed(h,Frame,PosF) ∧

Fixed(h,Chair,PosC) ∧

Feasible(h) ∧

DR(PlaceOf(Dog,Start(h)),Kitchen) ⇒

DR(PlaceOf(Dog,End(h)),Kitchen)

Feasible(h) ≡

∀x,y,s Solid(x) ∧ Solid(y) ∧ x≠y ∧ State(s,h) ⇒

DR(PlaceOf(x,s),PlaceOf(y,s))

∼[∃pc

∀h Fixed(h,Frame,PosF) ∧

Fixed(h,Chair,pc) ∧

Feasible(h) ∧

DR(PlaceOf(Dog,Start(h)),Kitchen) ⇒

DR(PlaceOf(Dog,End(h)),Kitchen)]

I put the chair in the center of the doorframe, because I know that, for any chair, if any placement blocks the dog, then there is a central placement that blocks the dog.

Spatio-temporal reasoning.

Alternating quantification over histories and object positions.

The cat does not initially see the rabbits, because they are behind her head.

The cat cannot see the rabbits under the basket.

No light can penetrate under the basket.

The rabbits cannot get out without lifting the basket, which would disturb the cat.

Many dogs, they say, are the death of a hare, a single dog cannot achieve it, even one much speedier and more enduring than Bashan. The hare can “double” and Bashan cannot --- and that is all there is to it. … The hare gives a quick, easy, almost malicious twitch at right angles to the course and Bashan shoots past from his rear … Before he can stop, turn around, and get going in the other direction, the hare has gained so much ground that it is out of sight.

- Before the double, Bashan and the hare are running in the same direction.
- Bashan is close behind the hare (a second or two)
- The distance of the “double” is at least comparable to the width of Bashan, but not large enough that jumping forward that distance would allow the hare to escape.

- The dog is faster than the hare, so could catch it if the hare didn’t double.
- The hare can change velocity discontinuously.
- The dog has a limit on the rate of deceleration.
- A pack of dogs can catch the hare by surrounding it, so doubling does not help.
The reader can visualize the scenario.

- Differential games
- Properties of the second derivative.

- No end of hard problems.
- Hard to separate physics from geometry
- The physics is often not easy.

“I tried to put the button through the hole, but it was too [large/small].”

“I tried to put the button through the hole, but it had been sewn too tightly to the coat.”

“I forgot that the top button was fastened, so when I took off the coat, it tore off.”

* “I forgot that the top button was unfastened …”

“[Peggotty] gave [my head] a good squeeze. I know it was a good squeeze because, being very plump, whenever she made any little exertion after she was dressed, some of the buttons on the back of her gown flew off. And I recollect two bursting to the opposite side of the parlour while she was hugging me.”

- The gown is too tight.
- The burst buttons are behind Peggotty and in front of David.
- The gown will now gap in back until the buttons are fixed.

- Characterize the position of a fastened button – doable.
- Characterizing the motion involved in fastening a button – difficult. (“Too small” refers to the action, not to the static state.)
- Sufficient characterization of the physics of fabric (“The gown is too tight”) – difficult.

“The power cord on the laptop would not reach from the desk to the outlet, so I got an extension cord. Then my wife objected to having an electric cord across the center of the living room (‘Men!’), so I laid it around the edge of the room, and hid it behind the furniture, but of course I had to get a much longer cord.”

Note the disambiguation of “it”, and the time sequence of “laid” and “had to get.”

Cord(c) ∧ FeasibleShape(r,c) ⇒

LengthOf(AxisOf(r)) = CordLength(c)

∃p FeasibleShape(p,C1) ^ Through(p,LivingRoom) ^ At(End1(p),Table) ^ At(End2(p),Outlet)

- Reasoning about arc lengths
- Quantifying over paths

[H]e set his foot against the stack of chimneys, fastened one end of the rope tightly and firmly around it, and with the other made a strong running noose … He could let himself down by the cord to within a less distance of the ground than his own height, and had his knife ready in his hand to cut it then and drop.

At the very instant when he brought the loop over his head previous to slipping it beneath his arm-pits … he lost his balance …

What is the cause of Sykes’ death?

What would have happened if he had fallen

- Before attaching the rope to the chimneys?
- Before slipping the rope over his head?
- After getting the rope under his armpits?
If the rope had been much shorter? 10 feet longer?

If things had gone right, what would have been the final state of the rope?

Advantages:

- Natural, interesting
- Undeniable qualitative information
- Wide range of
- Spatial relations
- Logical forms
- Directions of inference
Less susceptible to “Use simulation!” and “Use ML!” than other applications

- Finding or contriving texts takes work.
- Does not yield a well-defined class of problems.
- Some important spatial characteristics e.g. shape are entirely implicit.
- Hard to separate spatial from other issues
- Often hard to be sure of logical form
- Difficult to connect logical form to text

WARNING!

The following should not be taken as career advice, particularly if you are pursuing:

- A degree
- Tenure
- Funding (at least US)

- QSR research based on close reading of one or few texts.
- Look at small, logically complex, heterogeneous problems.
- We don’t know how to evaluate these, empirically or theoretically. Table the issue of evaluation.

- Study common, geometrically complex, relations e.g. Blocks(c,s,x,k)
The collection of objects c as placed in state s blocks object x from entering region k.

- Problem: These appear in many different variants.
- Problem: What inferences?
- Study common higher-level operators: “Too φ’’.

- Get help from linguists and cognitive psychologists.
- Perhaps attempt a taxonomy of the QSR problems in text.

At this point, automated commonsense reasoning is high-risk, high-payoff, like fusion reactors or SETI.

More fun than either of those, though.