slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
How do we process text with spatial information? PowerPoint Presentation
Download Presentation
How do we process text with spatial information?

Loading in 2 Seconds...

play fullscreen
1 / 31

How do we process text with spatial information? - PowerPoint PPT Presentation


  • 120 Views
  • Uploaded on

How do we process text with spatial information?. Marijn E. Struiksma*, Matthijs L. Noordzij**, Bas F.W. Neggers*** & Albert Postma* *Universiteit Utrecht **Radboud Universiteit Nijmegen *** Universitair Medisch Centrum Utrecht. Spatial Language.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'How do we process text with spatial information?' - dotty


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

How do we process text with spatial information?

Marijn E. Struiksma*, Matthijs L. Noordzij**,

Bas F.W. Neggers*** & Albert Postma*

*Universiteit Utrecht

**Radboud Universiteit Nijmegen

*** Universitair Medisch Centrum Utrecht

spatial language
Spatial Language
  • Spatial language: spatial configuration of the world
    • Simple sentences
      • Search directions, identifying people
    • Complex route finding instructions
  • Generate mental map
    • Also blind people
slide3

Spatial Language

  • Aim: gain insight in underlying process and nature of mental representations
    • Research with sighted and blind
    • Different input modalities
slide4

Spatial Language Processing

  • Propositional model
    • Verbal strategy
  • Strategic model
    • Verbal strategy

or

    • Visuo-spatial strategy
slide5

Different Strategies

  • Noordzij et al. (2005)
  • Sentence-sentence
  • Sentence-picture
  • 80% expectancy
slide6

triangle left of circle

80%

20%

Different Strategies

+

triangle

left of

circle

+

Spatial condition

slide7

triangle and circle

20%

80%

Different Strategies

+

triangle

and

circle

+

Non-spatial condition

different strategies
Different Strategies

Fig. 1 from Noordzij et al. (2005)

  • Spatial: slower on unexpected pictures
  • Propositional model: unexpected stimuli – general switch cost Identical for spatial and non-spatial
  • Strategic model: unexpected stimuli – general switch cost and incompatibility for both sentence and picture
slide9

Dual-representational model

  • Automatic propositional representation
  • Additional visual-spatial representation
    • Strategically dependent on context
  • Neuroimaging parietal areas:
    • Understanding spatial terms
    • Visual-spatial representation
slide10

EEG: processing spatial sentences

  • Noordzij et al. 2006
  • Similar sentence-sentence and sentence-picture paradigm
  • Similar behavioral pattern
  • Event-related potentials (ERPs)
slide11

EEG: processing spatial sentences

  • ERPs

Fig. 4 from Noordzij et al. (2006)

Parieto-occipital activation for spatial sentences, expecting a picture.

visual-spatial strategy

slide12

fMRI: processing spatial sentences

  • Sentence-sentence and sentence-picture paradigm
  • 100% expectancy for S2
  • RT faster for picture than sentence, especially for spatial
slide13

fMRI: processing spatial sentences

  • fMRI

Fig. 2 from Noordzij et al. (2008)

Spatial > Non-spatial for both stimulus modalities: activity in left Supramarginal gyrus (SMG)

nature of mental representations
Nature of mental representations
  • How is spatial language processed in the absence of vision?
  • Is language processing different for different input modalities?
  • Is the nature of mental representations modality specific?
fmri blind sighted
fMRI blind & sighted
  • Sentence-sentence paradigm
    • Comparing spatial and non-spatial
  • Auditory version
  • Scanner-details:
    • 3T Philips Achieva scanner
    • PRESTO-SENSE sequence
    • TR = 500ms
    • Voxel-size: 4*4*4 mm
subjects
Subjects
  • 13 congenitally blind
    • Age 36.5 ± 9.8
    • 7 male, 6 female
    • 5 right-handed, 5 left-handed, 3 ambidexter
  • 13 sighted controls
    • Age 37.2 ± 11.2
    • 8 male, 5 female
    • 6 right-handed, 4 left-handed, 3 ambidexter
spatial sentence comprehension
Spatial Sentence Comprehension
  • Compare two sentences
    • Do they describe the same situation?
spatial sentence comprehension1
Spatial Sentence Comprehension
  • Block-design: 4 sessions, 18 blocks per session, 2 trials of 7.5s per block
  • 4 conditions
spatial sentence comprehension2
Spatial Sentence Comprehension

Position = Left of/Right of

Size = Taller than/Smaller than

Conjunction = Together with

Age = Older than/Younger than

Position(15s)

Size(15s)

Conjunction(15s)

Interval (6-9s)

Age(15s)

Interval (6-9s)

Interval (6-9s)

Interval (6-9s)

Presented pseudo-randomly

performance
Performance
  • Performance is good
  • No difference CB or SC
behavioral results rt
Behavioral Results: RT
  • Main effects of Space and Category
  • No group difference
results supramarginal gyrus
Results: Supramarginal Gyrus
  • Contrast: Position > Combination
  • ROI around SMG from Noordzij et al. (2008)
  • Conjunction CB and SC

ROI, T=2.5, p = .045 (corrected)

results supramarginal gyrus1
Results: Supramarginal Gyrus
  • Contrast: Relational > Combination
  • ROI around SMG from Noordzij et al. (2008)
  • Conjunction CB and SC

ROI, T=2.5, p = .045 (corrected)

fmri results
fMRI results
  • Evidence for reorganization
    • Difference CB and SC for task vs. rest
  • Occipital areas
  • No dimension modulation

Whole brain, T=4.0,k=10 voxels, p < 0.03

conclusions 1
Conclusions 1
  • Behavioral results: sighted can generate propositional and visual-spatial representations
  • Target area: left SMG
  • Left SMG not influenced by
    • context (picture or sentence)
    • input modality (visual or auditory)
conclusions 2
Conclusions 2
  • Blind also activate left SMG and visual areas
  • Visual areas are not modulated by Space or Category
  • Left SMG important for processing spatial prepositions
    • Other function: ordering on 1 dimension
  • Activation in left SMG is modality-independent + hardwired
verb generation
Verb Generation
  • Block-design
  • 3 types of blocks
    • Rest
    • Non-words
    • Words
results
Results
  • Conjunction CB+SC
  • Language areas
    • bilateral Broca, Wernicke
  • Covert verb generation:
    • precentral gyrus
    • supplementary motor area
  • Cerebellum

Word - Nonword

Whole brain, T=4.0,k=20 voxels, p < 0.007

results reorganization
Results: Reorganization
  • Difference CB-SC
  • CB significantly activate occipital cortex
    • BA 19, 31, 37

CB vs. SC

Whole brain, T=4.5, k=10 voxels, p < 0.019