The dynamics of attending and scene analysis l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 79

The Dynamics of Attending and Scene Analysis PowerPoint PPT Presentation


  • 89 Views
  • Uploaded on
  • Presentation posted in: General

The Dynamics of Attending and Scene Analysis. By Mari Riess Jones. Thanks to . Ralph Barnes Devin McAuley Heather Moynihan Noah MacKenzie Amandine Penel Jennifer Puente Nate Vaughan. Special Thanks to Edward Large. Overview.

Download Presentation

The Dynamics of Attending and Scene Analysis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The dynamics of attending and scene analysis l.jpg

The Dynamics of Attending andScene Analysis

By

Mari Riess Jones


Thanks to l.jpg

Thanks to

Ralph Barnes

Devin McAuley

Heather Moynihan

Noah MacKenzie

Amandine Penel

Jennifer Puente

Nate Vaughan

Special Thanks to

Edward Large


Overview l.jpg

Overview

  • Scene Analysis: A Neo Gestalt view

  • Scene Analysis: A dynamic perspective

  • The Changing Scene: Tracking in real time


I scene analysis a neo gestalt view l.jpg

I. Scene Analysis: A Neo Gestalt view


What s a scene l.jpg

What’s a Scene?

The term scene is taken from vision. Two cases are most common:

1. A stationary viewer

2. A moving viewer

The scene is the static visual layout.


Sound scenes l.jpg

Sound Scenes

A common assumption:

We use the proximal sound stimulus to recover the location and/or identity of a distal, vibrating, object within a scene.

This is the source identity principle.


Bregman s theory of scene analysis l.jpg

Bregman’s Theoryof Scene Analysis

Bregman builds on this idea:

Goal of auditory perception… is identification of one or more sources of a proximal mixture of sounds.

If an unfolding sound pattern is perceived as a structurally intact ,

i.e. as one stream,

then it must come from ONE source.


Other patterns l.jpg

Other patterns?

Patterns NOT perceived as intact….

If a pattern is perceived to break up, i.e. form N different subjective units, streams,

then, it must … come from N separate sources.


Slide9 l.jpg

Example: A sound pattern

Another source?

high

A stream ?

frequency

One source?

low

Time


The big question l.jpg

The Big Question

What makes a pattern seem structurally intact?

Or… What determines a stream?


Bregman s answer l.jpg

Bregman’s Answer

Involves a two stage theory

in which

Neo Gestalt principles are critical.


The neo gestalt view l.jpg

The Neo Gestalt View

Two Stages

  • Primitive processesInnate, automatic, effortless, Determine coherent perceptual groups at fast rates.

  • II. Schema driven processesLearned, voluntary, effortful, Determine attention to groupings at slow rates.


Auditory scene perception l.jpg

Auditory Scene Perception

Stage I: Primitive Processes Operate on Gestalt grouping principles:

proximity in frequency and time; similarity (e.g., timbre); continuity; exclusive allocation… and so forth

These are domain general (I.e. speech, music, natural environs,etc.)


Proximity vs continuity l.jpg

Proximity vs Continuity

Tougas and Bregman (1985)

In a series of famous studies pitted proximity against continuity.

Goal:To assess which principle dominated, and to dispute a trajectory explanation.

Bouncing versus Crossing trajectories


Slide15 l.jpg

Possible Percepts

Time in seconds


Conclusion l.jpg

Conclusion

People hear bouncing patterns

Proximity in frequency dominates


Rhythm perception l.jpg

Rhythm Perception

A stream usually conveys a rhythm

Speeding up a sequence can break it ,perceptually, into interleaved streams

This changes its perceived rhythm!


Rhythm l.jpg

Rhythm

Rhythm perception… depends on the relationship of frequency change (f )to time change (t).

Hi

frequency

t

2 t

f

Lo frequency

Time

This slow melody…

is perceived to have a ‘galloping ‘rhythm


But that rhythm disappears l.jpg

But… That Rhythm Disappears

In a faster melody with the same tones.

Now we hear two melodies (streams) each with anisochronousrhythm.


A neo gestalt explanation l.jpg

A Neo Gestalt Explanation

  • Builds on Gestalt proximity:

  • Temporal proximity is heightened at fast rates.

  • But frequency proximity dominates.

  • Two streams form based on pitch proximity: A high tone stream and a low tone stream.


Scene analysis a dynamic perspective l.jpg

Scene Analysis:A Dynamic Perspective


Auditory patterns l.jpg

Auditory Patterns

Most of our environment comprises

sources that turn on and off and

modulate over time to project pattern

structure that is….

motion-like and rhythmic


Important pattern properties l.jpg

Important Pattern Properties

  • Motionlike properties: Rates of change.

  • [ e.g., f/t ]

2. Rhythmic properties:Relative timing.

[e.g., ti/tk ]


Motionlike trajectories l.jpg

Motionlike Trajectories

In 1976, I proposed that the rate-of-change of frequency (log scale) may be perceived as a sort of velocity.

Pitch Velocity: f / t


Slide25 l.jpg

Determinants of Pitch Velocity

high

f

frequency

f/ t

t

low

Time

A sudden increase in rate-of-change in an isochronous rhythm is perturbing.


Motionlike trajectory limits on pitch velocity l.jpg

Motionlike Trajectory limits on pitch velocity?

LARGE

streamed

Intact Serial Percepts

Velocity Limit

 f log

Stimulus patterns

SMALL

FAST t Slow

RATE

(Jones, 1976)


Rhythmic properties relative timing l.jpg

Rhythmic Properties: Relative Timing

Limit

Slower patterns

flog

FAST t =IOI SLOW

RATE


Alternative view a focus on time l.jpg

Alternative View: A focus on time

How do rhythmic properties of a pattern affect real time responding to it?

In the remainder of this talk, I address this question.


General assumptions l.jpg

General Assumptions

Environment

Scenes often project dynamic patterns with relative timing that is quasi- periodic.

Living things

Animate creatures possess internal periodicities that can synchronize to patterns.

Connecting through time

Pattern timing determines how living things connect with a changing scene.


Some quasi periodic patterns l.jpg

Some quasi-periodic patterns

t = IOI = 700 ms

Great Gray Owl

Horned Lark

Cardinal


Internal periodicities biological oscillations l.jpg

Internal Periodicities: Biological Oscillations

Many internal periodicities exist; periods of 1 ms, several minutes, years !.

Most studied are circadian rhythms; oscillatory periods of ca. 24 hours.


Connecting l.jpg

Connecting

“How do internal oscillations of an organism connect with pattern timing? “

One answer: By synchronizing.

More precisely: By entrainment


Biological entrainment l.jpg

Biological Entrainment

Pittendrigh & Daan (1976) . Circadian rhythms of 24 hours synchronize to light/dark cycles.

Daan et al. (2001) recently updated this two oscillator model.


Slide34 l.jpg

Example: Two Entraining Oscillations

Winter

M Phase locked

E Phase locked

Summer

From Daan et al. J.Biological Rhythms (2001)


Shorter periods l.jpg

Shorter Periods?

In species

with refined use of fast sound patterns,

Can we postulate,

internal oscillations with periods less than the circadian period ?


Generalizing l.jpg

Generalizing…..

Alternatively

Other Oscillators

Most Biological

Oscillators

Long natural periods

Respond to light changes

Short natural periods

Respond to sound

changes


Background rhythmic periodic expectancies l.jpg

Background:Rhythmic (periodic) Expectancies

Quasi-periodicities in

Rhythmical sound patterns

Induce Periodic expectancies

Perhaps such expectancies reflect entrainments to patterns of stimulus IOIs?


Back to synchrony a simple premise l.jpg

Back to Synchrony: Asimple premise

When we attend , an internal activity is elicited … that co-occurs with the attended object.

This activity involves Internal oscillations (Jones, 1976).


Attending to dynamic patterns l.jpg

Attending to Dynamic Patterns

Where elements appear… then disappear

Time

Attention must occur in synchrony with elements.


Dynamic attending l.jpg

Dynamic Attending

Postulates that :

Selective attending in time is paced by stimulus timing.

Attentional focus in time becomes phase-locked to elements.


Iii the changing scene tracking sounds in real time l.jpg

III. The Changing Scene:Tracking sounds in real time


Attending oscillates in time l.jpg

Attending oscillates in time

Resources are periodically distributed as internal oscillations.

Inter-onset-interval, IOI

Time

Pulses of attention energy


Slide43 l.jpg

Formalized as an Oscillator Model :

by Ed Large (Large & Jones, 1999)

Stimulus IOI

Expected point in time

Phase

Period

Periodic Expectancies

Stimulus timing paces the oscillator. Oscillator period and phase determine this entrainment .


Slide44 l.jpg

The adaptive oscillator

Tones, i

Period, Pi

Phase, i

Period and Phase change: They draw attending rhythms to “fit” a time pattern.

Period changes

Phase changes


A simple dynamical system l.jpg

A simple dynamical system.

The entrained oscillator gravitates to an attractor state of:

  • 1. Phase Synchrony:A zero time difference between an sound onset and a pulse peak.

  • 2. Period matching:A zero time difference between an IOI and oscillator period (limit cycle).


Slide46 l.jpg

Relative Phase, i Gives observed - expected times

i is the order variable in this system. It summarizes the state of the system at any instant i.

i+1 = i + IOIi+1/Pi - F(i)

The coupling term, F, here is sinusoidal:

F()= {1/2}sin 2 => 


Slide47 l.jpg

Theoretical

Entrainment Region

{  = 1.0 }

-

Phase Attractor: Synchrony

Phase Attractor:synchrony

.25

.25

-.25

-.25

Relative Phase,

i


What does this approach purchase l.jpg

What does this approach purchase ?

A few experiments….


Preview our experimental designs l.jpg

Preview: Our Experimental Designs

Will involve a context rhythm, given by tones:

C

Followed by a comparison, C, (of some sort)

Time JudgmentComparison IOI

Pitch JudgmentComparison tone


Time judgment tasks rationale l.jpg

Time Judgment TasksRationale

If relative timing in sound arrays entrains attending,

then the context rhythm should

Shape temporal expectancies

Affect time judgments.


Slide51 l.jpg

Example Study:

Judgment Task: Is a comparison time interval “shorter”, “same” or “ longer” ?

+/-T

Standard IOI

Comparison IOI

Context IOI

“Ignore context IOIs“

Context IOIs are 600 ms. This rhythm should synchronize attending.


Slide52 l.jpg

Standard Ending Varies:

Expected Standard Ending: = 600 ms

Unexpected Standard Endings:

Early = 579 ms

Late = 621 ms

Very Early = 524 ms

Very Late = 676 ms

A comparison IOI is always yoked to standard IOI

23


Slide53 l.jpg

Modeling Oscillatory Attending

Context IOIs

STANDARD IOI

COMPARISON IOI

Expected

Ending:

“Same”

Expected

Ending

600 ms

600

Period

Phase

When a standard ends as expected, the oscillator’s period is accurately preserved ….as a memory of the standard IOI.


Slide54 l.jpg

The Reason: The Attentional Pulse

Expected time

Pulse location

Pulse location depends on rhythmic context

15


Slide55 l.jpg

One Prediction:

A quadratic accuracy Expectancy Profile:

Attentional Pulse


Predicted and observed expectancy profiles large jones 1999 l.jpg

Predicted and ObservedExpectancy Profiles (Large & Jones 1999)

P < .01


Subsequent experiments l.jpg

Subsequent Experiments

Over a dozen experiments later (e.g., Barnes & Jones, 2000; McAuley & Jones, 2002)

  • We Find:

  • The profile vanishes with no context; sharpens with longer sequences .

  • Holds for different rates.

  • Reveals harmonic scaling of timing.


Different induction rates l.jpg

Different Induction Rates

Isochronous sequences

Three conditions: different induction rates.

Condition Induction IOIs Standard durations

1. 300 ms 300 ms524, 600, 676 ms

2. 500 ms 500 ms 524, 600, 676 ms

3. 600 ms 600 ms524, 600, 676 ms

This task required same vs different duration judgments


Observed expectancy profiles three time judgment conditions l.jpg

Observed Expectancy Profiles Three Time Judgment Conditions


Conclude l.jpg

Conclude

Back to Relative timing

1. Ratios of induction IOIs to expected standards seem influential.

2. Harmonic ratios of 1:1 and 1:2 yield different profiles than the 500 ms with a more complex ratio.


Pitch judgments l.jpg

Pitch Judgments

We generalized this approach to attending to tone frequency.

Does rhythmic context affect pitch judgments?


Slide62 l.jpg

A Pitch Judgment Task

Task: Is a comparison pitch: “same”, “higher”, ”lower” than a standard pitch?

“Ignore distracting pitches!”

Critical IOI

“higher”

same”

“lower”

Standard pitch

Comparison pitch

Independent Variable: Critical IOI

21


Slide63 l.jpg

Rationale

In a rhythmic context, attending is paced in time.

Pitch judgments will be best for temporally expected tones:

When critical IOI = context IOI.


Slide64 l.jpg

Similar Predictions

Aquadratic expectancy profile

Attentional Pulse


Slide65 l.jpg

P<.005


Related experiments l.jpg

Related Experiments

Found a:

flattened expectancy profile

with irregular context timing


Recent findings l.jpg

Recent findings

Suggest the importance of higher-order time structure …

In focal attending over time levels


Slide68 l.jpg

An Accent Rhythm

Accent Timing

Higher time level : 4 IOIs

Lower time level: IOI

Accented Tones

21


Slide69 l.jpg

Accent Rhythm is Varied

Regular

Irregular

Accented Tones

21


Slide70 l.jpg

Same Pitch Judgment Task

Critical IOI is Varied

Standard pitch

Comparison pitch

21


Two oscillators l.jpg

Two Oscillators?

Remember those two circadian oscillators (Morning, Evening)?

Well, these are different!

Shorter periods Sensitive to sound


Two different oscillators l.jpg

Two Different Oscillators

Two oscillators: Different periods

1. Lower-order oscillator, period ~ IOI.

2. Accent oscillator, period ~ 4 IOIs

Which dominates?

If the Accent oscillator dominates, then only the Regular Rhythm will give an Expectancy Profile.


Slide73 l.jpg

PC

P<.001


Two oscillators in listening to sound patterns l.jpg

Two Oscillators In listening to sound patterns

Differential responses to sounds: Accented versus Unaccented

Flexible attending:Dominance of one oscillator in focal attending.


Final comments l.jpg

Final Comments

Assumptions about rhythm and attentional synchrony can enhance theoretical power.

The dynamics of sound patterns together the dynamics of attending may explain real time tracking of events.

Entrainment to dynamic contexts may play a role in judgments of both time and pitch.


Thank you l.jpg

Thank you !


Two routes to synchrony l.jpg

Two routes to Synchrony

1. Anticipatory attending:

Attending heightens prior to an element.

Oscillator period is important.

  • Reactive attending:

  • If anticipation fails,

  • Attending shifts after an element occurs.

  • Oscillator phase is important.


  • Login