The dynamics of attending and scene analysis
1 / 79

The Dynamics of Attending and Scene Analysis - PowerPoint PPT Presentation

  • Uploaded on

The Dynamics of Attending and Scene Analysis. By Mari Riess Jones. Thanks to . Ralph Barnes Devin McAuley Heather Moynihan Noah MacKenzie Amandine Penel Jennifer Puente Nate Vaughan. Special Thanks to Edward Large. Overview.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'The Dynamics of Attending and Scene Analysis' - maxime

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
The dynamics of attending and scene analysis l.jpg
The Dynamics of Attending andScene Analysis


Mari Riess Jones

Thanks to l.jpg
Thanks to

Ralph Barnes

Devin McAuley

Heather Moynihan

Noah MacKenzie

Amandine Penel

Jennifer Puente

Nate Vaughan

Special Thanks to

Edward Large

Overview l.jpg

  • Scene Analysis: A Neo Gestalt view

  • Scene Analysis: A dynamic perspective

  • The Changing Scene: Tracking in real time

I scene analysis a neo gestalt view l.jpg
I. Scene Analysis: A Neo Gestalt view

What s a scene l.jpg
What’s a Scene?

The term scene is taken from vision. Two cases are most common:

1. A stationary viewer

2. A moving viewer

The scene is the static visual layout.

Sound scenes l.jpg
Sound Scenes

A common assumption:

We use the proximal sound stimulus to recover the location and/or identity of a distal, vibrating, object within a scene.

This is the source identity principle.

Bregman s theory of scene analysis l.jpg
Bregman’s Theoryof Scene Analysis

Bregman builds on this idea:

Goal of auditory perception… is identification of one or more sources of a proximal mixture of sounds.

If an unfolding sound pattern is perceived as a structurally intact ,

i.e. as one stream,

then it must come from ONE source.

Other patterns l.jpg
Other patterns?

Patterns NOT perceived as intact….

If a pattern is perceived to break up, i.e. form N different subjective units, streams,

then, it must … come from N separate sources.

Slide9 l.jpg

Example: A sound pattern

Another source?


A stream ?


One source?



The big question l.jpg
The Big Question

What makes a pattern seem structurally intact?

Or… What determines a stream?

Bregman s answer l.jpg
Bregman’s Answer

Involves a two stage theory

in which

Neo Gestalt principles are critical.

The neo gestalt view l.jpg
The Neo Gestalt View

Two Stages

  • Primitive processesInnate, automatic, effortless, Determine coherent perceptual groups at fast rates.

  • II. Schema driven processesLearned, voluntary, effortful, Determine attention to groupings at slow rates.

Auditory scene perception l.jpg
Auditory Scene Perception

Stage I: Primitive Processes Operate on Gestalt grouping principles:

proximity in frequency and time; similarity (e.g., timbre); continuity; exclusive allocation… and so forth

These are domain general (I.e. speech, music, natural environs,etc.)

Proximity vs continuity l.jpg
Proximity vs Continuity

Tougas and Bregman (1985)

In a series of famous studies pitted proximity against continuity.

Goal:To assess which principle dominated, and to dispute a trajectory explanation.

Bouncing versus Crossing trajectories

Slide15 l.jpg

Possible Percepts

Time in seconds

Conclusion l.jpg

People hear bouncing patterns

Proximity in frequency dominates

Rhythm perception l.jpg
Rhythm Perception

A stream usually conveys a rhythm

Speeding up a sequence can break it , perceptually, into interleaved streams

This changes its perceived rhythm!

Rhythm l.jpg

Rhythm perception… depends on the relationship of frequency change (f )to time change (t).




2 t


Lo frequency


This slow melody…

is perceived to have a ‘galloping ‘rhythm

But that rhythm disappears l.jpg
But… That Rhythm Disappears

In a faster melody with the same tones.

Now we hear two melodies (streams) each with anisochronousrhythm.

A neo gestalt explanation l.jpg
A Neo Gestalt Explanation

  • Builds on Gestalt proximity:

  • Temporal proximity is heightened at fast rates.

  • But frequency proximity dominates.

  • Two streams form based on pitch proximity: A high tone stream and a low tone stream.

Scene analysis a dynamic perspective l.jpg
Scene Analysis:A Dynamic Perspective

Auditory patterns l.jpg
Auditory Patterns

Most of our environment comprises

sources that turn on and off and

modulate over time to project pattern

structure that is….

motion-like and rhythmic

Important pattern properties l.jpg
Important Pattern Properties

  • Motionlike properties: Rates of change.

  • [ e.g., f/t ]

2. Rhythmic properties:Relative timing.

[e.g., ti/tk ]

Motionlike trajectories l.jpg
Motionlike Trajectories

In 1976, I proposed that the rate-of-change of frequency (log scale) may be perceived as a sort of velocity.

Pitch Velocity: f / t

Slide25 l.jpg

Determinants of Pitch Velocity




f/ t




A sudden increase in rate-of-change in an isochronous rhythm is perturbing.

Motionlike trajectory limits on pitch velocity l.jpg
Motionlike Trajectory limits on pitch velocity?



Intact Serial Percepts

Velocity Limit

 f log

Stimulus patterns


FAST t Slow


(Jones, 1976)

Rhythmic properties relative timing l.jpg
Rhythmic Properties: Relative Timing


Slower patterns




Alternative view a focus on time l.jpg
Alternative View: A focus on time

How do rhythmic properties of a pattern affect real time responding to it?

In the remainder of this talk, I address this question.

General assumptions l.jpg
General Assumptions


Scenes often project dynamic patterns with relative timing that is quasi- periodic.

Living things

Animate creatures possess internal periodicities that can synchronize to patterns.

Connecting through time

Pattern timing determines how living things connect with a changing scene.

Some quasi periodic patterns l.jpg
Some quasi-periodic patterns

t = IOI = 700 ms

Great Gray Owl

Horned Lark


Internal periodicities biological oscillations l.jpg
Internal Periodicities: Biological Oscillations

Many internal periodicities exist; periods of 1 ms, several minutes, years !.

Most studied are circadian rhythms; oscillatory periods of ca. 24 hours.

Connecting l.jpg

“How do internal oscillations of an organism connect with pattern timing? “

One answer: By synchronizing.

More precisely: By entrainment

Biological entrainment l.jpg
Biological Entrainment

Pittendrigh & Daan (1976) . Circadian rhythms of 24 hours synchronize to light/dark cycles.

Daan et al. (2001) recently updated this two oscillator model.

Slide34 l.jpg

Example: Two Entraining Oscillations


M Phase locked

E Phase locked


From Daan et al. J.Biological Rhythms (2001)

Shorter periods l.jpg
Shorter Periods?

In species

with refined use of fast sound patterns,

Can we postulate,

internal oscillations with periods less than the circadian period ?

Generalizing l.jpg


Other Oscillators

Most Biological


Long natural periods

Respond to light changes

Short natural periods

Respond to sound


Background rhythmic periodic expectancies l.jpg
Background:Rhythmic (periodic) Expectancies

Quasi-periodicities in

Rhythmical sound patterns

Induce Periodic expectancies

Perhaps such expectancies reflect entrainments to patterns of stimulus IOIs?

Back to synchrony a simple premise l.jpg
Back to Synchrony: Asimple premise

When we attend , an internal activity is elicited … that co-occurs with the attended object.

This activity involves Internal oscillations (Jones, 1976).

Attending to dynamic patterns l.jpg
Attending to Dynamic Patterns

Where elements appear… then disappear


Attention must occur in synchrony with elements.

Dynamic attending l.jpg
Dynamic Attending

Postulates that :

Selective attending in time is paced by stimulus timing.

Attentional focus in time becomes phase-locked to elements.

Iii the changing scene tracking sounds in real time l.jpg
III. The Changing Scene:Tracking sounds in real time

Attending oscillates in time l.jpg
Attending oscillates in time

Resources are periodically distributed as internal oscillations.

Inter-onset-interval, IOI


Pulses of attention energy

Slide43 l.jpg

Formalized as an Oscillator Model :

by Ed Large (Large & Jones, 1999)

Stimulus IOI

Expected point in time



Periodic Expectancies

Stimulus timing paces the oscillator. Oscillator period and phase determine this entrainment .

Slide44 l.jpg

The adaptive oscillator

Tones, i

Period, Pi

Phase, i

Period and Phase change: They draw attending rhythms to “fit” a time pattern.

Period changes

Phase changes

A simple dynamical system l.jpg
A simple dynamical system.

The entrained oscillator gravitates to an attractor state of:

  • 1. Phase Synchrony:A zero time difference between an sound onset and a pulse peak.

  • 2. Period matching:A zero time difference between an IOI and oscillator period (limit cycle).

Slide46 l.jpg

Relative Phase, i Gives observed - expected times

i is the order variable in this system. It summarizes the state of the system at any instant i.

i+1 = i + IOIi+1/Pi - F(i)

The coupling term, F, here is sinusoidal:

F()= {1/2}sin 2 => 

Slide47 l.jpg


Entrainment Region

{  = 1.0 }


Phase Attractor: Synchrony

Phase Attractor:synchrony





Relative Phase,


What does this approach purchase l.jpg
What does this approach purchase ?

A few experiments….

Preview our experimental designs l.jpg
Preview: Our Experimental Designs

Will involve a context rhythm, given by tones:


Followed by a comparison, C, (of some sort)

Time JudgmentComparison IOI

Pitch JudgmentComparison tone

Time judgment tasks rationale l.jpg
Time Judgment Tasks Experimental DesignsRationale

If relative timing in sound arrays entrains attending,

then the context rhythm should

Shape temporal expectancies

Affect time judgments.

Slide51 l.jpg

Example Study: Experimental Designs

Judgment Task: Is a comparison time interval “shorter”, “same” or “ longer” ?


Standard IOI

Comparison IOI

Context IOI

“Ignore context IOIs“

Context IOIs are 600 ms. This rhythm should synchronize attending.

Slide52 l.jpg

Standard Ending Varies: Experimental Designs

Expected Standard Ending: = 600 ms

Unexpected Standard Endings:

Early = 579 ms

Late = 621 ms

Very Early = 524 ms

Very Late = 676 ms

A comparison IOI is always yoked to standard IOI


Slide53 l.jpg

Modeling Oscillatory Attending Experimental Designs

Context IOIs








600 ms




When a standard ends as expected, the oscillator’s period is accurately preserved ….as a memory of the standard IOI.

Slide54 l.jpg

The Reason: Experimental DesignsThe Attentional Pulse

Expected time

Pulse location

Pulse location depends on rhythmic context


Slide55 l.jpg

One Prediction: Experimental Designs

A quadratic accuracy Expectancy Profile:

Attentional Pulse

Predicted and observed expectancy profiles large jones 1999 l.jpg
Predicted and Observed Experimental DesignsExpectancy Profiles (Large & Jones 1999)

P < .01

Subsequent experiments l.jpg
Subsequent Experiments Experimental Designs

Over a dozen experiments later (e.g., Barnes & Jones, 2000; McAuley & Jones, 2002)

  • We Find:

  • The profile vanishes with no context; sharpens with longer sequences .

  • Holds for different rates.

  • Reveals harmonic scaling of timing.

Different induction rates l.jpg
Different Induction Rates Experimental Designs

Isochronous sequences

Three conditions: different induction rates.

Condition Induction IOIs Standard durations

1. 300 ms 300 ms 524, 600, 676 ms

2. 500 ms 500 ms 524, 600, 676 ms

3. 600 ms 600 ms 524, 600, 676 ms

This task required same vs different duration judgments

Observed expectancy profiles three time judgment conditions l.jpg
Observed Expectancy Profiles Experimental Designs Three Time Judgment Conditions

Conclude l.jpg
Conclude Experimental Designs

Back to Relative timing

1. Ratios of induction IOIs to expected standards seem influential.

2. Harmonic ratios of 1:1 and 1:2 yield different profiles than the 500 ms with a more complex ratio.

Pitch judgments l.jpg
Pitch Judgments Experimental Designs

We generalized this approach to attending to tone frequency.

Does rhythmic context affect pitch judgments?

Slide62 l.jpg

A Pitch Judgment Task Experimental Designs

Task: Is a comparison pitch: “same”, “higher”, ”lower” than a standard pitch?

“Ignore distracting pitches!”

Critical IOI




Standard pitch

Comparison pitch

Independent Variable: Critical IOI


Slide63 l.jpg

Rationale Experimental Designs

In a rhythmic context, attending is paced in time.

Pitch judgments will be best for temporally expected tones:

When critical IOI = context IOI.

Slide64 l.jpg

Similar Predictions Experimental Designs

Aquadratic expectancy profile

Attentional Pulse

Slide65 l.jpg

P<.005 Experimental Designs

Related experiments l.jpg
Related Experiments Experimental Designs

Found a:

flattened expectancy profile

with irregular context timing

Recent findings l.jpg
Recent findings Experimental Designs

Suggest the importance of higher-order time structure …

In focal attending over time levels

Slide68 l.jpg

An Accent Rhythm Experimental Designs

Accent Timing

Higher time level : 4 IOIs

Lower time level: IOI

Accented Tones


Slide69 l.jpg

Accent Rhythm is Varied Experimental Designs



Accented Tones


Slide70 l.jpg

Same Pitch Judgment Task Experimental Designs

Critical IOI is Varied

Standard pitch

Comparison pitch


Two oscillators l.jpg
Two Oscillators? Experimental Designs

Remember those two circadian oscillators (Morning, Evening)?

Well, these are different!

Shorter periods Sensitive to sound

Two different oscillators l.jpg
Two Different Oscillators Experimental Designs

Two oscillators: Different periods

1. Lower-order oscillator, period ~ IOI.

2. Accent oscillator, period ~ 4 IOIs

Which dominates?

If the Accent oscillator dominates, then only the Regular Rhythm will give an Expectancy Profile.

Slide73 l.jpg

PC Experimental Designs


Two oscillators in listening to sound patterns l.jpg
Two Oscillators Experimental DesignsIn listening to sound patterns

Differential responses to sounds: Accented versus Unaccented

Flexible attending:Dominance of one oscillator in focal attending.

Final comments l.jpg
Final Comments Experimental Designs

Assumptions about rhythm and attentional synchrony can enhance theoretical power.

The dynamics of sound patterns together the dynamics of attending may explain real time tracking of events.

Entrainment to dynamic contexts may play a role in judgments of both time and pitch.

Thank you l.jpg
Thank you ! Experimental Designs

Two routes to synchrony l.jpg
Two routes to Synchrony Experimental Designs

1. Anticipatory attending:

Attending heightens prior to an element.

Oscillator period is important.

  • Reactive attending:

  • If anticipation fails,

  • Attending shifts after an element occurs.

  • Oscillator phase is important.