1 / 29

Outline

Audiovisual Multisensory Facilitation: A Fresh Look at Neural Coactivation and Inverse Effectiveness. Lynnette Leone North Dakota State University.

willow-beck
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Audiovisual Multisensory Facilitation: A Fresh Look at Neural Coactivation and Inverse Effectiveness.Lynnette Leone North Dakota State University

  2. “However, the primary somatic, visual and auditory cortices are not interconnected, and each projects to very restricted and entirely separate fields chiefly in their immediate vicinity…” (Jones and Powell, 1970)“Such integration is as critical for making sense of the inputs the brain receives from different modalities as it is for interpreting multiple inputs from any single modality…for the brain to integrate them [these inputs] the different senses must ultimately have access to the same neurons.” (Stein and Meredith 1993)

  3. Outline Multisensory Integration Redundant Signals Effect 3. Inverse Effectiveness 4. The Current Study 5. Future Directions

  4. Multisensory integration Definition: (MI) refers to the process by which convergence of information from two or more individual sensory systems onto particular neurons results in a neuronal response that is qualitatively and quantitatively different than the responses of these neurons to individual signals (Calvert, 2001).

  5. Multisensory integration McGurk Effect Visual Rabbit

  6. Multisensory integration Facilitation – more likely to occur when two things happen at the same time and / or in the same place. Suppression – more likely to occur if two event occur at widely different times and / or places.

  7. Redundant Signals Effect (RSE) Definition: the modulation of reaction time to pairings of sensory stimuli presented simultaneously over one or more sensory channels. - facilitative MI - not exclusively multisensory

  8. Redundant Signals Effect (RSE) Separate Activation vs. Neural Coactivation - RACE models – Miller’s Inequality P (RT < t|A and V) ≤ [ P (RT < t|A) + P (RT < t|V)] - [P (RT < t|A) ٭ P (RT < t|V)]

  9. Redundant Signals Effect (RSE) Some studies: Miller, J. Cognitive Psychology (1982) Experiment 1 • Subjects: 74, undergrads rt handed • Visual Stimuli: asterisk (٭) • Auditory Stim: 780 Hz tone (150ms) • Task: Simple RT • 3 conditions: A, V, AV • Randomly Interleaved

  10. Redundant Signals Effect (RSE) • Results - mean RT V = 412 ms, A = 409 ms, AV = 326 ms - violations of inequality occurred across a range of reaction times (250 – 350 ms) Miller, J.Cognitive Psychology, (1982)

  11. Redundant Signals Effect (RSE) Molholm et alCognitive Brain Research (2002) • Subjects: 12 (5 female, 11 RH), 23.8±2.7 yr • Visual Stimuli: 60 ms, 1.2 deg disc • Auditory Stim: 1000 Hz, 60 ms, 75 dB SPL • Task: Simple RT, right index finger button press • 3 conditions: A, V, AV • Randomly Interleaved • ISI: 750-3500 ms

  12. Redundant Signals Effect (RSE) Molholm, S. et al Cognitive Brain Research (2002)

  13. Redundant Signals Effect (RSE) Spatial and temporal constraints - Miller, (1986) - Stein et al, (1996) - Frassinetti et al, (2002)

  14. Inverse Effectiveness Rule: combinations of weaker stimuli lead to greater facilitative effects. Studies: - Stein and Meredith, 1993 - Diederich and Colonius, 2004 - Holmes, 2007

  15. Current Study How might changes in the processing time of one system influence the time-course of the RSE? - increasing stimulus contrast leads to decreases in reaction time. (Harworth and Levi, 1978, Murray and Plainis, 2003 and Vassilev, Mihaylova and Bonnet, 2002)

  16. Current Study B. Hypothesis – If neural coactivation is indeed responsible for the RSE, then changes in signal processing time in one modality will necessarily change the facilitative effects obtained when those signals are paired with signals from another modality.

  17. Method Signal Pretest Procedure: - Single-interval forced choice signal detection paradigm Response

  18. Method Pretest Procedure: - 15 Blocks; each block contained 24 unisensory visual (2 x 12 contrast intensities, spatial frequency 1 c/d) and 24 unisensory auditory (2 x 12 levels of dB attenuation), and 2 catch trials. - d’ calculated for responses to individual stimuli (d’ = Z(h) – Z (FA); nonlinear (sigmoid) least-squares regression interpolated stimulus intensities yielding criterion performance (d’ = 2).

  19. Visual stimulus (Gabor patch = 1 c/d; duration = 100ms;view distance = 114 cm, 2.25◦ eccentricity from fixation) Auditory Stimulus (1000 Hz pure tone duration = 100ms) Subjects: 7 (4 Male, 7 RH); 23 – 55 yrs

  20. Method Experimental Procedure: - Single-interval signal detection paradigm; - Each block: randomly interleaved trials of unisensory visual stimuli, unisensory auditory stimuli, catch trials, and audiovisual multisensory combinations at stimulus onset asynchronies ranging from -100 to +200 ms.   - Task: Subjects instructed to respond as quickly and accurately as possible when they perceived a stimulus. Calculated d’ for unisensory stimuli every 2 blocks and adjusted intensity settings to ensure criterion response accuracy (d’ = 2).

  21. Data Analysis

  22. Experiment 1: Results

  23. Experiment 2: Paradigm replicated experiment 1 except that visual stimulus contrast was increased (3x).

  24. Experiment 3: Paradigm replicated experiment 1 except that auditory stimulus volume increased (3x).

  25. Experiment 4: Paradigm replicated experiment 1 except that both visual stimulus contrast and auditory stimulus volume were increased (3x).

  26. Conclusions What the heck? Attentional effects: - endogenous attention - exogenous attention

  27. What’s next Experiment 5 - examines attentional contributions using a variation of a Posner cuing paradigm to separate the effects of attention from those of neuronal interaction. Experiment 6 - examines inverse effectiveness gradient.

  28. Future Directions • Dark adaptation - rods dominate our vision in the dark - rods operate on the order of 100ms slower than cones • ERP

  29. Acknowledgements: Dr. Mark McCourt Dr. Wolfgang Teder - SäleJärvi Tech Support: Dan Gu, Brian Pasieka All of the subjects (most of whom are in this room)

More Related