Perceptual delay for rapid direction alternations: A new account in terms of the dichotomy of first-order and second-ord

1. Shin'ya Nishida NTT Communication Science Laboratories, Japan Alan Johnston Department of Psychology, University College London, UK Perceptual delay for rapid direction alternations:A new account in terms of the dichotomy offirst-order and second-order temporal changes

2. Perceptual asynchrony of colour and motion In order to obtain perceptual synchrony, relative phase of colour and motion has to be changed. (click) In this movie, you can tell more easily than before that when the colour is green, the movement is upward, and when the colour is red, the movement is downward. But physically speaking, motion change leads colour change by about 100 ms. In order to obtain perceptual synchrony, relative phase of colour and motion has to be changed. (click) In this movie, you can tell more easily than before that when the colour is green, the movement is upward, and when the colour is red, the movement is downward. But physically speaking, motion change leads colour change by about 100 ms.

3. Processing Time Hypothesis Modularity of Vision Specialised processing modules for each visual attribute Colour: V4 Motion: V5 Neural activity of each module generates micro-consciousness of the processed attribute. The processing time difference between modules leads to perceptual asynchrony for different stimulus attributes. Processing time is longer for motion than for colour. Moutoussis and Zeki explained this illusion in terms of processing time differences. (click) The visual system involves several specialised processing modules for each visual attributes. For instance, it is suggested that the colour centre is V4, and the motion centre is V5. (click) They argued that neural activity of each module generates micro-consciousness of the processed attribute. (click) Therefore the processing time difference between modules leads to perceptual asynchrony for different stimulus attributes. (click) Perceptual asynchrony occurs because processing time is longer for motion than for colour.Moutoussis and Zeki explained this illusion in terms of processing time differences. (click) The visual system involves several specialised processing modules for each visual attributes. For instance, it is suggested that the colour centre is V4, and the motion centre is V5. (click) They argued that neural activity of each module generates micro-consciousness of the processed attribute. (click) Therefore the processing time difference between modules leads to perceptual asynchrony for different stimulus attributes. (click) Perceptual asynchrony occurs because processing time is longer for motion than for colour.

4. Evidence Against Processing Time Hypothesis Perceptual motion delay occurs only for rapid alternations. Perceptual motion delay is not accompanied by reaction time differences. Perceptual motion delay is task dependent. However, we obtain a few lines of evidence against the processing time hypothesis. (click) First, perceptual motion delay occurs only for rapid alternations. (click) Second, perceptual motion delay is not accompanied by reaction time difference. (click) Third, perceptual motion delay is task dependent.. However, we obtain a few lines of evidence against the processing time hypothesis. (click) First, perceptual motion delay occurs only for rapid alternations. (click) Second, perceptual motion delay is not accompanied by reaction time difference. (click) Third, perceptual motion delay is task dependent..

5. Perceptual motion delay relative to colour occurs only for rapid alternations (Nishida & Johnston, 1999, ARVO) Alternation rate First, we found that perceptual motion delay relative to colour occurs only for rapid alternations. (click) When we asked subjects to judge the temporal order of an isolated transition of colour and an isolated transition of motion direction, their judgements were accurate. (click) In addition, even when we asked subjects to judge the temporal order of an isolated transition of colour or motion, and a transition of colour or motion embedded in a sequence of rapid oscillations, their judgements were again accurate.First, we found that perceptual motion delay relative to colour occurs only for rapid alternations. (click) When we asked subjects to judge the temporal order of an isolated transition of colour and an isolated transition of motion direction, their judgements were accurate. (click) In addition, even when we asked subjects to judge the temporal order of an isolated transition of colour or motion, and a transition of colour or motion embedded in a sequence of rapid oscillations, their judgements were again accurate.

6. Perceptual asynchrony is NOT accompanied by a difference in manual reaction time. (Nishida & Johnston, 2000, ARVO). Reaction time The next point is reaction time. We found that perceptual asynchrony is not accompanied by corresponding difference in reaction time. In this experiment, using the same rapid random sequences of colour and motion, we measured the relative perceptual timing and reaction times differences. We then found that motion perception was largely delayed in the perceptual task, but significant difference was not found between reaction times for colour and motion. The next point is reaction time. We found that perceptual asynchrony is not accompanied by corresponding difference in reaction time. In this experiment, using the same rapid random sequences of colour and motion, we measured the relative perceptual timing and reaction times differences. We then found that motion perception was largely delayed in the perceptual task, but significant difference was not found between reaction times for colour and motion.

7. Task dependency: synchronous button press The last point is task dependency. In this experiment, we asked subjects to synchronously press a mouse button during the phase of downward motion. The stimulus change was perfectly predictable, but the button press was always delayed from the onset of downward motion by about 100 ms. This result is consistent with the perceptual motion delay.The last point is task dependency. In this experiment, we asked subjects to synchronously press a mouse button during the phase of downward motion. The stimulus change was perfectly predictable, but the button press was always delayed from the onset of downward motion by about 100 ms. This result is consistent with the perceptual motion delay.

8. Task dependency: synchronous mouse move However, when we asked subjects to move a mouse forwards and backwards in synchrony with the upward and downward stimulus oscillation, the subjects can perfectly synchronise the mouse movement with the stimulus oscillation. However, when we asked subjects to move a mouse forwards and backwards in synchrony with the upward and downward stimulus oscillation, the subjects can perfectly synchronise the mouse movement with the stimulus oscillation.

9. Temporal position marker For a stream of dynamic events, we cannot specify the temporal location of an arbitrarily defined point in time. The visual system assigns temporal markers to salient changes, by means of which temporal localisation judgements are made. A temporal position marker is a representation of event timing. It is distinct from, but somehow bound to, the representation of event content. To explain these complex results, we propose an account in terms of temporal position marker. (click) For a stream of dynamic events, we cannot specify the temporal location of an arbitrarily defined point in time. (click) The visual system assigns temporal markers to salient changes, by means of which temporal localisation judgements are made. (click) A temporal position marker is a representation of event timing. It is distinct from, but somehow bound to, the representation of event content.To explain these complex results, we propose an account in terms of temporal position marker. (click) For a stream of dynamic events, we cannot specify the temporal location of an arbitrarily defined point in time. (click) The visual system assigns temporal markers to salient changes, by means of which temporal localisation judgements are made. (click) A temporal position marker is a representation of event timing. It is distinct from, but somehow bound to, the representation of event content.

10. First-order change / second-order change Basic attribute: attribute that can be defined at single point in time Colour, Position First-order change: temporal change in a basic attribute Colour change, Position change (motion) Second-order change: change of a first-order variation Acceleration, Direction reversal For salient stimulus changes to which position markers are assigned, a distinction between first-order and second-order changes is important. (click) (click) First-order change is a temporal change in a basic attribute that can be defined at a single point in time, thus colour change and position change are first-order changes. (click) Second-order change is a change of a first-order variation, such as acceleration and direction reversal. (click) (click) (click) We conjecture that second-order changes are not available at rapid stimulus changes. For salient stimulus changes to which position markers are assigned, a distinction between first-order and second-order changes is important. (click) (click) First-order change is a temporal change in a basic attribute that can be defined at a single point in time, thus colour change and position change are first-order changes. (click) Second-order change is a change of a first-order variation, such as acceleration and direction reversal. (click) (click) (click) We conjecture that second-order changes are not available at rapid stimulus changes.

11. Marker misassignment generates motion delay (1) Our hypothesis is that marker misassignment for rapid direction reversal generates perceptual motion delay. For slow changes or single transitions, markers for colour changes are assigned to first-order changes. (click) Markers for motion changes are assigned to second-order changes, i.e., direction reversals, since they are the only salient temporal landmarks. (click) Thus accurate temporal judgement is possible. Our hypothesis is that marker misassignment for rapid direction reversal generates perceptual motion delay. For slow changes or single transitions, markers for colour changes are assigned to first-order changes. (click) Markers for motion changes are assigned to second-order changes, i.e., direction reversals, since they are the only salient temporal landmarks. (click) Thus accurate temporal judgement is possible.

12. Marker misassignment generates motion delay (2) For rapid changes, markers for colour changes are again assigned to first-order changes. (click) For motion changes, both first-order changes and second-order changes can be landmarks, but markers are assigned to more salient first-order changes. (click) Matching between first-order colour changes and first-order position changes produces relative motion delay. For rapid changes, markers for colour changes are again assigned to first-order changes. (click) For motion changes, both first-order changes and second-order changes can be landmarks, but markers are assigned to more salient first-order changes. (click) Matching between first-order colour changes and first-order position changes produces relative motion delay.

13. A spatial analogue of temporal asynchrony This is a spatial analogue of temporal asynchrony. For low spatial frequencies, the matching of first-order colour changes with second-order luminance changes is easy. For high spatial frequencies, however, it is very difficult, and we cannot help match the same order of changes between colour and luminance.This is a spatial analogue of temporal asynchrony. For low spatial frequencies, the matching of first-order colour changes with second-order luminance changes is easy. For high spatial frequencies, however, it is very difficult, and we cannot help match the same order of changes between colour and luminance.

14. How can temporal judgements be accurate for a direction change embedded in a rapid sequence? Subjects can specify the temporal location of less salient second-order changes when the task allows them to attend to a single change in the sequence. Why no corresponding difference in reaction time? Markers are the perceptual estimates of the time of occurrence of the external events. Markers are the basis of the subjective experience of the passage of time. Markers are not necessarily related to objective time course of neural processing. Marker misassignment generates motion delay (3) (click) How can temporal judgements be accurate for a direction change embedded in a rapid sequence? This is because subjects can specify the temporal location of less salient second-order changes when the task allows them to attend to a single change in the sequence. (click) Why is there no corresponding difference in reaction time? Markers are the perceptual estimates of the time of occurrence of the external events. Markers are the basis of the subjective experience of the passage of time. But markers are not necessarily related to objective time course of neural processing.(click) How can temporal judgements be accurate for a direction change embedded in a rapid sequence? This is because subjects can specify the temporal location of less salient second-order changes when the task allows them to attend to a single change in the sequence. (click) Why is there no corresponding difference in reaction time? Markers are the perceptual estimates of the time of occurrence of the external events. Markers are the basis of the subjective experience of the passage of time. But markers are not necessarily related to objective time course of neural processing.

15. Why CAN�T button press be synchronised with rapid direction change? Task requires temporal matching of first-order change (button press) with second-order change (stimulus direction reversal). Why CAN mouse move be synchronised? Task requires temporal matching of first-order change (forward/backward mouse movement) with first-order change (upward/downward stimulus movement). Marker misassignment generates motion delay (4) (click) Why CAN�T button press be synchronised with rapid direction change? This is because the task requires temporal matching of first-order change (button press) with second-order change (stimulus direction reversal). (click) The why CAN mouse move be synchronised? This is because the task requires temporal matching of first-order change (forward/backward mouse movement) with first-order change (upward/downward stimulus movement).(click) Why CAN�T button press be synchronised with rapid direction change? This is because the task requires temporal matching of first-order change (button press) with second-order change (stimulus direction reversal). (click) The why CAN mouse move be synchronised? This is because the task requires temporal matching of first-order change (forward/backward mouse movement) with first-order change (upward/downward stimulus movement).

16. Experiment: Four stimulus conditions If our hypothesis is correct, critical factor is the order of stimulus changes rather than the type of stimulus attributes. To test this, we made all four combinations of first-order and second-order changes of colour and position, then replicated the Moutoussis and Zeki�s original experiment with these stimuli.If our hypothesis is correct, critical factor is the order of stimulus changes rather than the type of stimulus attributes. To test this, we made all four combinations of first-order and second-order changes of colour and position, then replicated the Moutoussis and Zeki�s original experiment with these stimuli.

17. Tasks and predictions (click) The first condition is the combination of first-order colour change, and first-order position change. Subjects� task was to judge when the colour was red, whether the position was top or bottom. Since the task required matching between first-order changes, no perceptual delay was expected. (click) The second condition is the combination of first-order colour change, and second-order position change. Subjects� task was to judge when the colour was red, whether the movement was upward or downward. This is the same as the original Moutoussis and Zeki�s experiment, and perceptual motion delay was expected. (click) The third condition is the combination of second-order colour change, and first-order position change. Subjects� task was to judge when the colour was changing from grey to red, whether the position was top or bottom. In this case, matching between first-order changes would produce perceptual colour delay. (click) The final condition is the combination of second-order colour change, and second-order position change. Subjects� task was to judge when the colour was changing from grey to red, whether the movement was upward or downward. Since the task can be done by matching between first-order changes, no perceptual delay was expected. (click) The first condition is the combination of first-order colour change, and first-order position change. Subjects� task was to judge when the colour was red, whether the position was top or bottom. Since the task required matching between first-order changes, no perceptual delay was expected. (click) The second condition is the combination of first-order colour change, and second-order position change. Subjects� task was to judge when the colour was red, whether the movement was upward or downward. This is the same as the original Moutoussis and Zeki�s experiment, and perceptual motion delay was expected. (click) The third condition is the combination of second-order colour change, and first-order position change. Subjects� task was to judge when the colour was changing from grey to red, whether the position was top or bottom. In this case, matching between first-order changes would produce perceptual colour delay. (click) The final condition is the combination of second-order colour change, and second-order position change. Subjects� task was to judge when the colour was changing from grey to red, whether the movement was upward or downward. Since the task can be done by matching between first-order changes, no perceptual delay was expected.

18. Results (1) This is a hypothetical result predicted from our hypothesis. Horizontal axis is the relative time lag between colour and position changes, and the vertical axis is the rate of subject�s choice of the in-phase combination. No delay was expected for the top and bottom conditions, and the delays in the opposite directions were expected for the second and third conditions. (click) Here is the result of a naive observer, which is similar to our prediction. This is a hypothetical result predicted from our hypothesis. Horizontal axis is the relative time lag between colour and position changes, and the vertical axis is the rate of subject�s choice of the in-phase combination. No delay was expected for the top and bottom conditions, and the delays in the opposite directions were expected for the second and third conditions. (click) Here is the result of a naive observer, which is similar to our prediction.

19. Results (2) Here are the results of other two naive observers.Here are the results of other two naive observers.

20. Results (3) Here are my data. Colour and position were changed in the same objects in the left graph, and in separate objects in the right graph.Here are my data. Colour and position were changed in the same objects in the left graph, and in separate objects in the right graph.

21. Summary of the results Predictions supported Shift size was smaller than 90�presumably due to the residual effect of second-order changes. This graph shows the point of perceptual synchrony for each stimulus condition for four subjects. Dotted line is the predicted value for each condition. (click) In general, our predictions were supported. (click) The shift sizes for cross matching conditions were smaller than 90 deg. This is presumably due to the residual effects of second-order changes.This graph shows the point of perceptual synchrony for each stimulus condition for four subjects. Dotted line is the predicted value for each condition. (click) In general, our predictions were supported. (click) The shift sizes for cross matching conditions were smaller than 90 deg. This is presumably due to the residual effects of second-order changes.

22. Summary of the results (2) Position changes were more salient than colour changes. If they were always used as reference, attentional gating predicts the observed bias. (click) (click) There are general bias in the upward direction in this figure. (click) In our stimuli, position changes were more salient than colour changes. If they were always used as reference, attentional gating predicts the observed bias. (click) (click) In effect, the bias was reduced when the strategy to use the changes in colour as reference was facilitated by changing colour and position in separate objects. (click) (click) There are general bias in the upward direction in this figure. (click) In our stimuli, position changes were more salient than colour changes. If they were always used as reference, attentional gating predicts the observed bias. (click) (click) In effect, the bias was reduced when the strategy to use the changes in colour as reference was facilitated by changing colour and position in separate objects.

23. Conclusion Perceptual asynchrony of colour and motion Not due to neural processing time differences. But due to inappropriate matching between first-order changes -- colour change matches with position change (motion) rather than direction change. The visual system assigns temporal markers to salient changes, by means of which temporal judgements are made. In conclusion, Perceptual asynchrony of colour and motion is not due to neural processing time differences, but due to inappropriate matching between first-order changes. That is, colour change matches with position change rather than direction change. In addition, the visual system assigns temporal markers to salient changes, by means of which temporal judgements are made. In conclusion, Perceptual asynchrony of colour and motion is not due to neural processing time differences, but due to inappropriate matching between first-order changes. That is, colour change matches with position change rather than direction change. In addition, the visual system assigns temporal markers to salient changes, by means of which temporal judgements are made.

24. Appendix: Stimulus & Apparatus A pair of plaids 0.7 c/deg 100% contrast Colour change grey - red (Eq-L) Position change Up-down 0.5deg/jump 6 deg/sec Apparatus VSG2/3 120Hz frame rate