keithmay

1. Advanced Motion Keith May keith.may@ucl.ac.uk www.keithmay.org

2. where x is distance travelled and t is time taken • Speed is the gradient of the plot of distance against time What is motion? • Change in position over time What is speed? • Rate of change of position • Distance travelled in a unit of time • e.g. miles/hour, metres/second

3. Motion as a vector • Motion has speed and direction 1 metre • Represented as a vector, which gives distance and direction travelled in unit time (e.g. one second) • Length of the vector gives the distance travelled • Orientation of the vector gives the direction travelled

4. Speed of each feature given by Motion perception by feature tracking • Identify features in the retinal image at one point in time • Identify the same features in the retinal image a short time (t) later • See how far (x) each feature has moved • Direction of motion given by direction of displacement of the feature • In this kind of model, the key problem to be solved is the correspondence problem: which features at time t1 correspond to which features at time t2? • Until around 1980, many researchers on motion perception thought about motion in this way (e.g. Ullman, 1979) • But the correspondence problem is hard to solve • Modern models of low-level motion perception work directly on the image, and bypass the correspondence problem altogether

7. Missing fundamental motion illusion 1st frame 2nd frame

8. Missing fundamental motion illusion 1st frame 2nd frame Adelson & Bergen (1985)

9. Adelson & Bergen (1985) Edward H. Adelson

10. x t • Need receptive fields that are tiled in space-time Space-time from Adelson & Bergen (1985) • Motion is tilt in space-time • We can detect it using analogous methods to processing orientation in space

11. Spatial receptive field gives the neuron’s preferred image (See DeAngelis, Ohzawa & Freeman, 1993) http://ohzawa-lab.bpe.es.osaka-u.ac.jp/ohzawa-lab/teaching/RF/XTinseparable.html • In reality, a neuron has a preferred “movie”, the “spatiotemporal receptive field” Spatiotemporal receptive fields

12. In reality, a neuron has a preferred “movie”, the “spatiotemporal receptive field” Spatiotemporal receptive fields • Spatial receptive field gives the neuron’s preferred image (See DeAngelis, Ohzawa & Freeman, 1993) http://ohzawa-lab.bpe.es.osaka-u.ac.jp/ohzawa-lab/teaching/RF/XTinseparable.html

14. Put each neuron’s output through a squaring function Energy model (Adelson & Bergen, 1985) x x x x • At each point in the image, have simple cells with four types of receptive field t t t t

15. Energy model (Adelson & Bergen, 1985) x x x x • At each point in the image, have simple cells with four types of receptive field t t t t • Put each neuron’s output through a squaring function • Add the squared outputs of the two rightward-selective neurons to give rightward motion energy • This is a reasonable model of complex cells • Do the same with the two leftward-selective neurons • Then subtract leftward from rightward energy to give opponent energy

16. Linear receptive field followed by squaring is a model of simple cells • Adding outputs of simple cells with odd- and even-symmetric receptive fields is a model of complex cells • Qian & Andersen (1994) found evidence for motion opponency in MT Energy model (Adelson & Bergen, 1985) x x x x t t t t • Emerson, Bergen & Adelson (1992) showed that responses of direction-selective complex cells in V1 of cat behaved much like non-opponent motion energy stage

17. Energy model (Adelson & Bergen, 1985) x x x x • Opponent energy signal increases with signal contrast t t t t • To convert to a pure measure of velocity, divide the opponent energy by a static energy signal, S2, from neurons with non-directional receptive fields • This results in a largely contrast-invariant velocity signal, like MT responses, which are selective for velcocity but don’t vary much with contrast (Rodman & Albright, 1987; Sclar, Maunsell & Lennie, 1990)

18. Reichardt detector (Reichardt, 1957, 1961) Photoreceptors 0 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

19. Photoreceptors 0 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

20. Photoreceptors 10 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

21. Photoreceptors 10 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

22. Photoreceptors 10 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

23. Photoreceptors 0 spikes 10 spikes Delay 10 spikes Multiplication 0 spikes Rightward motion

24. Photoreceptors 0 spikes 10 spikes Delay 10 spikes Multiplication 100 spikes Rightward motion

25. Photoreceptors 0 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

26. Photoreceptors 0 spikes 10 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

27. Photoreceptors 0 spikes 10 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

28. Photoreceptors 0 spikes 10 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

29. Photoreceptors 10 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

30. Photoreceptors 10 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

31. Photoreceptors 0 spikes 0 spikes Delay 10 spikes Multiplication 0 spikes Rightward motion

32. Photoreceptors 0 spikes 0 spikes Delay 0 spikes Multiplication 0 spikes Rightward motion

33. Photoreceptors Delay Delay Multiplication Leftward motion Rightward motion

36. Aliasing • Happens with movies of periodic patterns when the temporal sampling is too coarse (not enough frames per second) • With Reichardt motion detector, we can also get aliasing in real-world vision (i.e. not movies) when the delay is too long, or the spatial sampling is too coarse (photoreceptors too far apart) • This is a good model of motion perception in insects because there is evidence that aliasing occurs in insect vision • One theory of why zebras have their stripes is that the periodic stripe pattern gives rise to aliasing in the visual systems of biting insects, making them less likely to land on the zebra (How & Zanker, 2014) • But humans and other mammals show little evidence of aliasing • van Santen & Sperling (1984) introduced a fix to the Reichardt detector to prevent aliasing • “Elaborated Reichardt detector”

37. Photoreceptors Delay Delay Multiplication Leftward motion Rightward motion

38. Input receptive fields have same location, but different phase Photoreceptors Delay Delay Multiplication Leftward motion Rightward motion

39. Elaborated Reichardt vs Energy Model x x x x • Opponent Energy stage of the Energy Model is mathematically equivalent to output of Elaborated Reichardt Detector (see Adelson & Bergen, 1985) t t t t • The output the same, but intermediate stages different • Intermediate stages of the Energy Model are closer to mammalian physiology (Emerson et al, 1992) • Energy model also has final normalization stage, which gives contrast-independent measure of velocity

40. What is motion? • Change in position over time What is speed? • Rate of change of position • Distance travelled in a unit of time • where x is distance travelled and t is time taken • e.g. miles/hour, metres/second • Speed is the gradient of the plot of distance against time

41. Gradients • Intensity profile

42. Gradients • Intensity profile • Intensity gradient =

43. Gradients • Intensity profile • Intensity gradient = • With curved profile, gradient is different at each point • Our previous definition of gradient required two points, so how do we define the gradient at a point?

44. Gradients • Intensity profile • Intensity gradient = • With curved profile, gradient is different at each point • Our previous definition of gradient required two points, so how do we define the gradient at a point? • Start with two points

45. Gradients • Intensity profile • Intensity gradient = • With curved profile, gradient is different at each point • Our previous definition of gradient required two points, so how do we define the gradient at a point? • Start with two points • Then gradually make x smaller

46. As x approaches zero, I/x approaches a limiting value, which we call the derivative, written Ix or Gradients • Intensity profile • Intensity gradient = • With curved profile, gradient is different at each point • Our previous definition of gradient required two points, so how do we define the gradient at a point? • Start with two points • Then gradually make x smaller • The derivative is interpreted as the gradient, or rate of change, at that point

47. Gradients • I is the Intensity profile

48. Gradients • I is the Intensity profile • Ix is the rate of change of I as we move across space, x

49. Gradients • I is the Intensity profile • Ix is the rate of change of I as we move across space, x • Ixx is the rate of change of Ix as we move across space, x • It is the rate of change of I as we move across time, t • Ixt is the rate of change of Ix as we move across time, t

50. Move in a positive direction across the image (rightwards) • Our velocity across the image is intensity gradient over time, It intensity gradient over space, Ix • Image velocity is Velocity from intensity gradients • In reality, our analysis mechanisms stay still, and the image content moves • Moving rightward across image is equivalent to staying still and having image move leftward (negative direction) (Fennema & Thompson, 1979)