Biologically Inspired Intelligent Systems

1. Biologically Inspired Intelligent Systems Lecture 07 Dr. Roger S. Gaborski

2. Feature Maps • Intensity contrast (HW#3) • Orientation (HW#4) • 0, 45, 90 and 135 degrees • Color Information (HW#5) • Red-Green opponent color • Blue-Yellow opponent color

3. Processing of Color Data • How is color coded in primates? • Three classes of cone photoreceptors • Arrangement of cone types seems to be random • Color opponent cells (cone signals brought together in opposition) • Red-green • Blue-yellow

4. Red-Green RF White overlap Red in center Green in surround | | | | | | | | | | | | | | | | | | | | |

5. Red-Green RF Green in center Red in center Green in center Green in surround Red in surround | | | | | | ||||||||||||||||| NO RESPONSE

6. Blue-Yellow RF Same operation as Red- Green

7. Color Channels (approximation) • Create broadly tuned color channels: R = r-(g+b)/2 G = g- (r+b)/2 B = b- (r+g)/2 Y = r+g – 2(|r-g| + b) (negative values set to zero) • Maximum response to pure hue the channel is tuned to (if a pixel contains both r and g it will have a smaller R response than if it only had r) • Zero response to white or black inputs

8. Color Difference Map • Create red-green, blue-yellow color channels • For each color plane, apply receptive fields at different scales, 7x7, 15x15, 31x31, 63x63… • Combine color receptive fields output to form color difference Map

9. Potential Red-Green RF Model Create Dog64 Red_Kernel: Green_Kernel R_ctr=filter red data with Red_Kernel G_sur=filter green data with Green_Kernel Form difference: R_ctr- G_sur RED GREEN GREEN

10. Note Colorbar Center Surround

11. Color Image

12. Gray Scale – “Lack of Interest”

15. Color opponent filters Red center Green surround Yellow center Blue surround Green center Red surround Blue center Yellow surround Gabor orientation filters Difference of Gaussian filters (Intensity contrast) On CenterOff Center 0 degree 45 degree 90 degree 135 degree Still Attention Processing Image

16. Still Attention Processing Orientation Salience map Color Salience map Intensity Contrast Salience map

17. Still Attention Module • Combine • Normalized Intensity Contrast Map • Normalized Orientation Maps • Normalized Color difference Map to form Still Attention Module

18. Orange trail marker no longer stands out as it does in color image

25. Partial model WHERE n = 0, 45, 90 and 135 degrees n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 31x31 Gabor n degrees 7x7 Gabor n degrees 15x15 Gabor n degrees 15x15 Gabor R-G B-Y Contrast Images imCon8 imCon16 imCon32 Retina Model 8x8, 16x16 and 32x32 circular receptive fields Color Opponent Image

26. Motion Detection • Types of motion • Object • Optical flow

27. Motion Detection • Types of motion • Object: Visual area V5 or MT (middle temporal), • Optical flow (MSTd –medial superior temporal/dorsal)

28. A Sense of time • We need temporal information to detect motion • Computer vision techniques: • Frame differencing • Pixel modeling (Gaussian) • ‘system’ level biological models

29. Motion- consider a bar moving to the right y x At time t0 bar is moving to the right

30. 3D Representation of moving bar y x t

31. (x,t) Plot – Ignore y axis x Slope of blue bar represents the velocity that the bar is moving Can you relate this plot to a stationary bar with the same slope?? t Since object is moving vertical bar, nothing is lost with this representation

32. Stationary bar detection (x,y) x Plot of a stationary bar y MOTION AS ORIENTATION

33. Stationary bar detection (x,y) x Plot of a stationary bar - + - + - + y Simple cell response

34. Moving bar detection (x,t) x Plot of a moving bar - + - + - + t TIME Spatial temporal receptive field

35. FASTER Moving bar detection (x,t) x Faster plot of a moving bar (moves more x distance in same amount of time t Spatial temporal (space and time) receptive field We need a separate spatiotemporal for each velocity

36. “Gaussian Derivative Model for spatial-temporal vision” (R.A. Young) • GENERAL EQUATION • Gn,o,p(x', y', t') = gn(x')go(y')gp(t')  for n = 0,1 2,... • G - 3 D GD spatial-temporal filterx', y' and t' are the coordinate axes of Gg are one dimensional GD functionsn, o, and p are derivative numbers along x', y' and t' • Additional parameters allow placement of model, spatial orientation, optimal speed (see paper for details)

37. M1n with no theta or phi component should detect a stationary edge image at 0 degrees Red is negative, Blue is positive

38. (x,y) only

39. M1ntheta should detect a stationary edge at theta degrees Theta = 45 degrees Red is negative, blue is positive

40. (x,y) only

41. M1ntheta_phi should detect a moving edge at theta degrees moving at a rate phi Theta = 45 degrees Phi = 45 degrees Red is negative, Blue is positive

42. (x,y) only

43. M2n with no theta or phi component should detect a stationary line image at 0 degrees

44. (x,y) only

45. M2ntheta should detect a stationary line at theta degrees

46. (x,y) only

47. M2ntheta_phi should detect a moving line at theta degrees moving at a rate phi

48. (x,y) only

49. Experimental ResultsFilter detects stationary edge at 0 degrees Note magnitude values for M1n For plane1, Max = .1 Min = -.1 For plane 21, Max = 1 Min = -1 Data block constant for all planes

50. Convolve each plane of data with corresponding filter plane Result of filter would be summation of all the data plane responses