Saliency & attention (P)

1. Saliency & attention (P) • Lavanya Sharan • April 4th, 2011

2. Looking for peopleEhinger et al. (2009) Question: Where do people look and can we model it?

3. Looking for peopleEhinger et al. (2009) • 14 observers with tracking accuracy 0.75 deg. • Observers asked to look for pedestrians as fast as possible and answer. • 912 images. Target (pedestrian) present in half the images. • Images 23.5 x 17.7 deg, target when present 0.9 x 1.8 deg.

4. Inter-observer agreementEhinger et al. (2009) • Question: Do observers agree with each other? • Use fixations from n-1 observers for one image, apply gaussian blur and use this region to predict fixation of remaining observer on that image. Repeat for n observers (Torralba et al. 2006). • Do this for all images.

5. Inter-observer agreementEhinger et al. (2009)

6. Cross-image controlEhinger et al. (2009) • Question: Do fixated locations depend on image content? Or do observers always look at the same places? • Use fixations from n-1 observers for one image, apply gaussian blur and use this region to predict fixation of remaining observer on a different, randomly selected image. Repeat for n observers (Torralba et al. 2006). • Do this for all images.

7. Performance at predicting fixation locationsEhinger et al. (2009) Inter-observer agreement (Target absent AUC=0.93, target present AUC = 0.95) Cross-image control (Target absent AUC=0.68, target present AUC = 0.62)

8. Modeling eye movements during pedestrian detectionEhinger et al. (2009)

9. Modeling eye movements during pedestrian detectionEhinger et al. (2009) • Low-level saliency. Use Torralba et al. 2006 model. • Target features. Use Dalal et al. 2005 person detector. • Scene context. Use Torralba et al. model but train on context (i.e. sidewalks not skies are good locations) + ‘Context Oracle’ (7 observers indicate good candidates for locations) • Use validation set to decide relative weighting.

10. Modeling eye movements during pedestrian detectionEhinger et al. (2009)

11. Modeling eye movements during pedestrian detectionEhinger et al. (2009) Performance of saliency component

12. Modeling eye movements during pedestrian detectionEhinger et al. (2009) Performance of target features component

13. Modeling eye movements during pedestrian detectionEhinger et al. (2009) Performance of scene context component

14. Modeling eye movements during pedestrian detectionEhinger et al. (2009) Performance of full model

15. What does the model get wrong?Ehinger et al. (2009)