310 likes | 313 Views
Representation in Vision. Derek Hoiem CS 598, Spring 2009 Jan 22, 2009. Pipeline for Prediction. Imagery. Representation. Classifier. Predictions. Representation is Key. Imagery. Representation. Classifier. Predictions. Example: 4’s and 9’s. General Principles of Representation.
E N D
Representation in Vision Derek Hoiem CS 598, Spring 2009 Jan 22, 2009
Pipeline for Prediction Imagery Representation Classifier Predictions
Representation is Key Imagery Representation Classifier Predictions Example: 4’s and 9’s
General Principles of Representation • Coverage • Ensure that all relevant info is captured • Concision • Minimize number of features without sacrificing coverage • Directness • Ideal features are independently useful for prediction
Right features depend on what you want to know • Shape: scene-scale, object-scale, detail-scale • 2D form, shading, shadows, texture, linear perspective • Material properties: albedo, feel, hardness, … • Color, texture • Motion • Optical flow, tracked points • Distance • Stereo, position, occlusion, scene shape • If known object: size, other objects
Cues for Shape • Shading • Most effective at object and detail level
Cues for Shape • Shading
Cues for Shape • Shading
Cues for Shape • Shading
Cues for Shape • Shading
Cues for Shape • Shading
Cues for Shape • Shading
Cues for Shape • Shading • Perspective
Cues for Shape • Shading • Perspective • Boundaries/Form
Cues for Shape • Shading • Perspective • Boundaries/Form • Shadows
Features for Shape • Shading • Image filter responses, intensity A good face detector LM Filter Bank
Features for Shape • Histograms of gradient SIFT – Lowe IJCV 2004 HOG – Dalal Triggs 2005
Features for Shape • Detected boundaries Canny Edge Detector
Features for Shape • Perspective Hoiem Efros Hebert 2005
Features for Shape Texture Location Color Perspective • Indirect cues Hoiem Efros Hebert 2005
Features for Material • Color • Texture (filter banks or HOG over regions) L*a*b* color space HSV color space
Features: Motion Efros et al 2003 Optical Flow Zitnick et al. 2005
Computing Features Compute Features over Image Quantize (Optional) Choose Spatial Support Compute Statistics of Features within Spatial Support 71% 29% Histogram Bin Features RGB Values Quantized to 10 Levels
Big Issue: Spatial Support “superpixels” Image from Mori 2005 multiple segmentations Image from Hoiem et al. 2007 regions from segmentation Image from Jianbo Shi (ncuts)
Things to remember • Think about the right features for the problem • Coverage • Concision • Directness • Think about what features represent • Spatial support, statistics of features important
Ranking of Feature Importance by European Art Byzantine: “Christ as the Good Shephard” 425AD Giotto: “The Mourning of Christ” 1305AD
Representing Shape • Assumption that light comes from the top