introduction to image processing and computer vision n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Introduction to Image Processing and Computer Vision PowerPoint Presentation
Download Presentation
Introduction to Image Processing and Computer Vision

Loading in 2 Seconds...

play fullscreen
1 / 48

Introduction to Image Processing and Computer Vision - PowerPoint PPT Presentation


  • 188 Views
  • Uploaded on

Introduction to Image Processing and Computer Vision. Rahul Sukthankar Intel Research Laboratory at Pittsburgh and The Robotics Institute, Carnegie Mellon rahuls@cs.cmu.edu. Image Processing vs. Computer Vision. Image processing: Image  image

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Introduction to Image Processing and Computer Vision


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
    Presentation Transcript
    1. Introduction toImage Processing andComputer Vision Rahul Sukthankar Intel Research Laboratory at Pittsburgh and The Robotics Institute, Carnegie Mellon rahuls@cs.cmu.edu

    2. Image Processing vs. Computer Vision • Image processing: • Image  image • e.g., de-noising, compression, edge detection • Computer vision: • Image  symbols • e.g., face recognition, object tracking • Most real-world applications combine techniques from both categories

    3. Outline • Operations on a single image • Operations on an image sequence • Multiple cameras • Extracting semantics from images • Applications

    4. Outline • Operations on a single image • Operations on an image sequence • Multiple cameras • Extracting semantics from images • Applications

    5. What is an Image? • 2D array of pixels • Binary image (bitmap) • Pixels are bits • Grayscale image • Pixels are scalars • Typically 8 bits (0..255) • Color images • Pixels are vectors • Order can vary: RGB, BGR • Sometimes includes Alpha

    6. What is an Image? • 2D array of pixels • Binary image (bitmap) • Pixels are bits • Grayscale image • Pixels are scalars • Typically 8 bits (0..255) • Color images • Pixels are vectors • Order can vary: RGB, BGR • Sometimes includes Alpha

    7. What is an Image? • 2D array of pixels • Binary image (bitmap) • Pixels are bits • Grayscale image • Pixels are scalars • Typically 8 bits (0..255) • Color images • Pixels are vectors • Order can vary: RGB, BGR • Sometimes includes Alpha

    8. What is an Image? • 2D array of pixels • Binary image (bitmap) • Pixels are bits • Grayscale image • Pixels are scalars • Typically 8 bits (0..255) • Color images • Pixels are vectors • Order can vary: RGB, BGR • Sometimes includes Alpha

    9. What is an Image? • 2D array of pixels • Binary image (bitmap) • Pixels are bits • Grayscale image • Pixels are scalars • Typically 8 bits (0..255) • Color images • Pixels are vectors • Order can vary: RGB, BGR • Sometimes includes Alpha

    10. Canny Edge Detector cvCanny(…) Images courtesy of OpenCV tutorial at CVPR-2001

    11. Morphological Operations • Simple morphological operations on binary images: • erosion: any pixel with 0 neighbor becomes 0 • dilation: any pixel with 1 neighbor becomes 1 • Compound morphological operations:(composed of sequences of simple morphological ops) • opening • closing • morphological gradient • top hat • black hat • Aside: what is the “right” definition of “neighbor”?

    12. Morphological Operations Dilatation IB Image I Erosion IB Opening IoB= (IB)B Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB)-I Grad(I)= (IB)-(IB) Images courtesy of OpenCV tutorial at CVPR-2001

    13. Hough Transform Goal: Finding straight lines in an edge image Canny edge + Hough xformcvHoughLines(…) Original image Images courtesy of OpenCV tutorial at CVPR-2001

    14. Distance Transform • Distance for all non-feature points to closest feature point cvDistTransform(…) Images courtesy of OpenCV tutorial at CVPR-2001

    15. Flood Filling cvFloodFill(…) grows from given seed point Images courtesy of OpenCV tutorial at CVPR-2001

    16. Image Statistics • Statistics are used to summarize the pixel values in a region, typically before making a decision • Some statistics are computed over a single image: • Mean and standard deviation: cvAvg(…), cvAvgSdv(…) • Smallest and largest intensities: cvMinMaxLoc(…) • Moments: cvGetSpatialMoment(…), cvGetCentralMoment(…) • Others are computed over pairs/differences of images: • Distances/norms C, L1, L2: cvNorm(…), cvNormMask(…) • Others are computed over pairs/differences of images: • Histograms: • Multidimensional histograms: (many functions to create/manipulate) • Earth mover distance – compare histograms: cvCalcEMD(…)

    17. Gaussian and Laplacian pyramids Image segmentation by pyramids Image Pyramids:Coarse to Fine Processing Images courtesy of OpenCV tutorial at CVPR-2001

    18. Image Pyramids:Coarse to Fine Processing Original image Gaussian Laplacian Images courtesy of OpenCV tutorial at CVPR-2001

    19. Pyramid-based Color Segmentation Images courtesy of OpenCV tutorial at CVPR-2001

    20. Outline • Operations on a single image • Operations on an image sequence • Multiple cameras • Extracting semantics from images • Applications

    21. Background Subtraction • Useful when camera is still and background is static or slowly-changing (e.g., many surveillance tasks) • Basic idea: subtract current image from reference image. Regions with large differences correspond to changes. • OpenCV supports several variants of image differencing: • Average • Standard deviation • Running average: cvRunningAvg(…) • Can follow up with connected components (segmentation): • could use “union find” or floodfill: cvFloodFill(…)

    22. Optical Flow • Goal: recover apparent motion vectors between a pair of images -- usually in a video stream • Several optical flow algorithms are available: • Block matching technique: cvCalcOpticalFlowBM(…) • Horn & Schunck technique: cvCalcOpticalFlowHS(…) • Lucas & Kanade technique: cvCalcOpticalFlowLK(…) • Pyramidal LK algorithm: cvCalcOpticalFlowPyrLK(…)

    23. Active Contours:Tracking by Energy Minimization • Snake energy: • Internal energy: • External energy: cvSnakeImage(…) Images courtesy of OpenCV tutorial at CVPR-2001

    24. Real cameras exhibit radial & tangential distortion: causes problems for some algorithms. First, calibrate by showing a checkerboard at various orientations:cvFindChessBoardCornerGuesses() Then apply an undistorting warp to each image (don’t use a warped checkerboard!)cvUndistort(…) If the calibration is poor, the “undistorted” image may be worse than the original. Camera Calibration Images courtesy of OpenCV tutorial at CVPR-2001

    25. Outline • Operations on a single image • Operations on an image sequence • Multiple cameras • Extracting semantics from images • Applications

    26. Stereo Vision • Extract 3D geometry from multiple views • Points to consider: • feature- vs area-based • strong/weak calibration • processing constraints • No direct support in OpenCV, but building blocks for stereo are there.

    27. View Morphing Images courtesy of OpenCV tutorial at CVPR-2001

    28. Outline • Operations on a single image • Operations on an image sequence • Multiple cameras • Extracting semantics from images • Applications

    29. Face Detection Images courtesy of Mike Jones & Paul Viola

    30. Large Scale Painful! Small Scale Classical Face Detection Images courtesy of Mike Jones & Paul Viola

    31. Viola/Jones Face Detector • Technical advantages: • Uses lots of very simple box features, enabling an efficient image representation • Scales features rather than source image • Cascaded classifier is very fast on non-faces • Practical benefits: • Very fast, compact footprint • You don’t have to implement it!(should be in latest version of OpenCV)

    32. Principal Components Analysis High-dimensional data Lower-dimensional subspace cvCalcEigenObjects(…) Images courtesy of OpenCV tutorial at CVPR-2001

    33. PCA for Object Recognition Images courtesy of OpenCV tutorial at CVPR-2001

    34. PCA for Object Recognition Images courtesy of OpenCV tutorial at CVPR-2001

    35. Outline • Operations on a single image • Operations on an image sequence • Multiple cameras • Extracting semantics from images • Applications

    36. Shadow Elimination Idea: remove shadows from projected displays using multiple projectors OpenCV Techniques: Image differencing Image warping Convolution filters Matrix manipulation PosterCam Idea: put cameras in posters and identify who reads which poster OpenCV Techniques: Face detection Face recognition Unsupervised clustering Examples of Simple Vision Systems

    37. Single Projector: Severe Shadows display screen P

    38. Two Projectors: Shadows Muted display screen P-1 P-2

    39. Dynamic Shadow Elimination display screen camera P-1 P-2

    40. display screen camera P-1 P-2 Shadow Elimination: Challenges • Occlusion detection: what does a shadow look like? • Geometric issues: which projectors are occluded? • Photometric issues: how much light removes a shadow? • Performance: how can we do this in near real-time?

    41. display screen camera P-1 P-2 Shadow Elimination: Solutions • Occlusion detection: difference image analysis • Geometric issues: single shadow-mask for all projectors! • Photometric issues: uncalibrated – feedback system • Performance: only modify texture map alpha values

    42. Shadow Removal witha Single Mask

    43. Shadow Elimination Algorithm Camera images Projected

    44. PosterCam Overview • PosterCam Hardware: • Camera in each poster • Embedded computer in each poster (~ iPAQ) • Network connection to other posters

    45. PosterCam Details • Face detection: Viola/Jones (no float ops) • Lighting compensation:histogram equalization • Pose variation:additional synthetic faces • Unsupervised clustering:k-means and nearest neighbor with non-standard distance metric

    46. Tips on Image Processing and Coding with OpenCV • Use the OpenCV documentation only as a guide(it is inconsistent with the code) • Read cv.h before writing any code • OpenCV matrix functions work on images: e.g., cvSub(…) • Beware camera distortion: cvUnDistort(…) may help • Beware illumination changes: • disable auto gain control (AGC) in your camera if you are doing background subtraction • histogram equalization (often good for object recognition) • Image processing algorithms may require parameter tuning: collect data and tweak until you get good results

    47. Digital Image ProcessingGonzalez & Woods,Addison-Wesley 2002 Computer VisionShapiro & Stockman,Prentice-Hall 2001 Computer Vision: A Modern ApproachForsyth & Ponce,Prentice-Hall 2002 Introductory Techniques for 3D Computer VisionTrucco & Verri,Prentice-Hall 1998 Reference Reading

    48. The End Acknowledgments • Significant portions of this lecture were derived from the Intel OpenCV tutorial by Gary Bradski et al. at CVPR-2001 • Thanks to my former colleagues at Compaq/HP CRL for additional slides and suggestions: Tat-Jen Cham, Mike Jones, Vladimir Pavlovic, Jim Rehg, Gita Sukthankar, Nuno Vasconcelos, Paul Viola Contact rahuls@cs.cmu.edu if you need more information