image processing and computer vision n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Image Processing and Computer Vision PowerPoint Presentation
Download Presentation
Image Processing and Computer Vision

Loading in 2 Seconds...

play fullscreen
1 / 86

Image Processing and Computer Vision - PowerPoint PPT Presentation


  • 259 Views
  • Uploaded on

Image Processing and Computer Vision. Lecture 4, Multimedia E-Commerce Course November 5, 2002 Mike Christel (significant input by Henry Schneiderman, http://www.cs.cmu.edu/~hws). Carnegie Mellon. Copyright 2002 Michael G. Christel and Alexander G. Hauptmann. Outline.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Image Processing and Computer Vision' - sandra_john


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
image processing and computer vision

Image Processing and Computer Vision

Lecture 4, Multimedia E-Commerce Course

November 5, 2002

Mike Christel

(significant input by Henry Schneiderman, http://www.cs.cmu.edu/~hws)

CarnegieMellon

  • Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
outline
Outline
  • Defining Image Processing and Computer Vision
  • Emerging Technology
    • Digitization of documents
    • Digitization of images/photographs
    • Biometrics
    • Management of images on computers
    • Other: manufacturing, military, games, …
  • Research in Image Processing and Computer Vision
    • Automatically Finding Faces and Cars
    • Content-based Image Retrieval
image processing vs computer vision

image

image

Image Processing vs. Computer Vision
  • Image Processing
    • Research area within electrical engineering/signal processing
    • Focus on syntax,

low level features

  • Computer Vision
    • Research area within computer science/artificial intelligence
    • Focus on semantics,

symbolic or geometric

descriptions

Faces

People

Chairs

etc.

image

optical character recognition ocr
Optical Character Recognition (OCR)
  • First patent in OCR in 19th century
  • First applications in post-office and banks
  • Documents easier to distribute, search, organize, and edit in digital form
    • Typewriter has been replaced by word processor
    • Lots of legacy materials (the world’s libraries of books) available only in print
  • State of the art not perfect, but 99% accurate on cleanly printed pages
  • Examples of errors. . .
heavy print
Heavy Print

Output from 3 commercial OCR systems

processing overlaid text in video
Processing Overlaid Text in Video

Text Area Detection

Video

The Video OCR (VOCR) process used by the Informedia research group at Carnegie Mellon

Text Area Preprocessing

Commercial OCR

ASCII Text

video frames filtered frames and ed frames
Video FramesFiltered Frames AND-ed Frames

(1/2 s intervals)

CarnegieMellon

  • Copyright 2002 Michael G. Christel and Alexander G. Hauptmann
handwriting recognition
Handwriting Recognition
  • Natural progression to OCR work for print
  • Works if constraints on writer, e.g. palm pilot, where user is asked to conform to specific style or convention
other document processing
Other Document Processing
  • Not just for text. . .
  • Examples:
    • Engineering document to CAD file
    • Maps to GIS format
    • Music score to MIDI representation
outline1
Outline
  • Defining Image Processing and Computer Vision
  • Emerging Technology
    • Digitization of documents
    • Digitization of images/photographs
    • Biometrics
    • Management of images on computers
    • Other: manufacturing, military, games, …
  • Research in Image Processing and Computer Vision
    • Automatically Finding Faces and Cars
    • Content-based Image Retrieval
digital cameras convenience
Digital Cameras = Convenience
  • Easy to capture photos
  • Easy to store and organize photos
  • Easy to duplicate photos
  • Easy to edit photos
  • Rough Multimedia eCommerce class survey:
    • 1999: 10% own digital cameras
    • 2000: 25%
    • 2001: 50%
    • 2002: ??
digital camera cautions
Digital Camera Cautions

Via “Photo Industry Reporter” e-Magazine at: http://www.photoreporter.com/2002/10-21/photokina_report_look_at_35mm.html

  • Film cameras still outsell digital cameras by almost three to one
  • The household penetration of digital is at about 15%
  • “But let’s face it: film’s days are numbered. Anyone staying solely with film these days will have a glorious buggy whip in a market that will be clamoring for cars.”
digital camera growth
Digital Camera Growth
  • Photo Marketing Association on US digital camera sales:
    • 4.5 million in 2000
    • 6.9 million in 2001
    • Projected 9.3 million for 2002
    • http://www.visioneer.com/About/press/june2402.html
  • InfoTrends Research Group estimates that the U.S. photo-enabled TV set-top installed base will grow from less than 1 million units in 2002, to over 114 million units in 2006. Household penetration will climb from under 1% to around 85%.
  • InfoTrends projects digital camera sales to grow at a rate of 38% through 2003
state of the art digital cameras
State of the Art: Digital Cameras
  • Film is currently better in resolution and color
    • Professional photographers
      • Digital for low quality newspaper advertisements
      • Film for portrait photos
  • Computer storage limitations: 1 high resolution digital image = 20-25 Megabytes
    • http://pic.templetons.com/brad/photo/pixels.html
    • 3500 line pairs/35 mm or about 5000 dots/inch, but grainy
    • At 3:2 frame size, ~20 million pixels
    • Conclusion: “a 5300 x 4000 digital camera would produce a shot equivalent to a scan from a quality 35mm camera -- provided you can get more than 8 bits per pixel. …A 3000 x 2000 digital camera would match the 35mm for a good percentage of shots.”
  • Printing: home printers not comparable to commercial printers
future of digital cameras
Future of Digital Cameras
  • Improved resolution and color
  • “Smart” cameras
  • More programmable features
    • Auto-focus on object of interest
    • “Everything in focus” photo
    • Capture photo when event X occurs
outline2
Outline
  • Defining Image Processing and Computer Vision
  • Emerging Technology
    • Digitization of documents
    • Digitization of images/photographs
    • Biometrics
    • Management of images on computers
    • Other: manufacturing, military, games, …
  • Research in Image Processing and Computer Vision
    • Automatically Finding Faces and Cars
    • Content-based Image Retrieval
biometrics
Biometrics
  • Technology for identification
    • Finger/palm print
    • Iris
    • Face
fingerprints
Fingerprints
  • Minutae – spits and merges of ridges
face identification
Face Identification
  • Not quite reliable yet.
    • Performance degrades rapidly with uncontrolled lighting, facial expression, and size of database
  • Several companies exist:
    • Visionics (Rockfeller University spin-off)
    • Viisage (MIT spin-off)
    • EyeMatic (USC spin-off)
    • Miros (MIT spin-off)
    • Banque-Tec Intl (Australia)
    • C-VIS Computer Vision (Germany)
    • LAU Technologies
  • Commercial systems installed in London and Brazil to catch criminals
automatic age progression

Original Image

(1962)

Computer-Aged

(1997)

Actual Photo

(1997)

Automatic Age Progression
outline3
Outline
  • Defining Image Processing and Computer Vision
  • Emerging Technology
    • Digitization of documents
    • Digitization of images/photographs
    • Biometrics
    • Management of images on computers
    • Other: manufacturing, military, games, …
  • Research in Image Processing and Computer Vision
    • Automatically Finding Faces and Cars
    • Content-based Image Retrieval
management of images on computers
Management of images on computers
  • Compression – reducing storage size needed for images
  • Watermarking – Protecting copyright
  • Microsoft, Bell Labs, NEC, etc.

Visible watermark

photo manipulation
Photo Manipulation
  • Adobe Photoshop, Corel PhotoPaint, Pixami, PhotoIQ, etc.
  • Image editing: crop an image, adjust the color, paint over part of any image, airbrush part of an image, combine images, etc.
  • Future: Applications of computer vision, e.g., discriminating foreground from background.
online digital image collections
Online Digital Image Collections
  • Stock photos of use to graphic designers, artists, etc.
  • Large collections of images exist
    • Corbis 67 million images
    • Getty 70 million stock photography images
    • AP collects 1000s of digitized images per day
outline4
Outline
  • Defining Image Processing and Computer Vision
  • Emerging Technology
    • Digitization of documents
    • Digitization of images/photographs
    • Biometrics
    • Management of images on computers
    • Other: manufacturing, military, games, …
  • Research in Image Processing and Computer Vision
    • Automatically Finding Faces and Cars
    • Content-based Image Retrieval
inspection for manufacturing
Inspection for Manufacturing
  • Occum – inspection of printed circuit boards ($100M / year)
  • Cognex – Do-it-yourself toolkits for inspection (400 employees)
automatic target recognition atr
Automatic Target Recognition (ATR)
  • Finding mines, tanks, etc.
  • Billion dollar a year industry
    • Martin-Lockheed, TSR, Northrup-Grumman, other aerospace contractors.
  • Various types of imagery:
    • Synthetic Aperture Radar (SAR), Sonar, hyper-spectral imagery (more than 3 colors)
aerial photo interpretation
Aerial Photo Interpretation
  • Also referred to as “automated cartography”
  • Classification of land-use: forest, vegetation, water
  • Identification of man-made objects: buildings, roads, etc.
better security cameras
Better Security Cameras
  • Cameras that are responsive to the environment
    • Track and zoom on moving objects
    • Automatic adjustment of contrast
medical imagery
Medical imagery
  • Medical image libraries for study and diagnosis
  • Image overlay to guide surgeons
history
History
  • 1980’s ~100 companies – manufacturing applications mostly
  • Early 1990’s less than 10 companies
  • Late 1990’s ~100 companies – face recognition, intelligent teleconferencing, inspection, digital libraries, medical imaging
outline5
Outline
  • Defining Image Processing and Computer Vision
  • Emerging Technology
    • Digitization of documents
    • Digitization of images/photographs
    • Biometrics
    • Management of images on computers
    • Other: manufacturing, military, games, …
  • Research in Image Processing and Computer Vision
    • Automatically Finding Faces and Cars
    • Content-based Image Retrieval
image processing filtering
Image Processing: Filtering

Enhancing an image’s quality for human viewing, e.g., in medical imaging or in telescopic views of space

image processing compression
Image Processing: Compression
  • Lossless – No loss in quality: gif, tiff
  • Lossy – Original image cannot be reconstructed: jpeg
  • New work on advancing lossy compression strategies with fewer visual artifacts: JPEG 2000 and wavelet transformations
image processing watermarking
Image Processing: Watermarking
  • Information hiding
    • Protecting copyright
image processing transformation
Image Processing: Transformation
  • Transforming image can make it easier to analyze

Wavelet transform of image

wavelet coefficients
Wavelet Coefficients

Horizontal LP,

Vertical LP

Horizontal LP,

Vertical HP

Horizontal HP,

Vertical LP

Horizontal HP, Vertical HP

5 3 linear phase wavelets
5/3 Linear Phase Wavelets

Linear phase 5/3: c[n] = {-1, 2,6,2,-1}, d[n]={1,-2,1} g[n] = {1, 2,-6,2, 1}, f[n]={1, 2,1}

computer vision 3d shape reconstruction
Computer Vision: 3D Shape Reconstruction
  • Use images to build 3D model of object or site

3D site model built from laser range scans collected by CMU autonomous helicopter

computer vision guiding motion
Computer Vision: Guiding Motion
  • Visually guided manipulation
    • Hand-eye coordination
  • Visually guided locomotion
    • robotic vehicles

CMU NavLab II

challenges in object recognition
Challenges in Object Recognition

245 267 234 142 22 28 38

121 156 187 98 73 32 12

123 21 21 38 209 237 121

99 87 59 197 216 244

object recognition research

LargeQuantityofData

Segmentationand HierarchicalAnalysis

Robust Algorithms

Lips

Face

Intra-class Object Variation

Large number of Object Classes

Hand Gesture

Text

License Plate

Clock

Vehicle

Building

Automated Learning

Advanced Image Enhancement

Low Image Quality

Object Recognition Research

Object Detection

Quality/Quantity Issues

Object Detection Issues

simpler problem classification
Simpler Problem: Classification
  • Fixed size input
  • Fixed object size, orientation, and alignment

“Object is present” (at fixed size and alignment)

Decision

“Object is NOT present”(at fixed size and alignment)

detection apply classifier exhaustively
Detection: Apply Classifier Exhaustively

Search in position

Search in scale

view based classifiers
View-based Classifiers

FaceClassifier #1

FaceClassifier #2

FaceClassifier #3

1 apply local operators
1) Apply Local Operators

f1(0, 0) = #5710

f1(0, 1) = #3214

fk(n, m) = #723

2 look up probabilities
2) Look Up Probabilities

P1( #5710, 0, 0 | obj) = 0.53

f1(0, 0) = #5710

P1( #5710, 0, 0 | non-obj) = 0.56

P1( #3214, 0, 1 | obj) = 0.57

f1(0, 1) = #3214

P1( #3214, 0, 1 | non-obj) = 0.48

fk(n, m) = #723

Pk( #723, n, m | obj) = 0.83

Pk( #723, n, m | non-obj) = 0.19

3 make decision
3) Make Decision

P1( #5710, 0, 0 | obj) = 0.53

P1( #5710, 0, 0 | non-obj) = 0.56

P1( #3214, 0, 1 | obj) = 0.57

0.53 * 0.57 * . . . * 0.83

> l

P1( #3214, 0, 1 | non-obj) = 0.48

0.56 * 0.48 * . . . * 0.19

Pk( #723, n, m | obj) = 0.83

Pk( #723, n, m | non-obj) = 0.19

probabilities estimated off line

H1(#567, 0, 0)

Hk(#350, 0, 0)

f1(0, 0) = #567

fk(n, m) = #350

H1(#567, 0, 0) = H1(567, 0, 0) + 1

Hk(#350, 0, 0) = Hk(#350, 0, 0) + 1

P1(#567, 0, 0) =

Pk(#350, 0, 0) =

SH1(#i, 0, 0)

SHk(#i, 0, 0)

Probabilities Estimated Off-Line
training classifiers
Training Classifiers
  • Cars: 300-500 images per viewpoint
  • Faces: 2,000 images per viewpoint
  • ~1,000 synthetic variations of each original image
    • background scenery, orientation, position, frequency
  • 2000 non-object images
    • Samples selected by bootstrapping
  • Minimization of classification error on training set
    • AdaBoost algorithm (Freund & Shapire ‘97, Shapire & Singer ‘99)
      • Iterative method
      • Determines weights for samples
web based demo of face detector
Web-based Demo of Face Detector

http://www.vasc.ri.cmu.edu/cgi-bin/demos/findface.cgi

applications of face detection
Applications of Face Detection
  • Automatic red-eye removal from photographs
  • Automatic color balancing in photo-finishing
  • Intelligent teleconferencing
  • Component in face identification system
difficulty increases with complexity of object
Difficulty Increases with Complexity of Object
  • 2D vs. 3D
  • Specific objects – e.g. my coffee mug
  • A category of objects – e.g. all coffee mugs
  • Amount of intra-category variation
    • Rigid or semi-rigid structure, e.g. face
    • Articulated objects, e.g. human body
    • Functionally defined objects, e.g. chairs
outline6
Outline
  • Defining Image Processing and Computer Vision
  • Emerging Technology
    • Digitization of documents
    • Digitization of images/photographs
    • Biometrics
    • Management of images on computers
    • Other: manufacturing, military, games, …
  • Research in Image Processing and Computer Vision
    • Automatically Finding Faces and Cars
    • Content-based Image Retrieval
spectrum of content based image retrieval
Spectrum of Content-Based Image Retrieval

Similar color distribution

Histogram matching

Similar texture pattern

Texture analysis

Image Segmentation,

Pattern recognition

Similar shape/pattern

Degree of difficulty

Life-time goal :-)

Similarrealcontent

status of image search
Status of Image Search
  • Typical Search Features
    • Color
    • Texture
    • Shape
    • Spatial attributes (local color regions, less common than global color, texture, shape metrics)
  • Commercial Activity
    • eVision (notes that “visual search engine market segment is projected to reach $1.4 billion by 2005 according to the McKenna Group” http://www.evisionglobal.com/about/index.html
    • Virage (www.virage.com)
    • IBM (QBIC part of database toolset)
reference a review of cbir
Reference: “A Review of CBIR”

Recommended reading:

A Review of Content-Based Image Retrieval Systems

Colin C. Venters and Dr. Matthew Cooper, University of Manchester

Available at http://www.jisc.ac.uk/jtap/htm/jtap-054.html

This review lists features from a number of image retrieval systems, along with heuristic evaluations on the interfaces for a subset of these systems.

search engines used by 2001 multimedia class
Search Engines Used by 2001 Multimedia Class
  • Search Engines used for 2001 multimedia retrieval homework (15 others answered a single query each):
search engines used in this 2002 class
Search Engines Used in This 2002 Class

Also answering 1 query each were: Excite+, Rexfeature, Webseek+, search.netscape.com+, animalplanet.com+, ask.com, naver.com+

for further reading on texture search
For Further Reading on Texture Search
  • Texture Search: “Texture features for browsing and retrieval of image data”, B.S. Manjunath and W.Y. Ma, IEEE Trans. on Pattern Analysis and Machine Intelligence18(8), Aug. 1996, pp. 837-842.
  • Texture search via http://www.engin.umd.umich.edu/ceep/tech_day/2000/reports/ECEreport2/ECEreport2.htm (texture features include coarseness, average gray scale value, and number of horizontal and vertical extrema of a specific image region)
  • For QBIC, texture search works on global coarseness, contrast and directionality features
for further exploration of image segmentation
For Further Exploration of Image Segmentation
  • BlobWorld work at UC Berkeley
  • Papers, description, sample system available at http://elib.cs.berkeley.edu/photos/blobworld/
further reading on wavelet compression and jpeg 2000
Further Reading on Wavelet Compression and JPEG 2000
  • http://www.gvsu.edu/math/wavelets/student_work/EF/how-works.html
  • http://www-ise.stanford.edu/class/psych221/00/shuoyen/
  • Henry Schneiderman Ph.D. Thesis “A Statistical Approach to 3D Object Detection Applied to Faces and Cars”, http://www.ri.cmu.edu/pub_files/pub2/schneiderman_henry_2000_2/schneiderman_henry_2000_2.pdf
  • http://www.jpeg.org/JPEG2000.html
summary image processing computer vision
Summary: Image Processing & Computer Vision
  • Not as mature as speech recognition
    • Technology not as reliable
    • Fewer companies, fewer products
  • Success on limited problems, e.g., documents
  • More applicable to fault tolerant problems
  • Technology will grow
    • Emergence of digital camera
    • Improved methods
decomposition in resolution frequency

coarse

intermediate

fine

intermediate

fine

Decomposition in Resolution/Frequency
wavelet decomposition
Wavelet Decomposition

Vertical subbands (LH)

wavelet decomposition1
Wavelet Decomposition

Horizontalsubbands (HL)