Statistics of Natural Image Categories
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

Jonathan Huang ([email protected]) 1/30/2006 PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on
  • Presentation posted in: General

Statistics of Natural Image Categories Antonio Torralba and Aude Oliva. Network: Computation in Neural Systems , 14(2003) 391-412. Jonathan Huang ([email protected]) 1/30/2006. Spatial Image Signatures.

Download Presentation

Jonathan Huang ([email protected]) 1/30/2006

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Jonathan huang jch1 cs cmu 1 30 2006

Statistics of Natural Image CategoriesAntonio Torralba and Aude Oliva. Network: Computation in Neural Systems, 14(2003) 391-412

Jonathan Huang ([email protected])

1/30/2006


Spatial image signatures

Spatial Image Signatures

Averaged pictures of categories of objects, scenes and objects in scenes, computed with 100 exemplars or more per category. Exemplars were chosen to have the same basic level and viewpoint in regard to an observer. The group objects in scenes (third row) represent examples of the averaged peripheral information around an object centered in the image.


Caltech 101

Caltech 101

  • Object Categories:

    • Airplanes

    • Brain

    • Brontosaurus

    • Chandelier

    • Garfield

    • Kangaroo

    • Octopus

    • Trilobyte

    • Etc…

By Antonio Torralba


100 special moments and conan

100 Special Moments (and Conan)

Conan O’Brien

Little Leaguer

Newlyweds

By Jason Salavon


Power spectra of natural images

Power Spectra of Natural Images

  • Fourier Transform:

  • Magnitude Spectrum:

  • Spectral Signature (for a set of images S):


Jonathan huang jch1 cs cmu 1 30 2006

“Demo”

  • Computing the Spectrum (Matlab):

    • Ifft = abs(fftshift(fft2(I,w,h)));

  • Visualization:

    • imshow(log(Ifft)/max(max(log(Ifft))));

    • colormap(cool);


Fft beach

FFT(Beach)


Fft pittsburgh

FFT(Pittsburgh)


1 f spectra

1/f Spectra

  • Natural Image Spectra follow a power law!

  • As() is called the Amplitude Scaling Factor

  • 2-() is the Frequency Exponent.  clusters around 0 for natural images.

  • Any guesses on why this law holds?


Main idea of torralba oliva papers

Main Idea of Torralba/Oliva Papers

  • “Statistics of Natural Images vary as a function of the interaction between the observer and the world”!


Some examples

Some examples


Spectral signatures

Spectral Signatures

  • Why are Fields, Beaches and Coasts less isotropic than other natural environments?


Scene scale

Scene Scale

  • “The point of view that any given observer adopts on a specific scene is constrained by the volume of the scene.”

  • How does the amount of clutter vary against scene scale in man-made environments? In natural environments?


Pca on natural images

PCA on Natural Images

Top Row: PCA on images

Bottom Row: PCA on power spectra


Openness naturalness

Openness/Naturalness

Projection of images onto the second and third principle components. SPC2 corresponds to “Openness” and SPC3 corresponds to “Naturalness”


Spatially localized statistics

Spatially Localized Statistics

  • Windowed FFT

  • Image statistics become non-stationary as scene scale increases.

Top Row: Man-made environments

Bottom Row: Natural environments


What do images statistics say about depth

What do Images Statistics say about Depth?

V: Vertical

H: Horizontal

O: Oblique


Comparing localized spectral signatures and depth

Comparing Localized Spectral Signatures and Depth

  • With increasing depth comes:

    • An increase in global roughness for man-made structures

    • A decrease in global roughness for natural structures

    • Nonuniformity in spatially localized spectral signatures


Examples man made

Examples (man-made)


Examples natural

Examples (Natural)


An algorithm

An Algorithm

  • First use PCA to reduce dimension

  • Goal: For a vector of features v, estimate

    • Since the man-made and natural cases should be treated separately, we model f(v|art) and f(v|nat) as Gaussians.

    • The joint distribution is modeled as a weighted sum of Gaussians:

      where


Model estimation

Model Estimation

  • Expectation-Maximization

    • E-step:

      • Compute posterior probabilities of the clusters given observed data.

    • M-step:

      • Update cluster parameters, weighting training data by the posterior probabilities from the E-step.

Read Torralba and Oliva: Depth Estimation, IEEE PAMI 2002 for the update equations. They do not all fit on one slide...


Some results

Some Results


F d category

f(D|category)

Distribution of Scene Categories as a function of mean depth.


Application scale selection

Application: Scale Selection


Context in images

Context in Images

  • Question: How can these small people possibly affect the image statistics in any significant way??


Object detection

Object Detection


Thank you

Thank You

  • References

    • Torralba and Oliva, Statistics of Natural Image Categories. Network: Computation in Neural Systems 14 (2003) 391-412.

    • Torralba and Oliva, Depth Estimation from Image Structure. IEEE PAMI Vol 14, No. 9 (2002).

    • Oliva and Torralba, Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. IJCV 42(3), 145-175 (2001).

    • Srivastava, Lee, Simoncelli, Zhu, On Advances in Statistical Modeling of Natural Images. JMIV 18:17-33 (2003)

    • Mumford, Pattern Theory: the Mathematics of Perception. ICM 2002. Vol III. 1-3


  • Login