non parametric learning
Download
Skip this Video
Download Presentation
Non-Parametric Learning

Loading in 2 Seconds...

play fullscreen
1 / 17

Non-Parametric Learning - PowerPoint PPT Presentation


  • 102 Views
  • Uploaded on

Non-Parametric Learning. Prof. A.L. Yuille Stat 231. Fall 2004. Chp 4.1 – 4.3. Parametric versus Non-Parametric. Previous lectures on MLE learning assumed a functional form for the probability distribution. We now consider an alternative non-parametric method based on window function.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Non-Parametric Learning' - reese-roberson


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
non parametric learning

Non-Parametric Learning

Prof. A.L. Yuille

Stat 231. Fall 2004.

Chp 4.1 – 4.3.

parametric versus non parametric
Parametric versus Non-Parametric
  • Previous lectures on MLE learning assumed a functional form for the probability distribution.
  • We now consider an alternative non-parametric method based on window function.
non parametric
Non-Parametric
  • It is hard to develop probability models for some data.
  • Example: estimate the distribution of annual rainfall in the U.S.A. Want to model p(x,y) – probability that a raindrop hits a position (x,y).
  • Problems: (i) multi-modal density is difficult for parametric models, (ii) difficult/impossible to collect enough data at each point (x,y).
intuition
Intuition
  • Assume that the probability density is locally smooth.
  • Goal: estimate the class density model p(x) from data
  • Method 1: Windows based on points x in space.
windows
Windows
  • For each point x, form a window centred at x with volume Count the number of samples that fall in the window.
  • Probability density is estimated as:
non parametric1
Non-Parametric
  • Goal: to design a sequence of windows

so that at each point x

  • (f(x) is the true density).
  • Conditions for window design:
  • increasing spatial resolution.

(ii) many samples at each point

(iii)

two design methods
Two Design Methods
  • Parzen Window: Fix window size:
  • K-NN: Fix no. samples in window:
parzen window
Parzen Window
  • Parzen window uses a window function
  • Example:
  • (i) Unit hypercube:

and 0 otherwise.

  • (ii) Gaussian in d-dimensions.
parzen windows
Parzen Windows
  • No. of samples in the hypercube is
  • Volume
  • The estimate of the distribution is:
  • More generally, the window interpolates the data.
parzen window example
Parzen Window Example
  • Estimate a density with five modes using Gaussian windows at scales h=1,0.5, 0.2.
convergence proof
Convergence Proof.
  • We will show that the Parzen window estimator converges to the true density at each point x with increasing number of samples.
proof strategy
Proof Strategy.
  • Parzen distribution

is a random variable which depends on the samples used to estimate it.

  • We have to take the expectation of the distribution with respect to the samples.
  • We show that the expected value of the Parzen distribution will be the true distribution. And the expected variance of the Parzen distribution will tend to 0 as no. samples gets large.
example of parzen window
Example of Parzen Window
  • Underlying density is Gaussian. Window volume decreases as
example of parzen window1
Example of Parzen Window
  • Underlying Density is bi-modal.
parzen window and interpolation
Parzen Window and Interpolation.
  • In practice, we do not have an infinite number of samples.
  • The choice of window shape is important. This effectively interpolates the data.
  • If the window shape fits the local structure of the density, then Parzen windows are effective.
ad