Basic probability
Download
1 / 49

Basic Probability - PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on

Basic Probability. Introduction. Probability is the study of randomness and uncertainty. In the early days, probability was associated with games of chance (gambling). Simple Games Involving Probability.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Basic Probability' - orli-sheppard


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Basic probability
Basic Probability

Pattern Classification, Chapter 1


Introduction
Introduction

  • Probability is the study of randomness and uncertainty.

  • In the early days, probability was associated with games of chance (gambling).

Pattern Classification, Chapter 1


Simple games involving probability
Simple Games Involving Probability

Game: A fair die is rolled. If the result is 2, 3, or 4, you win $1; if it is 5, you win $2; but if it is 1 or 6, you lose $3.

Should you play this game?

Pattern Classification, Chapter 1


Random experiment
Random Experiment

  • a random experiment is a process whose outcome is uncertain.

  • Examples:

  • Tossing a coin once or several times

  • Picking a card or cards from a deck

  • Measuring temperature of patients

  • ...

Pattern Classification, Chapter 1


Events sample spaces
Events & Sample Spaces

Sample Space

The sample space is the set of all possible outcomes.

Event

An event is any collection

of one or more simple events

Simple Events

The individual outcomes are called simple events.

Pattern Classification, Chapter 1


Example
Example

Experiment: Toss a coin 3 times.

  • Sample space 

  •  = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}.

  • Examples of events include

    • A = {HHH, HHT,HTH, THH}

      = {at least two heads}

    • B = {HTT, THT,TTH}

      = {exactly two tails.}

Pattern Classification, Chapter 1


Basic concepts from set theory
Basic Concepts (from Set Theory)

  • The union of two events A and B, AB, is the event consisting of all outcomes that are either in Aor in Bor in both events.

  • The complement of an event A, Ac, is the set of all outcomes in that are not in A.

  • The intersection of two events A and B, AB, is the event consisting of all outcomes that are in both events.

  • When two events A and B have no outcomes in common, they are said to be mutually exclusive, or disjoint, events.

Pattern Classification, Chapter 1


Example1
Example

Experiment: toss a coin 10 times and the number of heads is observed.

  • Let A = { 0, 2, 4, 6, 8, 10}.

  • B = { 1, 3, 5, 7, 9}, C = {0, 1, 2, 3, 4, 5}.

  • AB= {0, 1, …, 10} = .

  • AB contains no outcomes. So A and B are mutually exclusive.

  • Cc = {6, 7, 8, 9, 10}, A  C = {0, 2, 4}.

Pattern Classification, Chapter 1


Rules
Rules

  • Commutative Laws:

    • AB = B A, AB = B A

  • Associative Laws:

    • (AB) C = A (B C )

    • (AB)  C = A (B  C) .

  • Distributive Laws:

    • (AB)C = (A  C)  (B  C)

    • (AB)C = (A  C) (B  C)

  • DeMorgan’s Laws:

Pattern Classification, Chapter 1


Venn diagram
Venn Diagram

A

B

A∩B

Pattern Classification, Chapter 1


Probability
Probability

  • A Probability is a number assigned to each subset (events) of a sample space .

  • Probability distributions satisfy the followingrules:

Pattern Classification, Chapter 1


Axioms of probability
Axioms of Probability

  • For any event A, 0  P(A)  1.

  • P() =1.

  • If A1, A2, … An is a partition of A, then

  • P(A) = P(A1)+ P(A2)+...+ P(An)

  • (A1, A2, … An is called a partition of A if A1A2 …An = A and A1, A2, … An are mutually exclusive.)

Pattern Classification, Chapter 1


Properties of probability
Properties of Probability

  • For any event A, P(Ac) = 1 - P(A).

  • If AB, then P(A)  P(B).

  • For any two events A and B,

  • P(AB) = P(A) + P(B) - P(AB).

  • For three events, A, B, and C,

  • P(ABC) = P(A) + P(B) + P(C) -

  • P(AB) - P(AC) - P(BC) + P(AB C).

Pattern Classification, Chapter 1


Example2
Example

  • In a certain population, 10% of the people are rich, 5% are famous, and 3% are both rich and famous. A person is randomly selected from this population. What is the chance that the person is

    • not rich?

    • rich but not famous?

    • either rich or famous?

Pattern Classification, Chapter 1


Intuitive development agrees with axioms
Intuitive Development (agrees with axioms)

  • Intuitively, the probability of an event a could be defined as:

Where N(a) is the number that event a happens in n trials

Pattern Classification, Chapter 1



More formal
More Formal:

  • W is the Sample Space:

    • Contains all possible outcomes of an experiment

  • w inW is a single outcome

  • A inW is a set of outcomes of interest

Pattern Classification, Chapter 1


Independence
Independence

  • The probability of independent events A, B and C is given by:

    P(A,B,C) = P(A)P(B)P(C)

A and B are independent, if knowing that A has happened

does not say anything about B happening

Pattern Classification, Chapter 1


Bayes theorem
Bayes Theorem

  • Provides a way to convert a-priori probabilities to a-posteriori probabilities:

Pattern Classification, Chapter 1


Conditional probability
Conditional Probability

  • One of the most useful concepts!

W

A

B

Pattern Classification, Chapter 1


Bayes theorem1
Bayes Theorem

  • Provides a way to convert a-priori probabilities to a-posteriori probabilities:

Pattern Classification, Chapter 1


Using partitions
Using Partitions:

  • If events Ai are mutually exclusive and partition W

Pattern Classification, Chapter 1


Random variables
Random Variables

  • A (scalar) random variable X is a function that maps the outcome of a random event into real scalar values

W

X(w)

w

Pattern Classification, Chapter 1


Random variables distributions
Random Variables Distributions

  • Cumulative Probability Distribution (CDF):

  • Probability Density Function (PDF):

Pattern Classification, Chapter 1


Random distributions
Random Distributions:

  • From the two previous equations:

Pattern Classification, Chapter 1


Uniform distribution
Uniform Distribution

  • A R.V. X that is uniformly distributed between x1 and x2 has density function:

X1

X2

Pattern Classification, Chapter 1


Gaussian normal distribution
Gaussian (Normal) Distribution

  • A R.V. X that is normally distributed has density function:

m

Pattern Classification, Chapter 1


Statistical characterizations
Statistical Characterizations

  • Expectation (Mean Value, First Moment):

  • Second Moment:

Pattern Classification, Chapter 1


Statistical characterizations1
Statistical Characterizations

  • Variance of X:

  • Standard Deviation of X:

Pattern Classification, Chapter 1


Mean estimation from samples
Mean Estimation from Samples

  • Given a set of N samples from a distribution, we can estimate the mean of the distribution by:

Pattern Classification, Chapter 1


Variance estimation from samples
Variance Estimation from Samples

  • Given a set of N samples from a distribution, we can estimate the variance of the distribution by:

Pattern Classification, Chapter 1



Chapter 1 introduction to pattern recognition sections 1 1 1 6

Chapter 1: Introduction to Pattern Recognition (Sections 1.1-1.6)

Machine Perception

An Example

Pattern Recognition Systems

The Design Cycle

Learning and Adaptation

Conclusion


Machine perception
Machine Perception 1.1-1.6)

  • Build a machine that can recognize patterns:

    • Speech recognition

    • Fingerprint identification

    • OCR (Optical Character Recognition)

    • DNA sequence identification

Pattern Classification, Chapter 1


An example
An Example 1.1-1.6)

  • “Sorting incoming Fish on a conveyor according to species using optical sensing”

    Sea bass

    Species

    Salmon

Pattern Classification, Chapter 1


  • Problem Analysis 1.1-1.6)

    • Set up a camera and take some sample images to extract features

      • Length

      • Lightness

      • Width

      • Number and shape of fins

      • Position of the mouth, etc…

      • This is the set of all suggested features to explore for use in our classifier!

Pattern Classification, Chapter 1


  • Preprocessing 1.1-1.6)

    • Use a segmentation operation to isolate fishes from one another and from the background

  • Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features

  • The features are passed to a classifier

Pattern Classification, Chapter 1



  • Classification 1.1-1.6)

    • Select the length of the fish as a possible feature for discrimination

Pattern Classification, Chapter 1



The 1.1-1.6)length is a poor feature alone!

Select the lightness as a possible feature.

Pattern Classification, Chapter 1



Pattern Classification, Chapter 1


Lightness

Width

Pattern Classification, Chapter 1



Pattern Classification, Chapter 1


Pattern Classification, Chapter 1 ones we already have. A precaution should be taken not to reduce the performance by adding “noisy features”


Pattern Classification, Chapter 1


Pattern Classification, Chapter 1 aim of designing a classifier is to correctly classify novel input


ad