Chapter 7
Download
1 / 46

CHAPTER 7 - PowerPoint PPT Presentation


  • 413 Views
  • Uploaded on
  • Presentation posted in: Sports / Games

CHAPTER 7. Subjective Probability and Bayesian Inference. 7. 1. Subjective Probability. Personal evaluation of probability by individual decision maker Uncertainty exists for decision maker: probability is just a way of measuring it

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

CHAPTER 7

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


CHAPTER 7

Subjective Probability and Bayesian Inference


7. 1. Subjective Probability

  • Personal evaluation of probability by individual decision maker

  • Uncertainty exists for decision maker: probability is just a way of measuring it

  • In dealing with uncertainty, a coherent decision maker effectively uses subjective probability


7.2. Assessment of Subjective Probabilities

Simplest procedure:

  • Specify the set of all possible events,

  • ask the decision maker to directly estimate probability of each event

  • Not a good approach from psychological point of view

  • Not easy to conceptualize, especially for DM not familiar with probability


Standard Device

  • Physical instrument or conceptual model

  • Good tool for obtaining subjective probabilities

  • Example

    • A box containing 1000 balls

    • Balls numbered 1 to 1000

    • Balls have 2 colors: red, blue


Standard Device example

  • To estimate a students’ subjective probability of getting an “A” in SE 447,

  • we ask him to choose between 2 bets:

  • Bet X: If he gets an A, he win SR 100

    If he doesn’t get A, he wins nothing

  • Bet Y:If he picks a red ball, he win SR 100

    If he picks a blue ball, he wins nothing

  • We start with proportion of red balls P = 50%,

    then adjust successively until 2 bets are equal


Other standard devices

Pie diagram (spinner)

Circle divided into 2 sectors:

1 red

1 blue

  • Bet Y: If he spins to red section, he win SR 100

    If he spins to blue section, he wins nothing

  • Size of red section is adjusted until 2 bets are equal


Subjective Probability Bias

  • Standard device must be easy to perceive, to avoid introducing bias

    2 kinds of bias:

  • Task bias: resulting from assessment method (standard device)

  • Conceptual bias: resulting from mental procedures (heuristics) used by individuals to process information


Mental Heuristics Causing Bias

  • Representativeness

    • If x highly represents set A, high probability is given that X  A

    • Frequency (proportion) ignored

    • Sample size ignored

  • Availability

    • Limits of memory and imagination

  • Adjustment & anchoring

    • Starting from obvious reference point, then adjusting for new values.

    • Anchoring: adjustment is typically not enough

  • Overconfidence

    • Underestimating variance


Fractile Probability Assessment

  • Quartile Assessment:

  • Determine 3 values

  • x1, for which p(x > x1) = 0.5

  • x2, for which p(x < x2) = p(x2 < x< x1)

  • x3, for which p(x > x3) = p(x1 < x< x3)

  • x x2x1x3

  • F(x)0.250.50.75


Fractile Probability Assessment

  • Quartile Assessment: 4 intervals

  • Octile Assessment:8 intervals

  • Tertile Assessment:3 intervals.

    avoids anchoring

    at the median


Histogram Probability Assessment

  • Fix the points x1, x2, …, xm.

  • Ask the decision maker to assess probabilities

    p(x1 < x< x2)

    p(x2 < x< x3)

    x1x2x3…x

    Gives probability distribution

    (not cumulative p.d. as fractile method)


Assessment Methods & Bias

  • No evidence to favor either fractile or histogram methods

  • One factor that reinforces anchoring bias is self-consistency

  • Bias can be reduced by “pre-assessment conditioning”: training, for/against arguments

  • The act of probability assessment causes a re-evaluation of uncertainty


7.3. Impact of New Information(Bayes’ Theorem)

  • After developing subjective probability distribution

  • Assume new information becomes available

    Example: new data is collected

  • According to coherence principle, DM must take new information in consideration, thus

    Subjective probability must be revised

  • How? using Bayes’ theorem


Bayes’ Theorem Example

  • Suppose your subjective probability distribution for weather tomorrow is:

    • chances of being sunnyP(S) = 0.6

    • chances of being not sunnyP(N) = 0.4

  • If the TV weather forecast predicted a cloudy day tomorrow. How should you change P(S)?

  • Assume we are dealing with mutually exclusive and collectively exhaustive events such as sunny or not sunny.


Impact of Information

  • We assume the weather forecaster predicts either

    • cloudy dayC, or

    • bright dayB.

  • To change P(S), we use the joint probability = conditional probability * marginal probability

    • P(C,S) = P(C|S)P(S)P(B,S) = P(B|S)P(S)

    • P(C,N) = P(C|N)P(N) P(B,N)= P(B|N)P(N)


Impact of Information

  • To obtain j.p.m.f we need the conditional probabilities P(C|S) and P(C|N).

  • These can be obtained from historical data. How?

  • In past 100 sunny days, cloudy forecast in 20 days

    P(C|S) = 0.2

    P(B|S) = 0.8

  • In past 100 cloudy days, cloudy forecast in 90 days

    P(C|N) = 0.9

    P(B|N) = 0.1


Joint probabilityCalculations

  • Joint probability P(A,B) =

    conditional probability (likelihood) P(A|B) * marginal probability P(B)

  • Cloudy forecast

    • P(C,S) = P(C|S)P(S) = 0.2(0.6)= 0.12

    • P(C,N) = P(C|N)P(N) = 0.9(0.4)= 0.36

  • Sunny forecast

    • P(B,S) = P(B|S)P(S) = 0.8(0.6)= 0.48

    • P(B,N) = P(B|N)P(N) = 0.1(0.4)= 0.04


Bayes’ Theorem

SN

CP(C,S) P(C,N)P(C)

BP(B,S)P(B,N) P(B)

P(S)P(N)

  • P(S|C) = P(C,S) / P(C)

    = P(C|S)P(S ) / P(C)

  • P(S|C) P(C|S)P(S )


Joint probabilityTable

SN

C0.120.360.48

B0.480.040.52

0.60.4

  • P(S|C) = 0.12/0.48 = 0.25 posterior (conditional)

    probability

  • Compare to P(S) = 0.6 prior (marginal) prob.

  • P(S) decreased because of C forecast


Prior and Posterior Probabilities

  • Prior means before.

    Prior probability is the probability P(S) before the information was heard.

  • Posterior means after.

    It is probability obtained after incorporating the new forecast information. It is P(S|C).

    It is obtained using Bayes’ theorem.


Example with 3 states

3 demand possibilities for new product

  • HighP(H)= 0.6

  • MediumP(M)= 0.1

  • LowP(L)= 0.3

    Market research gives Average result:

  • 30% of timeif true demand is High

  • 50% of timeif true demand is Medium

  • 90% of timeif true demand is Low


Example : Probability Table for Average result

StatePriorLikelihoodJointPosterior

SP(S)P(A|S)P(S,A)P(S|A)

H0.60.30.180.36

M0.10.50.050.10

L0.30.90.270.54

1.00.501.00

  • If market research gives Average result:

    P(H),P(L),P(M)


Ex: Sequential Bayesian Analysis

  • An oil company has 3 drilling sites: X, Y, Z

    3 possible reserve states:

  • No reservesP(N) = 0.5

  • Small reservesP(S) = 0.3

  • Large reservesP(L) = 0.2

    3 possible drilling outcomes:

  • DryD

  • WetW

  • GushingG


Ex: Sequential Bayesian Analysis

If reserves are:

  • None (N) all wells will be dry (D)

  • Large (L) all wells will be Gushing (G)

  • Small (S) some Dry (D) and some wet (W) wells P(1D/1) = 0.8

    P(2D/2) = 0.2

    P(3D/3) = 0

  • All sites are equally favorable

  • Assume order of drilling is: XYZ

  • Notation: DX = probability of Dry well at site X


Ex: Probability Table for Site X

StatePriorConditionalJoint

KP(K)P(D|K)P(W|K)P(G|K)DXWXGX

N0.51000.500

S0.30.80.200.240.060

L0.2001000.2

1.00.740.060.2

  • Stop exploratory drilling at X if you get:

    W: Reserves are S, or G: Reserves are L

  • If you get D, drill at Y..

    (Res. N: P = 0.5/0.74 = 0.68, S: P = 0.24/0.74 = 0.32)


Ex: Probability Table for Site Y

StatePriorConditionalJoint

KP(K)P(D|K)P(W|K)DYWY

N0.68100.680

S0.320.50.50.160.16= 0.4/0.8

1.00.840.16

  • Stop exploratory drilling at Y if you get W:

    Reserves are S

  • If you get D, drill at Z

    (Res. N: P = 0.68/0.84 = 0.81, S: P = 0.16/0.84 = 0.19)


Ex: Probability Table for Site Z

StatePriorConditionalJoint

KP(K)P(D|K)P(W|K)DZWZ

N0.81100.810

S0.19010 0.19

1.00.810.19

  • If you get W:

    Reserves are S

  • If you get D:

    Reserves are N


Ex: Change in P(N)

Prior Probability

  • Before drillingP(N) = 0.5

    Posterior Probabilities

  • After 1 DryP(N|Dx) = 0.68

  • After 2 DryP(N|Dx, DY) = 0.81

  • After 3 DryP(N|Dx, DY, DZ) = 1


7.4. Conditional Independence

Two events, A and B, are independent iff:

  • P(A, B) = P(A)*P(B)

    Implying

  • P(A|B) = P(A)

  • P(B|A) = P(B)

  • Posterior probability is the same the prior

  • New information about 1 event does not affect the probability of the other


Conditional Independence

Two events, A and B, are conditionally independent iff:

  • P(A, B)  P(A)*P(B)

    But their conditional probabilities on a 3rd event, C, are independent

  • P(A, B|C) = P(A|C)*P(B|C)

  • Useful property in Bayesian analysis


Ex: Horse Race Probability

Horse named WR will race at 3:00 p.m.

Probability of WR winning P(WR) depends on track condition

  • Firm (F)P(F) = 0.3 P(WR|F) = 0.9

  • Soft (S)P(S) = 0.7 P(WR|S) = 0.2


Ex: Horse Race Probability

  • Given the results of 2 previous races

  • At 1:30, horse named MW won the race

    P(MW|F) = 0.8P(MW|S) = 0.4

  • At 2:00, horse named AJ won the race

    P(AJ|F) = 0.9 P(AJ|S) = 0.5

  • What the new WR win probability P(WR|MW, AJ)?


Ex: Horse Race Probability

  • P(WR) wins given MW and AJ have won must sum conditional probabilities of both possible track conditions: F or S

  • P(WR|MW, AJ) =

    P(WR|F) * P(F|MW, AJ) + P(WR|S) * P(S|MW, AJ)


Ex: Horse Race Probability

  • The 2 events MW and AJ are conditionally independent with respect to a 3rd event: track condition F or S

  • Recall:P(A|B) = P(B|A)P(A)/P(B)

    P(A|B)  P(B|A)P(A)

  • P(F|MW, AJ)P(MW, AJ|F) P(F)

    P(MW|F) P(AJ|F) P(F)

  • P(S|MW, AJ)P(MW, AJ|S) P(S)

    P(MW|S) P(AJ|S) P(S)


Ex: Horse Race Probability

  • P(F|MW, AJ)P(MW|F) P(AJ|F) P(F)

    0.8 * 0.9 * 0.3 = 0.216

  • P(S|MW, AJ)P(MW|S) P(AJ|S) P(S)

    0.4 * 0.5 * 0.7 = 0.14

    Normalizing

    P(F|MW, AJ) = 0.216/(0.216 + 0.14) = 0.61

    P(F|MW, AJ) = 0. 14/(0.216 + 0.14) = 0.39


Ex: Horse Race Probability

  • Substituting into

  • P(WR|MW, AJ) =

    P(WR|F) * P(F|MW, AJ) + P(WR|S) * P(S|MW, AJ)

    = 0.9(0.61) + 0.2(0.39)

    = 0.63


7.5. Bayesian Updating with Functional Likelihoods

  • Posterior probability

    P(A|B)= P(A,B )/P(B)

    =P(B|A)P(A)/P(B)

  • Conditional probability (likelihood) P(B|A) can be described by particular probability distribution:

    • (1) Binomial

    • (2) Normal


Binomial likelihood distribution

  • Dichotomous (2-value) data: defective, not

  • Sequence of dichotomous outcomes: series of quality tests

  • In each test, constant probability (p) of one of the 2 outcomes: defective

  • Outcome of each test is independent of others

  • Total number (r) of one kind of outcomes (defective) out of (n) tests is

  • f(r|n, p) =Given in tables


Binomial likelihood example

Demand for new product can be:

  • High (H)P(H) = 0.2

  • Medium (M)P(M) = 0.3

  • Low (L)P(L) = 0.5

    For each case, the probability (p) that an individual customer buys the product is

  • H:p = 0.25

  • M:p = 0.1

  • L:p = 0.05

    In random sample of 5 customers, 1 will buy


Binomial likelihood example

For (n = 5, r = 1), likelihoods are obtained from table, or calculated by:

  • H: [5!/(4!*1)]0.25(0.75)4 = 0.3955

  • M: [5!/(4!*1)]0.1(0.9)4 = 0.3281

  • L: [5!/(4!*1)]0.05(0.95)4 = 0.2036

    The joint probability table can now be constructed


Binomial Likelihood Example

StatePriorLikelihoodJointPosterior

pP(p)P(1|5,p)P(p,1/5)P(p|1/5)

H: 0.250.20.39550.07910.2832

M: 0.10.30.32810.09840.3523

L: 0.050.50.20360.10180.3645

1.00.27931.00

  • If 1 in a sample of 5 customers buys:

    P(H),P(M),P(L)


Normal likelihood distribution

  • Most common, symmetric,

  • Continuous data, can approximate discrete

  • f(y|, ) =

  • Given in tables. Formula usually not used.

  • Two parameters: mean () and standard deviation ().


Normal likelihood Updating

  • Mean () and standard deviation () Can be updated individually (assuming one is known) or together

  • P(|y, )  f(y|, ) P(|)

  • P(|y, )  f(y|, ) P(|)

  • P(,  |y)  f(y|, ) P(, )


Normal likelihood example

Updating Mean ()

Average weight setting has 2 possibilities:

  • High (H)P(H) = 0.5  = 8.2, = 0.1

  • Low (L)P(L) = 0.5  = 7.9, = 0.1

    A sample of 1 bottle has weight = 8.0 oz.

    What is the posterior probability of H and L?


Normal likelihood example

  • Likelihood values are obtained from table, or calculated by

  • If  = 8.2

    Z = (8.0 – 8.2)/0.1 = – 2

    f(8|, ) = 0.054

  • If  = 7.9

    Z = (8.0 – 7.9)/0.1 = 1

    f(8|, ) = 0.242


Normal Likelihood Example

StatePriorLikelihoodJointPosterior

 P()P(8|, )P(8,,)P( |8)

H: 8.20.50.0540.0270.18

L: 7.90.50.2420.1210.82

1.00.1481.00

  • After a sample of 1 bottle with weight = 8:

    P(H),P(L)


ad
  • Login