1 observations and random experiments
Download
1 / 84

1. Observations and random experiments - PowerPoint PPT Presentation


  • 141 Views
  • Updated On :

1. Observations and random experiments. Observations are viewed as outcomes of a random experiment. Observations. Observation  random experiment (controlled) Outcome cannot be predicted with certainty Range of possible outcomes known

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '1. Observations and random experiments' - van


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
1 observations and random experiments l.jpg
1. Observations and random experiments

Observations are viewed as outcomes of a random experiment.


Observations l.jpg
Observations

  • Observation  random experiment (controlled)

  • Outcome cannot be predicted with certainty

  • Range of possible outcomes known

  • With each outcome of an observation may be associated a unique numeric value

  • The outcome variable, X, is a random variable because, until the experiment is performed, it is uncertain what value X will take.

  • To quantify this uncertainty, probabilities are associated with values (x) of the R.V. X (and outcomes of experiment)


Slide3 l.jpg

2.1 Continuousrandom variables


Continuous random variables l.jpg
Continuousrandom variables

  • Normal r.v.  probit model

  • Logistic r.v.  logit model

  • Uniform r.v.  waiting time to event

  • Exponential r.v.  waiting time to event

  • Gompertz r.v.


Gaussian probability model l.jpg
Gaussian probability model

Time at event follows a normal distribution with mean  and variance 2 (random variable is normally distributed)


Slide6 l.jpg

Normal distribution: density

With  the mean and 2 the variance

Linear predictor:


Link function l.jpg
Link function

The link function relates the linear predictor  to the expected value  of the datum y

(McCullagh and Nelder, 1989, p. 31)

 = 


Slide8 l.jpg

Standard normal density

With  = 0 the mean and 2 = 1 the variance

The probit model relies on a standard normal distribution (cumulative): it is the INVERSE of the standard normal


Slide9 l.jpg

Cumulative normal distribution

Approximation by Page (1977)

where

Page, E. (1977) Approximations to the cumulative normal function and its inverse for use on a pocket calculator. Applied Statistics, 26:75-76

Azzalini, 1996, p. 269


Slide10 l.jpg

Excel: NORMDIST

Returns the normal cumulative distribution for the specified mean and standard deviation.

Syntax: NORMDIST(x,mean,standard_dev,cumulative)

X is the value for which you want the distribution.

Mean is the arithmetic mean of the distribution.

Standard_dev is the standard deviation of the distribution.

Cumulative is a logical value that determines the form of the function.

If cumulative is TRUE, NORMDIST returns the cumulative distribution function;

if FALSE, it returns the probability mass function.

Example: NORMDIST(42,40,1.5,TRUE) equals 0.90879

NORMDIST(42,40,1.5,FALSE) equals 0.10934


Slide11 l.jpg

SPSS

RV.NORMAL

COMPUTE variable = RV.NORMAL (mean, standard deviation)

COMPUTE test = RV.NORMAL(24,2) .

CDF.NORMAL

Returns the cumulative probability that the a value of a normal distribution

with given mean and standard deviation, will be less than a given quantity Q.

COMPUTE variable = CDF.NORMAL(Q,mean,standard deviation)

COMPUTE test2 = CDF.NORMAL(24,24,2) .

Test2 = 0.50


Inverse of standard normal cumulative distribution l.jpg
Inverse of standard normal cumulative distribution

The probit is the value zp from the normal distribution for which

the cumulative distribution is equal to a given probability p.


Excel normsinv inverse of standard normal cumulative distribution l.jpg
Excel: NORMSINV Inverse of standard normal cumulative distribution

NORMSINV: Probability is a probability corresponding to the normal distribution.

NORMSINV uses an iterative technique for calculating the function.

Given a probability value, NORMSINV iterates until the result is accurate

to within ± 3x10^-7.

If NORMSINV does not converge after 100 iterations,

the function returns the #N/A error value.

Example: NORMSINV(0.908789) equals 1.3333

E.g. (z) = 0.025 for z = -1.96 Probit(0.025) = -1.96

(z) = 0.975 for z = 1.96 Probit(0.975) = 1.96


Slide14 l.jpg

SPSS: IDF.NORMAL

Returns the value from the normal distribution for which the

cumulative distribution is the given probability P.

COMPUTE variable = IDF.NORMAL(P,mean,stddev)

COMPUTE test3 = IDF.NORMAL(0.025,24,2) .

Test3 = 20.08

IDF.NORMAL (0.5,24,2) = 24


Example 1 age at migration l.jpg
Example 1. Age at migration

  • A sample of 20 males and 20 females

  • Sample generated on computer: random number generator


Slide16 l.jpg

Example 1

Random sample of 20 males and 20 females: Age at migration

E:\f\life\rnumber\normal\mig\2.xls



Slide18 l.jpg

Example 1

SPSS linear regression: y = a + b x

(y = age, x = sex)

1 = 24.3 for males

2 = 24.3 - 3.1 = 21.2 for females

Cte: Lower bound: 24.3 - 1.96 * 0.535 = 23.2

Upper bound: 24.3 + 1.96 * 0.535 = 25.4

: Lower bound: -3.1 - 1.96 * 0.757 = -4.6

Upper bound: -3.1 + 1.96 * 0.756 = -1.6


Random number generation age at migration 200 respondents l.jpg
Random number generationAge at migration200 respondents

  • Normal random number in SPSS

    • COMPUTE variable = RND(RV.NORMAL(24,2)) .

  • Logistic random number in SPSS

    • COMPUTE variable = RND(RV.LOGISTIC(24,2)) .

  • Create frequency table in SPSS


  • Random number generation spss age at migration 200 and 2000 respondents l.jpg
    Random number generation (SPSS)Age at migration200 and 2000 respondents

    COMPUTE NORMAL1 = RND(RV.NORMAL(24,2)) .

    VARIABLE LABELS normal1 "NORMAL N(24,4)".

    VARIABLE WIDTH normal1 (6) .

    COMPUTE LOGIST = RND(RV.LOGISTIC(24,2)) .

    VARIABLE LABLES logist "LOGISTIC L(24,1)".

    VARIABLE WIDTH logist(6).

    COMPUTE ONE = 1 .

    /* Table of Frequencies.

    TABLES

    /FTOTAL $t 'Total' /* INCLUDE TOTAL

    /FORMAT BLANK MISSING('.') /TABLES

    (LABELS) + $t BY

    one > ( normal1 + logist )

    /STATISTICS COUNT ((F5.0) 'Count' ) .


    Slide21 l.jpg

    Age at migration

    200 respondents

    N(mean, variance) = N(24,4)

    L(mean, scale parameter) = L(24,1)


    Slide22 l.jpg

    Age at migration

    2000 respondents

    N(mean, variance) = N(24,4)

    L(mean, scale parameter) = L(24,1)

    Theoretical logistic: lambda = 1/1.81




    Slide25 l.jpg

    Example 2

    SPSS


    Slide26 l.jpg

    Example 3

    Heaping!


    Slide27 l.jpg

    2. The logistic modelDuration = logistic r.v.Time at event = logistic r.v.


    Slide28 l.jpg

    Standard logistic distribution

    Probability of being in category 1 instead of categ. 0:

    Cumulative distribution:

    Probability density function:

    With  (logit) the linear predictor

    ‘Standard’ logistic distribution with mean 0 and variance 2 = 2/3  3.29 hence  = 1.81

    The logit model relies on a standard logistic distribution (variance  1 !)


    Slide29 l.jpg

    ‘Standardised’ logistic distribution

    Cumulative distribution:

    Probability density function:

    =/3  1.8138 = 1.8138

    Standardized logistic with mean 0 and variance 1


    Slide30 l.jpg

    = 1.81


    Slide32 l.jpg

    Link function

    The link function relates the linear predictor  to the expected value p () of the datum y

    (McCullagh and Nelder, 1989, p. 31)

    Logit:  = logit(p) = ln [p/(1-p)]


    Slide33 l.jpg

    Link functions

    Translate probability scale (0,1) into real number scale (-,+ )

    Logit

    E.g. logit(0.25) = -1.0986

    logit(0.1235) = -1.96

    logit(0.8765) = 1.96

    Probit

    E.g. (z) = 0.025 for z = -1.96 Probit(0.025) = -1.96



    Slide35 l.jpg
    Demography:Uniform and exponential distributions of events [in an(age) interval]Probability densityIntensity


    Slide36 l.jpg

    3. The uniform distributionThe linear modelDuration = uniform r.v.Time at event = uniform r.v.

    Density


    Uniform distribution l.jpg
    Uniform distribution

    Time at event follows uniform distribution (events are uniformly distributed during episode), implying a constant probability density

    for A  t  B

    Or: f(t) = 1/h for 0  t  h and h = B - A


    Uniform distribution38 l.jpg
    Uniform distribution

    Survival function is linear


    Slide39 l.jpg

    Uniform distribution: expectancies

    Since d = 1/h when S(h) = 0



    Slide41 l.jpg

    Uniform distribution: exposure

    a2+b2 = (a+b) (a-b)

    The exposure function associated with the linear survival function is quadratic.


    Slide42 l.jpg

    Uniform distribution: exposure

    Relation between exposure function and survival function:

    where 0F(x,y) is the probability of an event in interval (x,y)


    Slide43 l.jpg

    Uniform distribution: exposure

    Exposure (waiting time to event) in interval (0,h):

    L(h) = h -  f h2 = h S(h) +  f h2 = h [1-  f h]

    Alternative specification:

    L(h) = h S(h) + E[X | 0X h] [1 - S(h)]

    Exposure during interval of length h, provided survival at beginning of interval:


    Slide44 l.jpg

    Uniform distribution: rate

    Since f = 1/ :

    and

    If length of interval is one, rate is 2!!


    Slide45 l.jpg

    Uniform distributionRelation between rate and probability

    Since xF(x,y) =1 - S(y)/S(x) :


    Slide46 l.jpg

    Uniform distributionNumerical illustration

    Let density f = 0.10

    Survival function: S(h) = 1 - f h => 1 - 0.10 h


    Slide47 l.jpg

    1833=2026-0.5*386

    0.0262= 48/1833

    0.9738=1-0.0262


    Slide48 l.jpg

    d=0.00218

    =0.0022113=-ln(0.9738)/12


    Slide49 l.jpg

    4. The exponential distributionDuration = exponential r.v.Time at event = exponential r.v.ln(duration) = uniform r.v.

    Intensity


    Exponential distribution l.jpg
    Exponential distribution

    Time at event is exponentially distributed random variable, implying a constant intensity ()


    Slide51 l.jpg

    Exponential distribution

    Probability density function

    Intensity:

    Distribution function

    Expected value

    Variance

    Number of events during interval: Poisson distribution


    2 2 discrete random variables l.jpg

    2.2 Discreterandom variables


    Nominal variable categories cannot be ordered l.jpg
    Nominal variable:categories cannot be ordered

    • Binomial: two possible values (categories, states)

    • Multinomial: more than two possible values (categories, states)

      • multistate = multinomial


    Slide55 l.jpg

    Bernoulli random variable

    • Binary outcome: two possible outcomes

      • Random experiment: Bernoulli trial (e.g. toss a coin: head/tail)

      • Attribute present/absent

      • Event did occur/did not occur during interval

      • Success/failure


    Bernoulli random variable l.jpg
    Bernoulli random variable

    • Binary random variable: coding

      • 0 and 1: reference category gets 0 (dummy coding; contrast coding)

        • Model parameters are deviations from reference category.

      • -1 and +1 (effect coding: mean is reference category)

        • Model parameters are deviations from the mean.

      • 1 and 2, etc.

    INTERPRETATION

    Vermunt, 1997, p. 10


    Slide57 l.jpg

    Bernoulli random variable

    • Parameter of Bernoulli distribution: expected value of ‘success’: p

    • Variance: p(1-p) = pq

    • Bernoulli process: sequence of independent Bernoulli observations (trials) in homogeneous population [identical B. trials](e.g 30 observations)

      • 10 1 1 0 0 0 1 1 1 1 0 1 0 1 0 1 1 0 1 0 1 1 0 0 1 1 1 0 1


    Slide59 l.jpg

    Binomial distribution

    • COUNT of ‘successes’ in m Bernoulli observations: Nm(Binomial random variable)

    • Probability mass function:

    Binomial distribution with parameter p and index m

    Number of observations is fixed (m)


    Slide60 l.jpg

    Binomial distribution

    • Expected number of ‘successes’: E[Nm]=mp

    • Variance of Nm: Var [Nm]=mp(1-p)

      • Binomial variance is smaller than the mean

    • For one of m observations:

      • Expected value: p

      • Variance: Var[Nm/m] = Var[Nm]/m2 = [p(1-p)]/m

        • Variance decreases with number of observations


    Application the life table l.jpg
    Application: the life table

    If there are l0 individuals who are subject to the same force of mortality, the number lx of survivors at age x is a binomial random variable with probability p0x of surviving to x. The probability distribution of lx is therefore (Chiang, 1984, p. 196):

    k = 0, 1, …, l0


    Application the life table63 l.jpg
    Application: the life table

    Expected value of lx:

    Variance of number of survivors lx:

    Variance of probability of surviving (survival function):


    Application the life table64 l.jpg
    Application: the life table

    If there are lx individuals at age x, the number ly of survivors at age y is a binomial random variable with probability pxy of surviving to y. The probability distribution of ly|x is therefore (Chiang, 1984, p. 197):

    ky = 0, 1, …, lx


    Application the life table65 l.jpg
    Application: the life table

    Expected value of ly:

    Variance of number of survivors ly:

    Variance of probability of surviving (survival function):




    Slide68 l.jpg

    Application: IMR, KeralaIMR = 0.0262 = 1-pp = 0.9738l0 = 2026 (births)

    Expected value of lx:

    Variance of number of survivors lx:

    Variance of probability of surviving (survival function):


    Slide69 l.jpg

    Multinomial random variable

    • Polytomous random variable: more than two possible outcomes

      • Random experiment: e.g. toss a die (6 possible outcomes)

      • Attribute (e.g marital status, region of residence)

    • Sequence of independent observations (trials) in homogeneous population [identical trials](e.g 30 observations, each observation has 3 possible outcomes)

      • 1 2 1 3 2 3 1 3 1 2 2 2 3 1 1 3 2 3 3 1 2 1 1 2 2 3 1 1 2 2

    • (11 times obs. 1; 11 times obs. 2; 8 times obs. 3)


    Slide70 l.jpg

    Multinomial distribution

    • Two categories (binomial):

    Binomial distribution with parameters 1 and2and index m with m = n1+n2


    Slide71 l.jpg

    Multinomial distribution

    • Several categories: I possible outcomes

    With I the number of categories, m the number of observations, ni the number of observations in category/state i and i the probability of an observation being in category I (parameter) [state probability].


    Slide72 l.jpg

    Multinomial distribution

    • Expected number in category i: E[Ni]=mi

    • Variance of Ni: Var [Ni]= mi(1- i)

      • Multinomial variance is smaller than the mean

    • Covariance: Cov[NiNk] = -mi k

    • For one of m observations:

      • Expected value: i

      • Variance: Var[Ni/m] = Var[Ni]/m2 = [i(1- i)]/m

        • Variance decreases with number of observations


    Poisson distribution l.jpg
    Poisson distribution

    • Number of events or occurrences of type i during given period (number of observations not fixed)

    • Probability mass function:

    With ni given number of observations of type i during unit interval, i the expected number of observations during unit interval (parameter).

     = ln  is linear predictor


    Poisson distribution75 l.jpg
    Poisson distribution

    • Expected number of events: E[Ni] = i

    • Variance of Ni: Var[Ni] = i

      • Equality of mean and variance is known as equidispersion

      • Variance greater than mean is known as overdispersion

    • Wheni increases, Poisson distribution approximates normal distribution with mean i and variance i .

    • Binomial distribution with parameter p and index m converges to Poisson distribution with parameter  if m and p 0 (rare events).


    Slide76 l.jpg

    Overdispersion is problem for random variables where variance is related to the mean

    e.g. binomial, Poisson

    In case of overdispersion:

    Var = (1+)  Negative binomial model [NB1]

    Var = (1+ )  Negative binomial model [NB2]

    with  the dispersion parameter

    (Comeron and Trivedi, 1998, p. 63)

    Normal random variable has a separate parameter for dispersion (spread around the mean)


    Discrete time models l.jpg
    Discrete-time models

    • Probability that event occurs during given episode (interval) (event analysis)

      • Bernoulli model (Bernoulli random variable)

      • Binomial model

    • Probability that event occurs in n-th episode (survival analysis)

      • Geometric model (Geometric random variable)


    Geometric distribution l.jpg
    Geometric distribution

    • Time at first ‘success’ in series of observations


    Geometric distribution79 l.jpg
    Geometric distribution

    • Expected time at first ‘success’:

    • Variance:


    Negative binomial distribution l.jpg
    Negative binomial distribution

    • Time at k-th ‘success’ in series of m observations. Sum of independent and identically distributed geometric random variables

    • It is the product of (i) probability of k-1 ‘successes’ and h ‘failures’ in m-1 (=k-1+h) observations [binomial], and (ii) probability of ‘success’p


    Negative binomial distribution82 l.jpg
    Negative binomial distribution

    • Expected time at k-th ‘success’:

    • Variance: