Markov chains transitional modeling l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 28

Markov Chains: Transitional Modeling PowerPoint PPT Presentation


  • 115 Views
  • Uploaded on
  • Presentation posted in: General

Markov Chains: Transitional Modeling. Qi Liu. content. Terminology Transitional Models without Explanatory Variables Inference for Markov chains Data Analysis : Example 1 (ignoring explanatory variables) Transitional Models with Explanatory Variables

Download Presentation

Markov Chains: Transitional Modeling

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Markov chains transitional modeling l.jpg

Markov Chains: Transitional Modeling

Qi Liu


Content l.jpg

content

  • Terminology

  • Transitional Models without Explanatory Variables

  • Inference for Markov chains

  • Data Analysis :Example 1 (ignoring explanatory variables)

  • Transitional Models with Explanatory Variables

  • Data Anylysis: Example 2 (with explanatory variables)


Terminology l.jpg

Terminology

  • Transitional models

  • Markov chain

  • K th-order Markov chain

  • Tansitional probabilitiesand Tansitional matrix


Transitional models l.jpg

Transitional models

{y0,y1,…,yt-1} are the responses observed previously. Our focus is on the dependence of Yt on the {y0,y1,…,yt-1} as well as any explanatory variables. Models of this type are called transitional models.


Markov chain l.jpg

Markov chain

  • A stochastic process, for all t, the conditional distribution of Yt+1,given Y0,Y1,…,Yt is identical to the conditional distribution of Yt+1 given Yt alone. i.e, given Yt, Yt+1 is conditional independent of Y0,Y1,…,Yt-1. So knowing the present state of a Markov chain,information about the past states does not help us predict the future

  • P(Yt+1|Y0,Y1,…Yt)=P(Yt+1|Yt)


K th order markov chain l.jpg

K th-order Markov chain

  • For all t, the conditional distribution of Yt+1 given Y0,Y1,…,Yt is identical to the conditional distribution of Yt+1 ,given (Yt,…,Yt-k+1)

    P(Yt+1|Y0,Y1,…Yt)=P(Yt+1|Yt-k+1,Yt-k+2,….Yt)

  • i.e, given the states at the previous k times, the future behavior of the chain is independent of past behavior before those k times. We discuss here is first order Markov chain with k=1.


Tansitional probabilities l.jpg

Tansitional probabilities


Transitional models without explanatory variables l.jpg

Transitional Models without Explanatory Variables

At first, we ignore explanatory variables. Let f(y0,…,yT) denote the joint probability mass function of (Y0,…,YT),transitional models use the factorization:

f(y0,…,yT) =f(y0)f(y1|y0)f(y2|y0,y1)…f(yT|y0,y1,…,yT-1)

This model is conditional on the previous responses.

For Markov chains,

f(y0,…,yT) =f(y0)f(y1|y0)f(y2|y1)…f(yT|yT-1) (*)

From it, a Markov chain depends only on one-step transition probabilities and the marginal distribution for the initial state. It also follows that the joint distribution satisfies loglinear model (Y0Y1, Y1Y2,…, YT-1YT)

For a sample of realizations of a stochastic process, a contingency table displays counts of the possible sequences. A test of fit of this loglinear model checks whether the process plausibly satisfies the Markov property.


Inference for markov chains l.jpg

Inference for Markov chains


Inference for markov chains continue l.jpg

Inference for Markov chains(continue)


Slide11 l.jpg

Example 1 (ignoring explanatory variables)A study at Harvard of effects of air pollution on respiratory illness in children.The children were examined annually at ages 9 through 12 and classified according to the presence or absence of wheeze. Let Yt denote the binary response at age t, t=9,10,11,12.

1 wheeze;2 no wheeze


Code of example 1 l.jpg

Code of Example 1

  • Code of 11.7

  • data breath;

  • input y9 y10 y11 y12 count;

  • datalines;

  • 1 1 1 1 94

  • 1 1 1 2 30

  • 1 1 2 1 15

  • 1 1 2 2 28

  • 1 2 1 1 14

  • 1 2 1 2 9

  • 1 2 2 1 12

  • 1 2 2 2 63

  • 2 1 1 1 19

  • 2 1 1 2 15

  • 2 1 2 1 10

  • 2 1 2 2 44

  • 2 2 1 1 17

  • 2 2 1 2 42

  • 2 2 2 1 35

  • 2 2 2 2 572

  • ;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 y9*y10 y10*y11 y11*y12 /dist=poi lrci type3 residuals obstats;

  • run;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 y9*y10 y9*y11 y10*y11 y10*y12 y11*y12 y9*y10*y11 y10*y11*y12/dist=poi lrci type3 residuals obstats;

  • run;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 y9*y10 y9*y11 y9*y12 y10*y11 y10*y12 y11*y12 /dist=poi lrci type3 residuals obstats;

  • run;

  • data breath_new;set breath;

  • a=y9*y10+y10*y11+y11*y12;

  • b=y9*y12+Y10*y12+y9*y11;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 a b /dist=poi lrci type3 residuals obstats;

  • run;


Data analysis l.jpg

Data analysis

  • The loglinear model (y9y10,y10y11,y11y12) a first order Markov chain. P(Y11|Y9,Y10)=P(Y11|Y10)

    P(Y12|Y10,Y11)=P(Y12|Y11)

  • G²=122.9025, df=8, with p-value<0.0001, it fits poorly. So given the state at time t, classification at time t+1 depends on the states at times previous to time t.


Data analysis cont l.jpg

Data analysis (cont…)

  • Then we consider model (y9y10y11, y10y11y12),a second-order Markov chain, satisfying conditional independence at ages 9 and 12, given states at ages 10 and 11.

  • This model fits poorly too, with G²=23.8632,df=4 and p-value<0.001.


Data analysis cont15 l.jpg

Data analysis (cont)

  • The loglinear model (y9y10,y9y11,y9y12,y10y11,y10y12,y11y12) that permits association at each pair of ages fits well, with G²=1.4585,df=5,and p-value=0.9178086.

    Parameter Estimate Error Limits Square Pr > ChiSq

    y9*y10 1.8064 0.1943 1.4263 2.1888 86.42 <.0001

    y9*y11 0.9478 0.2123 0.5282 1.3612 19.94 <.0001

    y9*y12 1.0531 0.2133 0.6323 1.4696 24.37 <.0001

    y10*y11 1.6458 0.2093 1.2356 2.0569 61.85 <.0001

    y10*y12 1.0742 0.2205 0.6393 1.5045 23.74 <.000

    y11*y12 1.8497 0.2071 1.4449 2.2574 79.81 <.0001


Data analysis cont16 l.jpg

Data analysis (cont)

  • From above, we see that the association seems similar for pairs of ages1 year apart, and somewhat weaker for pairs of ages more than 1 year apart. So we consider the simpler model in which

  • It also fits well, with G²=2.3, df=9, and p-value= 0.9857876.


Estimated conditonal log odds ratios l.jpg

Estimated Conditonal Log Odds Ratios


Transitional models with explanatory variables l.jpg

Transitional Models with Explanatory Variables


Data anylysis l.jpg

Data Anylysis

  • Example 2 (with explanatory variables)

  • At ages 7 to 10, children were evaluated annually on the presence of respiratory illness. A predictor is maternal smoking at the start of the study, where s=1 for smoking regularly and s=0 otherwise.


Child s respiratory illness by age and maternal smoking l.jpg

Child’s Respiratory Illness by Age and Maternal Smoking


Data analysis cont22 l.jpg

Data analysis (cont)


Code of example 2 l.jpg

data illness;

input t tp ytp yt s count;

datalines;

8 7 0 0 0 266

8 7 0 0 1 134

8 7 0 1 0 28

8 7 0 1 1 22

8 7 1 0 0 32

8 7 1 0 1 14

8 7 1 1 0 24

8 7 1 1 1 17

9 8 0 0 0 274

9 8 0 0 1 134

9 8 0 1 0 24

9 8 0 1 1 14

9 8 1 0 0 26

9 8 1 0 1 18

9 8 1 1 0 26

9 8 1 1 1 21

9 8 1 0 0 26

9 8 1 0 1 18

9 8 1 1 0 26

9 8 1 1 1 21

10 9 0 0 0 283

10 9 0 0 1 140

10 9 0 1 0 17

10 9 0 1 1 12

10 9 1 0 0 30

10 9 1 0 1 21

10 9 1 1 0 20

10 9 1 1 1 14

;

run;

proclogistic descending;

freq count;

model yt = t ytp s/scale=none aggregate;

run;

Code of Example 2


Output from sas l.jpg

Output from SAS

  • Deviance and Pearson Goodness-of-Fit Statistics

  • Criterion DF Value Value/DF Pr > ChiSq

  • Deviance 8 3.1186 0.3898 0.9267

  • Pearson 8 3.1275 0.3909 0.9261

  • Analysis of Maximum Likelihood Estimates

  • Standard Wald

  • Parameter DF Estimate Error Chi-Square Pr > ChiSq

  • Intercept 1 -0.2926 0.8460 0.1196 0.7295

  • t 1 -0.2428 0.0947 6.5800 0.0103

  • ytp 1 2.2111 0.1582 195.3589 <.0001

  • s 1 0.2960 0.1563 3.5837 0.0583


Analysis l.jpg

Analysis


Slide26 l.jpg

The model fits well, with G²=3.1186, df=8, p-value=0.9267.

The coefficient of is 2.2111 with SE 0.1582 , Chi-Square statistic 195.3589 and p-value <.0001 ,which shows that the previous observation has a strong positive effect. So if a child had illness when he was t-1, he would have more probability to have illness at age t than a child who didn’t have illness at age t-1.

The coefficient of s is 0.2960, the likelihood ratio test of H0 :=0 is 3.5837,df=1,with p-value 0.0583. There is slight evidence of a positive effect of maternal smoking.


Interpratation of paramters l.jpg

Interpratation of Paramters ß


Slide28 l.jpg

Thank you !


  • Login