Markov chains transitional modeling l.jpg
Sponsored Links
This presentation is the property of its rightful owner.
1 / 28

Markov Chains: Transitional Modeling PowerPoint PPT Presentation


  • 123 Views
  • Uploaded on
  • Presentation posted in: General

Markov Chains: Transitional Modeling. Qi Liu. content. Terminology Transitional Models without Explanatory Variables Inference for Markov chains Data Analysis : Example 1 (ignoring explanatory variables) Transitional Models with Explanatory Variables

Download Presentation

Markov Chains: Transitional Modeling

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Markov Chains: Transitional Modeling

Qi Liu


content

  • Terminology

  • Transitional Models without Explanatory Variables

  • Inference for Markov chains

  • Data Analysis :Example 1 (ignoring explanatory variables)

  • Transitional Models with Explanatory Variables

  • Data Anylysis: Example 2 (with explanatory variables)


Terminology

  • Transitional models

  • Markov chain

  • K th-order Markov chain

  • Tansitional probabilitiesand Tansitional matrix


Transitional models

{y0,y1,…,yt-1} are the responses observed previously. Our focus is on the dependence of Yt on the {y0,y1,…,yt-1} as well as any explanatory variables. Models of this type are called transitional models.


Markov chain

  • A stochastic process, for all t, the conditional distribution of Yt+1,given Y0,Y1,…,Yt is identical to the conditional distribution of Yt+1 given Yt alone. i.e, given Yt, Yt+1 is conditional independent of Y0,Y1,…,Yt-1. So knowing the present state of a Markov chain,information about the past states does not help us predict the future

  • P(Yt+1|Y0,Y1,…Yt)=P(Yt+1|Yt)


K th-order Markov chain

  • For all t, the conditional distribution of Yt+1 given Y0,Y1,…,Yt is identical to the conditional distribution of Yt+1 ,given (Yt,…,Yt-k+1)

    P(Yt+1|Y0,Y1,…Yt)=P(Yt+1|Yt-k+1,Yt-k+2,….Yt)

  • i.e, given the states at the previous k times, the future behavior of the chain is independent of past behavior before those k times. We discuss here is first order Markov chain with k=1.


Tansitional probabilities


Transitional Models without Explanatory Variables

At first, we ignore explanatory variables. Let f(y0,…,yT) denote the joint probability mass function of (Y0,…,YT),transitional models use the factorization:

f(y0,…,yT) =f(y0)f(y1|y0)f(y2|y0,y1)…f(yT|y0,y1,…,yT-1)

This model is conditional on the previous responses.

For Markov chains,

f(y0,…,yT) =f(y0)f(y1|y0)f(y2|y1)…f(yT|yT-1) (*)

From it, a Markov chain depends only on one-step transition probabilities and the marginal distribution for the initial state. It also follows that the joint distribution satisfies loglinear model (Y0Y1, Y1Y2,…, YT-1YT)

For a sample of realizations of a stochastic process, a contingency table displays counts of the possible sequences. A test of fit of this loglinear model checks whether the process plausibly satisfies the Markov property.


Inference for Markov chains


Inference for Markov chains(continue)


Example 1 (ignoring explanatory variables)A study at Harvard of effects of air pollution on respiratory illness in children.The children were examined annually at ages 9 through 12 and classified according to the presence or absence of wheeze. Let Yt denote the binary response at age t, t=9,10,11,12.

1 wheeze;2 no wheeze


Code of Example 1

  • Code of 11.7

  • data breath;

  • input y9 y10 y11 y12 count;

  • datalines;

  • 1 1 1 1 94

  • 1 1 1 2 30

  • 1 1 2 1 15

  • 1 1 2 2 28

  • 1 2 1 1 14

  • 1 2 1 2 9

  • 1 2 2 1 12

  • 1 2 2 2 63

  • 2 1 1 1 19

  • 2 1 1 2 15

  • 2 1 2 1 10

  • 2 1 2 2 44

  • 2 2 1 1 17

  • 2 2 1 2 42

  • 2 2 2 1 35

  • 2 2 2 2 572

  • ;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 y9*y10 y10*y11 y11*y12 /dist=poi lrci type3 residuals obstats;

  • run;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 y9*y10 y9*y11 y10*y11 y10*y12 y11*y12 y9*y10*y11 y10*y11*y12/dist=poi lrci type3 residuals obstats;

  • run;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 y9*y10 y9*y11 y9*y12 y10*y11 y10*y12 y11*y12 /dist=poi lrci type3 residuals obstats;

  • run;

  • data breath_new;set breath;

  • a=y9*y10+y10*y11+y11*y12;

  • b=y9*y12+Y10*y12+y9*y11;

  • procgenmod; class y9 y10 y11 y12;

  • model count= y9 y10 y11 y12 a b /dist=poi lrci type3 residuals obstats;

  • run;


Data analysis

  • The loglinear model (y9y10,y10y11,y11y12) a first order Markov chain. P(Y11|Y9,Y10)=P(Y11|Y10)

    P(Y12|Y10,Y11)=P(Y12|Y11)

  • G²=122.9025, df=8, with p-value<0.0001, it fits poorly. So given the state at time t, classification at time t+1 depends on the states at times previous to time t.


Data analysis (cont…)

  • Then we consider model (y9y10y11, y10y11y12),a second-order Markov chain, satisfying conditional independence at ages 9 and 12, given states at ages 10 and 11.

  • This model fits poorly too, with G²=23.8632,df=4 and p-value<0.001.


Data analysis (cont)

  • The loglinear model (y9y10,y9y11,y9y12,y10y11,y10y12,y11y12) that permits association at each pair of ages fits well, with G²=1.4585,df=5,and p-value=0.9178086.

    Parameter Estimate Error Limits Square Pr > ChiSq

    y9*y10 1.8064 0.1943 1.4263 2.1888 86.42 <.0001

    y9*y11 0.9478 0.2123 0.5282 1.3612 19.94 <.0001

    y9*y12 1.0531 0.2133 0.6323 1.4696 24.37 <.0001

    y10*y11 1.6458 0.2093 1.2356 2.0569 61.85 <.0001

    y10*y12 1.0742 0.2205 0.6393 1.5045 23.74 <.000

    y11*y12 1.8497 0.2071 1.4449 2.2574 79.81 <.0001


Data analysis (cont)

  • From above, we see that the association seems similar for pairs of ages1 year apart, and somewhat weaker for pairs of ages more than 1 year apart. So we consider the simpler model in which

  • It also fits well, with G²=2.3, df=9, and p-value= 0.9857876.


Estimated Conditonal Log Odds Ratios


Transitional Models with Explanatory Variables


Data Anylysis

  • Example 2 (with explanatory variables)

  • At ages 7 to 10, children were evaluated annually on the presence of respiratory illness. A predictor is maternal smoking at the start of the study, where s=1 for smoking regularly and s=0 otherwise.


Child’s Respiratory Illness by Age and Maternal Smoking


Data analysis (cont)


data illness;

input t tp ytp yt s count;

datalines;

8 7 0 0 0 266

8 7 0 0 1 134

8 7 0 1 0 28

8 7 0 1 1 22

8 7 1 0 0 32

8 7 1 0 1 14

8 7 1 1 0 24

8 7 1 1 1 17

9 8 0 0 0 274

9 8 0 0 1 134

9 8 0 1 0 24

9 8 0 1 1 14

9 8 1 0 0 26

9 8 1 0 1 18

9 8 1 1 0 26

9 8 1 1 1 21

9 8 1 0 0 26

9 8 1 0 1 18

9 8 1 1 0 26

9 8 1 1 1 21

10 9 0 0 0 283

10 9 0 0 1 140

10 9 0 1 0 17

10 9 0 1 1 12

10 9 1 0 0 30

10 9 1 0 1 21

10 9 1 1 0 20

10 9 1 1 1 14

;

run;

proclogistic descending;

freq count;

model yt = t ytp s/scale=none aggregate;

run;

Code of Example 2


Output from SAS

  • Deviance and Pearson Goodness-of-Fit Statistics

  • Criterion DF Value Value/DF Pr > ChiSq

  • Deviance 8 3.1186 0.3898 0.9267

  • Pearson 8 3.1275 0.3909 0.9261

  • Analysis of Maximum Likelihood Estimates

  • Standard Wald

  • Parameter DF Estimate Error Chi-Square Pr > ChiSq

  • Intercept 1 -0.2926 0.8460 0.1196 0.7295

  • t 1 -0.2428 0.0947 6.5800 0.0103

  • ytp 1 2.2111 0.1582 195.3589 <.0001

  • s 1 0.2960 0.1563 3.5837 0.0583


Analysis


The model fits well, with G²=3.1186, df=8, p-value=0.9267.

The coefficient of is 2.2111 with SE 0.1582 , Chi-Square statistic 195.3589 and p-value <.0001 ,which shows that the previous observation has a strong positive effect. So if a child had illness when he was t-1, he would have more probability to have illness at age t than a child who didn’t have illness at age t-1.

The coefficient of s is 0.2960, the likelihood ratio test of H0 :=0 is 3.5837,df=1,with p-value 0.0583. There is slight evidence of a positive effect of maternal smoking.


Interpratation of Paramters ß


Thank you !


  • Login