1 / 28

Matrix Analytic methods in Markov Modelling

Matrix Analytic methods in Markov Modelling. Continous Time Markov Models. X: R -> X µ Z (integers) X(t): state at time t X : state space (discrete – countable) R : real numbers (continuous time line). Continous Time Markov Models. X is piecewise constant

hammer
Download Presentation

Matrix Analytic methods in Markov Modelling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Matrix Analytic methods inMarkov Modelling

  2. Continous Time Markov Models • X: R -> XµZ (integers) • X(t): state at time t • X: state space (discrete – countable) • R: real numbers (continuous time line)

  3. Continous Time Markov Models • X is piecewise constant • X is typically cadlag ("continue à droite, limite à gauche") = RCLL (“right continuouswithleftlimits”)

  4. Transition probabilities • P(X(t+h)=j|X(t)=i) ¼ h ¸ij for i  j • P(X(t+h)=j|X(t)=j) = 1- i j P(X(t+h)=i|X(t)=j) ¼ 1- hi j¸ji • P(X(t+h)=j)= i P(X(t+h)=j and X(h)=i) = i P(X(t+h)=j | X(h)=i) P(X(h)=i) =  i j ¸ij h P(X(h)=i) + h(1-  i j¸ji) P(X(h)=j)

  5. Transition probabilities • P(X(t+h)=j)=i P(X(t+h)=j and X(h)=i) + (1- k j¸ik) P(X(h)=j) • P(t)=[P(X(t)=0) P(X(t)=1) P(X(t)=2) ..] • P(t+h) ¼ P(t)H • H=I+hQ • Qij=¸ij • Qjj= 1- i j¸ji

  6. Taking limits • P(t+h) ¼ P(t)H • H=I+hQ • P(t+h) ¼ P(t)(I+hQ)=P(t)+hP(t)Q • (P(t+h)-P(t))/h ¼ P(t)Q • d/dt P(t) = P(t)Q • P(t)=P(0)exp(Qt)

  7. Irreducibility • X is irreducible when states are mutually reachable. • X is irreducible iff for every i,j 2X there is a sequence {i(1),i(2),..,I(N) 2X} such that i(1)=i i(N)=j and ¸i(k),i(k+1) > 0 for every k 2 1,..,N-1

  8. Recurrence • Assume X(t n-)=j and X(tn)=i  j then tn is a transition time • Let {tn} be the sequence of consequetive transition times. • {Xn=X(tn)} is called the embedded chain • Let t0=0, X0=i then k(i)=inf{k>0: X(k)=i}=inf{k>1: X(k)=i} • Ti=tk(i) • Ti is the time to next visit at i • X is recurrent if P(Ti<1)=1 for all i (almost certain return to all states) • X is positive recurrent if E(Ti) ·1 for all i

  9. Stationary probability • d/dt P(t) = P(t)Q • P(t)=P(0)exp(Qt) • When X is irreducible and positive recurrent there is a unique probability vector ¦ such that P(t) -> ¦ • ¦ solves ¦ Q=0

  10. Stationary probability • Ergodicity: ¦i = E(Di)/E(Ti) • ¦i is the fraction of the time in state i • Statistically intuitively appealing X(t) ==i time Di Ti

  11. ExamplePoisson Counting Process • Qi,i+1=¸ ¸ ¸ ¸ ¸ 0 1 2 3 • Counts Poisson events • Birth Chain • Not irreducible • Not recurrent

  12. ExampleBirth/Death(BD)-chain • Qi,i+1=¸ • Qi+1,i=¹ ¸ ¸ ¸ ¸ 0 1 2 3 ¹ ¹ ¹ ¹ • Models a queueing system with Poisson arrival process and independent exponentially distributed service times • ¸ is arrival rate • ¹ is service rate • Irreducible • Positive recurrent for ¸<¹

  13. ExampleBirth/Death(BD)-chain ¸ ¸ ¸ ¸ 0 1 2 3 ¹ ¹ ¹ ¹ • ½ = ¸/¹ • ¦n=½¦n-1 • ¦n=½n¦0 • P0=1/n=01½n =1-½ • ¦n=½n (1-½) • E(X) = n=01 n ¦n = ½/(1-½)

  14. ExampleBirth/Death(BD)-chain ¸1 ¸2 ¸3 ¸4 0 1 2 3 ¹1 ¹2 ¹3 ¹4 • ½n = ¸n/¹n • ¦n=½n¦n-1 • ¦n=¦i=1n½i¦0 • P0=1/n=01¦i=1n½i

  15. Markov Modulated Poisson Process • Has two modes: Modes={ON,OFF} • M(t) 2 Modes is a two state CTMC • Transmits with rate ¸ in ON mode. • Counting proces combines state spaces, i.e. X = Modes £ {0,1,2,..} • Q(ON,i),(ON,i+1)=¸ • Q(ON,i),(OFF,i)=¯ • Q(OFF,i),(ON,i)=® • Qi,j=0 otherwise

  16. Markov Modulated Poisson Process • Q(ON,i),(ON,i+1)=¸ • Q(ON,i),(OFF,i)=¯ • Q(OFF,i),(ON,i)=® • Qi,j=0 otherwise OFF 0 1 2 3 ® ¯ ® ¯ ® ¯ ® ¯ ¸ ¸ ¸ ¸ ON 0 1 2 3

  17. MMPP with exponential service • Q(ON,i),(ON,i+1)=¸ • Q(ON,i),(OFF,i)=¯ • Q(OFF,i),(ON,i)=® • Q(ON,i),(ON,i-1)=¹ • Q(OFF,i),(OFF,i-1)=¹ • Qi,j=0 otherwise OFF 0 1 2 3 ¹ ¹ ¹ ¹ ® ¯ ® ¯ ® ¯ ® ¯ ¸ ¸ ¸ ¸ ON 0 1 2 3 ¹ ¹ ¹ ¹

  18. State ordering • For a state (i,M) we denote i the level of the state • We order states so that equal levels are gathered • (i,OFF),(i,ON)(i+1,OFF)(i+1,ON)(i+2,OFF)(i+2,ON)

  19. Generator matrix

  20. Sub matrices

  21. Generator matrix by submatrices Balance equations: P0A0 + P1 B = 0 eq(0) P0 C + P1 A + P2 B = 0 eq(1) Pi C + Pi+1 A + Pi+2 B = 0 eq(i+1) We look for a matrix geometric solution, i.e. P1=P0 R P i+1= PiR Inserting in eq(0): P0A0 + P0 R B = 0 and eq(i+1) Pi(C+R A + R2B)=0 for all Pi

  22. Solving for R • Pi (C+R A + R2 B)=0 for all Pi • Sufficient that C+R A + R2 B=0 (Ricatti equation) • Iterative solution R0=0 repeat Rn+1=-(C+Rn2B) A-1 • Converges for irreducible positive recurrent Q

  23. MM - service • Q(ON,i),(ON,i+1)=¸ • Q(OFF,i),(OFF,i+1)=¸ • Q(ON,i),(OFF,i)=¯ • Q(OFF,i),(ON,i)=® • Q(ON,i),(ON,i-1)=¹ • Q(OFF,i),(OFF,i-1)=0 • Qi,j=0 otherwise ¸ ¸ ¸ ¸ OFF 0 1 2 3 ® ¯ ® ¯ ® ¯ ® ¯ ¸ ¸ ¸ ¸ 0 1 2 3 ON ¹ ¹ ¹ ¹

  24. Generator matrix

  25. Sub matrices

  26. Generally We still look for a matrix geometric solution: Now: i=01RiAi = 0  A0 + R A1i=21RiAi Iteration: Rn = -A1-1 (A0 + i=21 Rn-1iAi) Solving for P0: P0i=01RiBi = 0 Conditions for solution: Irreducibility and pos. recurrence

  27. Miniproject (i) • Let traffic be generated by an on/off Markov process with on rate: ¸=1, mean rate 0.1 ¸, average on time: T=1 • Let service be exponential with rate ¹ = 0.2¸ • Construct the generator matrix for the data given above. • Use the iterative algoritme to solve for the load matrix R • Solve for P0 and Pi • Compute the mean queue length • Compare with M/M/1 results

  28. Miniproject (ii) • Collect file size or web session duration data • Check for power tails and estimate tail power • Find appropriate parameters for a hyperexponential approximation of the reliability estimated reliability • Construct the generator matrix of an equivalent ME/M/1 queue

More Related