Lecture 6. Calculating P n – how do we raise a matrix to the n th power? Ergodicity in Markov Chains. When does a chain have equilibrium probabilities? Balance Equations Calculating equilibrium probabilities without the fuss. The leaky bucket queue
where i are the eigenvalues
Therefore pij(n)=A1n+B2n+ C3n
assuming the eigenvalues are distinct
(where states are no’s 1, 2 and 3)
where I is the identity matrix
Eigenvalues are 1, i/2, -i/2. Therefore p11(n) has the form:
where the substitution can be made since p11(n) must be real
we can calculate that p11(0)=1, p11(1)=0 and p11(2)=0
For what values of and
is this chain irreducible?
By irreducibility there exists r, s 0 with
pji(r),pik(s) > 0
Therefore there is an n such that for all m > n:
And therefore all the states are aperiodic (consider j=k in the above equation).
always exist and are independent of the starting distribution. Either:
=( ½ , ½ ) solves =P
at 1/r seconds)
Exit queue for packets
. . .
. . .
Similarly, we can get expressions for 3 in terms
of 2 ,1 and 0. And so on...
No of iterations
taken to get out
of queue from
Amount of time
spent in given state
Time taken for each
iteration of chain
For those states with queue