Download
1 / 60
600 likes | 693 Views
A brief review. The exponential distribution. The memoryless property. Exponentially distributed random variables are memoryless. The exponential distribution is the only distribution that has the memoryless property. The minimum of n exponentially distributed random variables.
E N D
Exponentially distributed random variables are memoryless.
The exponential distribution is the only distribution that has the memoryless property.
The minimum of n exponentially distributed random variables Suppose that X1, X2, ..., Xn are independent exponential random variables, with Xi having rate li, i=1, ..., n. What is P(min(X1, X2, ..., Xn )>x)?
Comparing two exponentially distributed random variables Suppose that X1 and X2 are independent exponentially distributed random variables with rates l1 and l2. What is P(X1 < X2)?
The Poisson process The counting process {N(t) t ≥ 0} is said to be a Poisson process having rate l, l > 0, if (i)N(0) = 0. (ii) The process has independent increments. (iii) The number of events in any interval of length t is Poisson distributed with mean lt. That is for all s, t ≥ 0
The distribution of interarrival times for a Poisson process Let Tn denote the inter-arrival time between the (n-1)th event and the nth event of a Poisson process, then the Tn (n=1, 2, ...) are independent, identically distributed exponential random variables having mean 1/l.
A CTMC is a continuous time analog to a discrete time Markov chain • A CTMC is defined with respect to a continuous time stochastic process {X(t): t ≥0} • If X(t) = i the process is said to be in state i at time t • {i: i=0, 1, 2, ...} is the state space
A stochastic process {X(t): t ≥0} is a continuous time Markov chain if for all s, t , u ≥0 and 0 ≤ u < s • P{X(t+s)=j|X(s)=i, X(u)=x(u), 0 ≤ u < s} = P{X(t+s)=j|X(s)=i}
A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t).
A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t). • Pij(t) = P{X(t+s)=j|X(s)=i}
A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t). • Pij(t) = P{X(t+s)=j|X(s)=i} • Note: We shall always assume that the stationary property holds
Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable).
Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable). • Example 1:X(0)=2, the first three transitions occur at t1 =3, t1 = 4.2 and t1 = 6.5 and X(3)=4, X(4.2) = 2 and X(6.5) =1.
Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable). • Example 1:X(0)=2, the first three transitions occur at t1 =3, t2 = 4.2 and t3 = 6.5 and X(3)=4, X(4.2) = 2 and X(6.5) =1. • The first sojourn time in state 4 = 4.2-3 = 1.2 • The second sojourn time in state 2 = 6.5 – 4.2 = 1.3
Example 2: Suppose the process has been in state 3 for 10 minutes, what is the probability that it will not leave state 3 in the next 5 minutes.
Example 2: Suppose the process has been in state 3 for 10 minutes, what is the probability that it will not leave state 3 in the next 5 minutes. • P(T3 > 15|T3> 10) = P(T3 > 5)
More generally, • P(Ti > s+t|Ti> s) = P(Ti > t) • Ti is memoryless and therefore has the exponential distribution
An alternative definition of a CTMC A CTMC is a stochastic process having the properties that each time it enters a state i (i) the amount of time it spends in that state before making a transition into a different state is exponentially distributed with some mean 1/vi (or transition rate vi ) (ii) when the process leaves state i, it next enters state j with some probability Pij(Pii=0 and SjPij=1, for all i)
A CTMC is a stochastic process that moves from state to state according to a probability transition matrix (similar to a discrete time Markov chain) , but the amount of time it spends in each state is exponentially distributed.
To define a CTMC, we need to define a state space, a probability transition matrix, and a set of transition rates.
Example 1: Customers arrive to a store according to a Poisson process with rate l. Let N(t) be the total number of customers that have arrived by time t.
Example 1: Customers arrive to a store according to a Poisson process with rate l. Let N(t) be the total number of customers that have arrived by time t. State space is {0, 1, 2, ...}; Ti is exponentially distributed with mean 1/l Pij =1 if j=i+1 and Pij = 0 otherwise
Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.
Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m. State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ...
Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m. State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ... The above is an example of an M/M/1 queue. The M/M/1 queue is an example of a birth and death process.
Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m. State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ... The above is an example of an M/M/1 queue. The M/M/1 queue is an example of a birth and death process.
Birth and death process Example 3: Customers arrive to a service center according to a Poisson process with rate lnwhen there are n customers in the system. Customers take an amount Tn that is exponentially distributed with mean 1/mn when there are n customers in the system.
Birth and death process Example 3: Customers arrive to a service center according to a Poisson process with rate lnwhen there are n customers in the system. Customers take an amount Tn that is exponentially distributed with mean 1/mn when there are n customers in the system. State space is {0, 1, 2, ...} P0,1 = 1; Pn,n+1=ln/(ln+mn); Pn,n-1=mn/(ln+mn) T0 =1/l0v0 =l0; Tn = 1/(ln +mn) vn =ln+mnfor n =1, 2, ...
State transition diagrams for a B&D process l0 l1 l2 0 1 2 3 m1 m2 m3
The Poisson process is a birth and death process with rate ln=l and mn=0. • The M/M/1 queue is described by a birth and death process with rate ln=l and mn=m.
Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m.
Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m. The system can be modeled as a birth and death process with transition rates ln = l; mn = nm if 1 ≤ n < m mn = mm if n ≥ m
Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m. The system can be modeled as a birth and death process with transition rates ln = l; mn = nm if 1 ≤ n < m mn = mm if n ≥ m The above is an example of an M/M/m queue.
Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i) qij = Pijvi (qij is also called the instantaneous transition rate from state i to j)
Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i) qij = Pijvi (qij is also called the instantaneous transition rate from state i to j) vi =Sj viPij= Sj qij Pij =qij/vi =qij/Sj qij
Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i) qij = Pijvi (qij is also called the instantaneous transition rate from state i to j) vi =Sj viPij= Sj qij Pij =qij/vi =qij/Sj qij Specifying the instantaneous transition rates determines the parameters of the CTMC
State transition diagrams q0,2 q0,1 q1,2 0 1 2 q2,0 q2,1 q2,0
Properties It can also be shown that
