1 / 60

A brief review

A brief review. The exponential distribution. The memoryless property.  Exponentially distributed random variables are memoryless. The exponential distribution is the only distribution that has the memoryless property. The minimum of n exponentially distributed random variables.

A brief review

E N D

Presentation Transcript

1. A brief review

2. The exponential distribution

3. The memoryless property

4. The exponential distribution is the only distribution that has the memoryless property.

5. The minimum of n exponentially distributed random variables Suppose that X1, X2, ..., Xn are independent exponential random variables, with Xi having rate li, i=1, ..., n. What is P(min(X1, X2, ..., Xn )>x)?

6. Comparing two exponentially distributed random variables Suppose that X1 and X2 are independent exponentially distributed random variables with rates l1 and l2. What is P(X1 < X2)?

7. The Poisson process The counting process {N(t) t ≥ 0} is said to be a Poisson process having rate l, l > 0, if (i)N(0) = 0. (ii) The process has independent increments. (iii) The number of events in any interval of length t is Poisson distributed with mean lt. That is for all s, t ≥ 0

8. The distribution of interarrival times for a Poisson process Let Tn denote the inter-arrival time between the (n-1)th event and the nth event of a Poisson process, then the Tn (n=1, 2, ...) are independent, identically distributed exponential random variables having mean 1/l.

9. A CTMC is a continuous time analog to a discrete time Markov chain • A CTMC is defined with respect to a continuous time stochastic process {X(t): t ≥0} • If X(t) = i the process is said to be in state i at time t • {i: i=0, 1, 2, ...} is the state space

10. A stochastic process {X(t): t ≥0} is a continuous time Markov chain if for all s, t , u ≥0 and 0 ≤ u < s • P{X(t+s)=j|X(s)=i, X(u)=x(u), 0 ≤ u < s} = P{X(t+s)=j|X(s)=i}

11. A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t).

12. A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t). •  Pij(t) = P{X(t+s)=j|X(s)=i}

13. A CTMC is said to have stationary transition probabilities if • P{X(t+s)=j|X(s)=i} • is independent of s (and depends only on t). •  Pij(t) = P{X(t+s)=j|X(s)=i} • Note: We shall always assume that the stationary property holds

14. Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable).

15. Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable). • Example 1:X(0)=2, the first three transitions occur at t1 =3, t1 = 4.2 and t1 = 6.5 and X(3)=4, X(4.2) = 2 and X(6.5) =1.

16. Sojourn times • Ti: time the process spends in state i once it enters state i (the length of a visit to state i, a random variable). • Example 1:X(0)=2, the first three transitions occur at t1 =3, t2 = 4.2 and t3 = 6.5 and X(3)=4, X(4.2) = 2 and X(6.5) =1. •  The first sojourn time in state 4 = 4.2-3 = 1.2 • The second sojourn time in state 2 = 6.5 – 4.2 = 1.3

17. Example 2: Suppose the process has been in state 3 for 10 minutes, what is the probability that it will not leave state 3 in the next 5 minutes.

18. Example 2: Suppose the process has been in state 3 for 10 minutes, what is the probability that it will not leave state 3 in the next 5 minutes. • P(T3 > 15|T3> 10) = P(T3 > 5)

19. More generally, • P(Ti > s+t|Ti> s) = P(Ti > t) •  Ti is memoryless and therefore has the exponential distribution

20. An alternative definition of a CTMC A CTMC is a stochastic process having the properties that each time it enters a state i (i) the amount of time it spends in that state before making a transition into a different state is exponentially distributed with some mean 1/vi (or transition rate vi ) (ii) when the process leaves state i, it next enters state j with some probability Pij(Pii=0 and SjPij=1, for all i)

21. A CTMC is a stochastic process that moves from state to state according to a probability transition matrix (similar to a discrete time Markov chain) , but the amount of time it spends in each state is exponentially distributed.

22. To define a CTMC, we need to define a state space, a probability transition matrix, and a set of transition rates.

23. Example 1: Customers arrive to a store according to a Poisson process with rate l. Let N(t) be the total number of customers that have arrived by time t.

24. Example 1: Customers arrive to a store according to a Poisson process with rate l. Let N(t) be the total number of customers that have arrived by time t.  State space is {0, 1, 2, ...}; Ti is exponentially distributed with mean 1/l Pij =1 if j=i+1 and Pij = 0 otherwise

25. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.

26. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.  State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ...

27. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.  State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ... The above is an example of an M/M/1 queue. The M/M/1 queue is an example of a birth and death process.

28. Example 2: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes the single agent at the counter to check-in a customer is exponentially distributed with mean 1/m.  State space is {0, 1, 2, ...} P0,1 = 1; Pi,i+1 = l/(l+m); Pi,i-1 = m/(l+m) T0 =1/l v0 =l ; Ti = 1/(l+m) vi =l + m for i =1, 2, ... The above is an example of an M/M/1 queue. The M/M/1 queue is an example of a birth and death process.

29. Birth and death process Example 3: Customers arrive to a service center according to a Poisson process with rate lnwhen there are n customers in the system. Customers take an amount Tn that is exponentially distributed with mean 1/mn when there are n customers in the system.

30. Birth and death process Example 3: Customers arrive to a service center according to a Poisson process with rate lnwhen there are n customers in the system. Customers take an amount Tn that is exponentially distributed with mean 1/mn when there are n customers in the system.  State space is {0, 1, 2, ...} P0,1 = 1; Pn,n+1=ln/(ln+mn); Pn,n-1=mn/(ln+mn) T0 =1/l0v0 =l0; Tn = 1/(ln +mn) vn =ln+mnfor n =1, 2, ...

31. State transition diagrams for a B&D process l0 l1 l2 0 1 2 3 m1 m2 m3

32. The Poisson process is a birth and death process with rate ln=l and mn=0. • The M/M/1 queue is described by a birth and death process with rate ln=l and mn=m.

33. Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m.

34. Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m.  The system can be modeled as a birth and death process with transition rates ln = l; mn = nm if 1 ≤ n < m mn = mm if n ≥ m

35. Example 4: Customers arrive to an airline check-in counter according to a Poisson process with rate l. The time it takes one of the m agents at the counter to check-in a customer is exponentially distributed with mean 1/m.  The system can be modeled as a birth and death process with transition rates ln = l; mn = nm if 1 ≤ n < m mn = mm if n ≥ m The above is an example of an M/M/m queue.

36. Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i)  qij = Pijvi (qij is also called the instantaneous transition rate from state i to j)

37. Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i)  qij = Pijvi (qij is also called the instantaneous transition rate from state i to j)  vi =Sj viPij= Sj qij Pij =qij/vi =qij/Sj qij

38. Transition rates vi: rate with which the process leaves state i (once it enters state i) qij: rate with which the process goes state j (once it enters state i)  qij = Pijvi (qij is also called the instantaneous transition rate from state i to j)  vi =Sj viPij= Sj qij Pij =qij/vi =qij/Sj qij Specifying the instantaneous transition rates determines the parameters of the CTMC

39. State transition diagrams q0,2 q0,1 q1,2 0 1 2 q2,0 q2,1 q2,0

40. Properties It can also be shown that

41. The Chapman-Kolmogrov equations

42. The Chapman-Kolmogrov equations

43. The Chapman-Kolmogrov equations

44. The Chapman-Kolmogrov equations

45. The Chapman-Kolmogrov equations

46. The Chapman-Kolmogrov equations

47. Kolmogrov’s backward equations

48. Kolmogrov’s backward equations

More Related