CS433 : Modeling and Simulation
Download

CS433 : Modeling and Simulation

Advertisement
Download Presentation
Comments
jada
From:
|  
(192) |   (0) |   (0)
Views: 31 | Added: 05-11-2012
Rate Presentation: 0 0
Description:

CS433 : Modeling and Simulation

An Image/Link below is provided (as is) to

Download Policy: Content on the Website is provided to you AS IS for your information and personal use only and may not be sold or licensed nor shared on other sites. SlideServe reserves the right to change this policy at anytime. While downloading, If for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.











- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -




2. Markov Processes

3. Classification of States: 1 A path is a sequence of states, where each transition has a positive probability of occurring. State j is reachable (or accessible) (???? ?????? ????) from state i (i?j) if there is a path from i to j ?equivalently Pij (n) > 0 for some n=0, i.e. the probability to go from i to j in n steps is greater than zero. States i and j communicate (i?j) (????) if i is reachable from j and j is reachable from i. (Note: a state i always communicates with itself) A set of states C is a communicating class if every pair of states in C communicates with each other, and no state in C communicates with any state not in C.

4. Classification of States: 1 A state i is said to be an absorbing state if pii = 1. A subset S of the state space X is a closed set if no state outside of S is reachable from any state in S (like an absorbing state, but with multiple states), this means pij = 0 for every i ?S and j ?S A closed set S of states is irreducible(??? ???? ???????) if any state j ?S is reachable from every state i ?S. A Markov chain is said to be irreducible if the state space X is irreducible.

5. Example Irreducible Markov Chain

6. Classification of States: 2 State i is a transient state (???? ?????)if there exists a state j such that j is reachable from i but i is not reachable from j. A state that is not transient is recurrent (???? ??????) . There are two types of recurrent states: Positive recurrent: if the expected time to return to the state is finite. Null recurrent (less common): if the expected time to return to the state is infinite (this requires an infinite number of states). A state i is periodic with period k >1, if k is the smallest number such that all paths leading from state i back to state i have a multiple of k transitions. A state is aperiodic if it has period k =1. A state is ergodic if it is positive recurrent and aperiodic.

7. Classification of States: 2

8. Transient and Recurrent States We define the hitting time Tij as the random variable that represents the time to go from state j to stat i, and is expressed as: k is the number of transition in a path from i to j. Tij is the minimum number of transitions in a path from i to j. We define the recurrence time Tii as the first time that the Markov Chain returns to state i. The probability that the first recurrence to state i occurs at the nth-step is Ti Time for first visit to i given X0 = i. The probability of recurrence to state i is

9. Transient and Recurrent States

10. Transient and Recurrent States We define Ni as the number of visits to state i given X0=i, Theorem: If Ni is the number of visits to state i given X0=i, then Proof

11. Transient and Recurrent States The probability of reaching state j for first time in n-steps starting from X0 = i. The probability of ever reaching j starting from state i is

12. Three Theorems If a Markov Chain has finite state space, then: at least one of the states is recurrent. If state i is recurrent and state j is reachable from state i then: state j is also recurrent. If S is a finite closed irreducible set of states, then: every state in S is recurrent.

13. Positive and Null Recurrent States Let Mi be the mean recurrence time of state i A state is said to be positive recurrent if Mi<8. If Mi=8 then the state is said to be null-recurrent. Three Theorems If state i is positive recurrent and state j is reachable from state i then, state j is also positive recurrent. If S is a closed irreducible set of states, then every state in S is positive recurrent or, every state in S is null recurrent, or, every state in S is transient. If S is a finite closed irreducible set of states, then every state in S is positive recurrent.

14. Example

15. Periodic and Aperiodic States Suppose that the structure of the Markov Chain is such that state i is visited after a number of steps that is an integer multiple of an integer d >1. Then the state is called periodic with period d. If no such integer exists (i.e., d =1) then the state is called aperiodic. Example

16. Steady State Analysis Recall that the state probability, which is the probability of finding the MC at state i after the kth step is given by:

17. 17 Multi-step (t-step) Transitions

18. 18 The Tax Auditing Example

20. 20 Steady-State Solutions ? n Steps

21. 21 Steady-State Solutions ? n Steps

22. 22 Steady State Transition Probability

23. Steady State Analysis Recall the recursive probability

24. Steady State Analysis

25. 25 Comments on Steady-State Results

26. 26 Interpretation of Steady-State Conditions

27. Discrete Birth-Death Example

28. Discrete Birth-Death Example

29. Discrete Birth-Death Example

30. Discrete Birth-Death Example

31. Reducible Markov Chains

32. Reducible Markov Chains

33. Reducible Markov Chains


Other Related Presentations

Copyright © 2014 SlideServe. All rights reserved | Powered By DigitalOfficePro