scene from the television series “ Time Tunnel ” (1970s). Data Mining and Forecast Management MGMT E-5070. Markov Processes System Change Over Time. Markov Process Models. Also known as Markov Chains. Analyze how systems change over time. Common applications include:.

Download Presentation

Markov Processes System Change Over Time

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

Markov Process Models • Also known as Markov Chains. • Analyze how systems change over time. • Common applications include: • consumer brand loyalty tendencies • consumer brand-switching tendencies • reliability analysis for equipment • the aging / writeoff of accounts receivable • the spoilage tendencies of perishable items over time

Andrei Andreyevich Markov ( 1856 – 1922 ) • Ph.D , St. Petersburg University (1884) • Studied under Pafnuty Chebyshev. • Professor, St. Petersburg University (1886-1905) , but taught informally until 1922. • Early work in number theory, algebraic continual fractions, limits of integrals, and least squares method. • Launched the theory of stochastic processes ( Markov Chains ) : an all new branch of probability theory. Андрєй AндрєєвичМарков

The states and transitions are “discrete” Markov Process Models DISCRETE This means, for example, that market share among different stores can only change once per week or once per month, but not second - to - second.

The number of states is “finite” Markov Process Models FINITE Means, for example, that an accounts receivable can only age 3 months before it is written off as a bad debt. Once a bottle of wine spoils, there are no additional aging periods for it.

Markov Process Models STATIONARY Transitions depend only on the current state……….not on prior states The chances of an accounts receivable being paid off are higher if it is one month old as opposed to two months old.

Markov Process Models FIRST-ORDER PROCESSES Transition probabilities remain constant over time For example, the defection rates of customers from a local supermarket to its competitors month-by-month will always stay the same.

Markov Process Models BASIC EXAMPLE TWO BARBERS IN A SMALL TOWN EACH HAVE A 50% MARKET SHARE. THEREFORE, THE VECTOR OF STATE PROBABILITIES AT THIS TIME IS: π ( 1 ) =(0.5,0.5) PERIOD NUMBER ONE 50% MARKET SHARE FOR BARBER “A” 50% MARKET SHARE FOR BARBER “B”

Markov Process Models BASIC EXAMPLE THE MATRIX OF TRANSITION PROBABILITIES PER MONTH ARE: LOYALTY PROBABILITY FOR BARBER “A” IN ANY GIVEN PERIOD DEFECTION PROBABILITY FOR BARBER “A” IN ANY GIVEN PERIOD [ ] .90 .10 P = .25 .75 DEFECTION PROBABILITY FOR BARBER “B” IN ANY GIVEN PERIOD LOYALTY PROBABILITY FOR BARBER “B” IN ANY GIVEN PERIOD

Markov Process Models BASIC EXAMPLE THE MARKET SHARE EACH BARBER HAS IN THE 2nd MONTH IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD ( MONTH ) 1 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIES: π ( 2 ) = π ( 1 ) x P [ ] .45 .05 BARBER ‘A’ BARBER ‘B’ .90 .10 ( 0.5 , 0.5 ) = ( 0.575 , 0.425 ) .125 .375 .25 .75 BARBER ‘A’ BARBER ‘B’ .575 .425

Markov Process Models BASIC EXAMPLE THE MARKET SHARE EACH BARBER HAS IN THE 3rd MONTH IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD (MONTH) 2 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIES: π ( 3 ) = π ( 2 ) x P [ ] .5175 .0575 BARBER ‘A’ BARBER ‘B’ .90 .10 ( 0.575 , 0.425 ) = ( 0.62 , 0.38 ) .10625 .31875 .25 .75 BARBER ‘A’ BARBER ‘B’ .62375 .37625

Markov Process Models BASIC EXAMPLE THE MARKET SHARE EACH BARBER HAS IN THE 4th MONTH IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD (MONTH) 3 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIES: π ( 4 ) = π ( 3 ) x P [ ] .558 .062 BARBER ‘A’ BARBER ‘B’ .90 .10 ( 0.62 , 0.38 ) = ( 0.653 , 0.347 ) .095 .285 BARBER ‘A’ BARBER ‘B’ .25 .75 .653 .347

Steady – State Probabilities Reached when the before and after state probabilities stay the same forever, assuming no changes in the matrix oftransition probabilities Here, the eventual market share in the “barber” problem. ALSO KNOWN AS THE EQUILIBRIUM ORSTEADY-STATE SOLUTION

Equilibrium Condition BARBER EXAMPLE • The eventual market shares of the two barbers can be calculated directly from the matrix of transition probabilities. • Prior period vectors of stateprobabilities are not required. • The equations: .90 π1 + .25 π2 = π1 .10 π1 + .75 π2 = π2 [ ] .90 .10 .25 .75 MATRIX OF TRANSITION

Equilibrium Condition BARBER EXAMPLE Since π1 + π2 = 1.0 , π1 = 1.0 – π2 Therefore, “ 1.0 – π2 “ may be substituted for π1 in either of the two equations below: .90 ( 1 – π2 ) + .25 π2 = 1 – π2 .10 ( 1 – π2 ) + .75 π2 = π2 or

Equilibrium Condition BARBER EXAMPLE SUBSTITUTING IN THE 1st EQUATION, WE GET: .90 ( 1 – π2 ) + .25 π2 = 1 – π2 .90 - .90 π2 + .25 π2 = 1 – π2 - .10 = - .35 π2 π2 = .2857 and π1 = ( 1 - .2857 ) = .7143 EVENTUALLY BARBER ‘A’ WILL HAVE 71% OF THE MARKET WHILE BARBER ‘B’ WILL HAVE THE REMAINING 29%

We specify the number of states. Here, it is the market share for the two barbers

We desire to find the market shares over “4” periods ( months ) The “ initial “ market shares (loyalty rates) are 50% / 50% respectively. ( Barber “A” is “1” ) ( Barber “B” is “2”) The Matrix of Transition is inserted to the right.

The “ End of Period 1 “ is actually the end of period “2” The “ End of Period 2 “ Is actually the end of period “3” , etc. The market share ( loyalty rates ) after 4 months: Barber A ( 1 ) - 66% Barber B ( 2 ) - 34%

Gas Station Example MARKOV PROCESSES A town has three gas stations: A,B,C. The only factor influencing the choice of station for the next purchase is the prior purchase. Each station is concerned about brand share. The town’s weekly gas sales are $10,000.00, and we assume each driver buys gas once per week.

Gas Station Example MARKOV PROCESSES THE MARKET SHARES AT THIS PARTICULAR TIME ARE AS FOLLOWS: STATION ‘A’ - 30% STATION ‘B’ - 40% STATION ‘C’ - 30% THEREFORE, THE VECTOR OF STATE PROBABILITIES IS: π (1) = ( 0.3 , 0.4 , 0.3 ) PERIOD NUMBER ONE

Gas Station Example MARKOV PROCESSES THE MATRIX OF TRANSITIONPROBABILITIES PER WEEK ARE: LOYALTY RATE STATION ‘A’ LOYALTY RATE STATION ‘B’ .90.05 .05 .10.80.10 .20 .10.70 P = LOYALTY RATE STATION ‘C’ ALL OTHERS ARE DEFECTION RATES

Gas Station Example THE MARKET SHARE THAT EACH GAS STATION HAS IN THE NEXT WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATEPROBABILITIES IN PERIOD ( WEEK ) 1 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIES: π ( 2 ) = π ( 1 ) x P P (A) = 0.30 A .90 .05 .05 .27 .015 .015 P (B) = 0.40 X B .10 .80 .10 = .04 .32 .04 P (C) = 0.30 C .20 .10 .70 .06 .03 .21 .37 .365 .265 π ( 1 ) P A B C

Gas Station Example THE MARKET SHARE THAT EACH GAS STATION HAS IN THE 3rd WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATEPROBABILITIES IN PERIOD ( WEEK ) 2 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIES: π ( 3 ) = π ( 2 ) x P P (A) = 0.37 A .90 .05 .05 .333 .018 .018 P (B) = 0.365 X B .10 .80 .10 = .037 .292 .037 P (C) = 0.265 C .20 .10 .70 .053 .027 .185 .423 .337 .24 π ( 2 ) P A B C

Gas Station Example THE MARKET SHARE THAT EACH GAS STATION HAS IN THE 4th WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATE PROBABILITIES IN PERIOD ( WEEK ) 3 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIES: π ( 4 ) = π ( 3 ) x P P (A) = 0.423 A .90 .05 .05 .381 .021 .021 P (B) = 0.337 X B .10 .80 .10 = .034 .269 .034 P (C) = 0.240 C .20 .10 .70 .048 .024 .168 .463 .314 .223 π ( 3 ) P A B C

Steady–State Probabilities WE KNOW THAT WE HAVE ARRIVED AT THE EVENTUAL MARKET SHARES WHEN THE STARTING PROBABILITIES ARE EQUAL TO THE NEXT STATE PROBABILITIES: π ( Final State ) = π ( Starting ) x P P (A) = 0.589 A .90 .05 .05 .530 .029 .029 P (B) = 0.235X B .10 .80 .10 = .024 .188 .024 P (C) = 0.176 C .20 .10 .70 .035 .018 .123 .589 .235 .176 π ( starting ) P A B C

Gas Station Example CONCLUSION Unless there is some change, the share of market will stay approximately 58.9% for station ‘A’, 23.5% for station ‘B’, and 17.6% for station ‘C’. Given that the total weekly gas sales are $10,000.00, the average weekly sales per station are: A : ( 0.589 ) ( 10,000 ) = $5,890.00 B : ( 0.235 ) ( 10,000 ) = $2,350.00 C : ( 0.176 ) ( 10,000 ) = $1,760.00

Calculation of Steady-State Probabilities GAS STATION EXAMPLE INITIAL STATE PROBABILITIES TRANSITION PROBABILITIES NEW STATE PROBABILITIES = X A B C .90 .05 .05 .10 .80 .10 .20 .10 .70 A B C .90X1 .05X1 .05X1 .10X2 .80X2 .10X2 .20X3 .10X3 .70X3 X1 X2 X3 A B C P(A) = X1 P(B) = X2 P(C) = X3