# Markov chains - PowerPoint PPT Presentation

1 / 13

Markov chains. Probability distributions. Exercise. Use the Matlab function nchoosek(n,k) to implement a generic function BinomialPMF(k,n,p) for calculating the Binomial PMF with k successes in n trials with probability p.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

### Download Presentation

Markov chains

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## Markov chains

### Exercise

Use the Matlab function nchoosek(n,k) to implement a generic function BinomialPMF(k,n,p) for calculating the Binomial PMF with k successes in n trials with probability p.

Use the barplot function to plot the pmf values (k=0..10) for Bin(10,0.5) as below:

### Markov chains

Example: A very simple weather model

(modified from Wikipedia page ”Examples of Markov chains”)

The probabilities of weather conditions, modeled as either sunny=0 or rainy=1, given the weather on the preceding day, can be represented by a transition matrix

### Weather model example

• The matrix P represents the weather model in which a sunny day is 90% likely to be followed by another sunny day, and a rainy day is 50% likely to be followed by another rainy day. The columns can be labelled ”sunny” and ”rainy” respectively, and the rows can be labelled in the same order.

• Pij is the probability that, if a given day is of type i, it will be followed by a day of type j.

• Note that the rows of P add up to 1:

sum(P’)

### Exercise

• If the the weather on the first day has 50% probability of being sunny or rainy, then what is the probability of the ninth day being sunny? What about the 30th day?

### Eigenvectors / Eigenvalues

We can calculate eigenvalues and eigenvectors in Matlab using the built-in function eig. The default behavior is for right eigenvalues/eigenvectors, but left eigenvalues/eigenvectors are easily obtained by transposing the matrix.

We will be using the form (from Matlab help for eig):

[V,D] = eig(A) produces matrices of eigenvalues (D) and eigenvectors (V) of matrix A, so that A*V = V*D. Matrix D is the canonical form of A a diagonal matrix with A’s eigenvalues on the main diagonal. Matrix V is the modal matrix - its columns are the eigenvectors of A.

Note that Matlab always returns eigenvectors with norm 1.

### Exercise

• Use the eig function to find out, in the long term, what is the percentage of sunny days in the weather example.