Random Walks. An abstraction of student life. Eat. No new ideas. Hungry. Wait. 0.3. Work. 0.4. 0.99. 0.3. 0.01. probability. Work. Solve HW problem. Eat. No new ideas. Hungry. Wait. 0.3. Work. 0.4. 0.99. 0.3. 0.01. Work. Solve HW problem. Markov Decision Processes.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Eat
No new
ideas
Hungry
Wait
0.3
Work
0.4
0.99
0.3
0.01
probability
Work
Solve HW
problem
No new
ideas
Hungry
Wait
0.3
Work
0.4
0.99
0.3
0.01
Work
Solve HW
problem
Markov Decision Processes0.95
0.05
0.5
Working
Broken
0.5






We’ll just walk in a straight line.
0
n
k
0
n
Xt
Xt = k + d1 + d2 + ... + dt,
(di is a RV for the change in your money at time i.)
E[di] = 0, since E[diA] = 0 for all situations A at time i.
So, E[Xt] = k.
0
n
k
0
+ n × Pr(Xt = n)
+ (somethingt × Pr(neither))
0
n
k
What is chance I reach green before red?

Same as voltage if edges are resistors and we put
1volt battery between green and red.

Same as equations for voltage if edges all have same resistance!
0
n
k
1 volt
0 volts
What is chance I reach green before red?

Of course, it holds for general graphs as well…
some other questions on general graphs


If the graph has n nodes and m edges, then
E[ time to visit all nodes ] ≤ 2m × (n1)
E[ time to reach home ] is at most
this
If the graph G has n nodes and m edges, then
the cover time of G is
C(G) ≤ 2m (n – 1)
Pr[ eventually get home ] = 1
Andrei A. Markov
since X is nonneg
Pr[ X exceeds k × expectation ] ≤ 1/k.
Hence, if we know that
Expected Cover Time
C(G) < 2m(n1)
then
Pr[ home by time 4k m(n1) ] ≥ 1 – (½)k
Now for a bound on the cover time of any graph….
Cover Time Theorem
If the graph G has n nodes and m edges, then
the cover time of G is
C(G) ≤ 2m (n – 1)
u
v
0
n
H0,n + Hn,0 = 2n × n
But H0,n = Hn,0 H0,n = n2
u
v
u
v
3
1

5
2
6
4
If the graph G has n nodes and m edges, then
the cover time of G is
C(G) ≤ 2m (n – 1)
Random walks on
infinite graphs
0
i
t
(ti)/2
2 lost forever
p
2
(
(
)
)
n
n
£
2
n
e
p
(
(
)
)
n
£
n
n
e
Unbiased Random Walk0
2t
t
0
Sterling’s
approx.
2t
t
In lost forevern steps, you expect to return to the origin Θ(√n) times!
x
start
y
3
1
2
1
3
2
1
3
2
1
2
3
3
1
2
1
3
2
1
3
2
1
2
3
3
1
2
1
3
2
1
3
2
1
2
3
3
1
2
1
3
2
1
3
2
1
2
3
3
1
2
1
3
2
1
3
2
1
2
3
1
2
1
1
2
2
1
2
1
2
2
1
2
1
2
2
1
1
122
Strings bad for G lost forever1 and start node u
≤ 1/n(3n+1) of
all strings
Strings bad for G1 and start node v
All length 4km(n1) length random strings