Download
1 / 21
210 likes | 281 Views
Ｐａｒｔｉａｌ Ｓｏｌｕｔｉｏｎ ａｎｄ Ｅｎｔｒｏｐｙ. Tadao Takaoka Department of Computer Science University of Canterbury Christchurch, New Zealand. Genera framework and motivation. If the given problem is partially solved, how much more time is needed to solve the problem completely.
E N D
Ｐａｒｔｉａｌ Ｓｏｌｕｔｉｏｎ ａｎｄ ＥｎｔｒｏｐｙＰａｒｔｉａｌ Ｓｏｌｕｔｉｏｎ ａｎｄ Ｅｎｔｒｏｐｙ Tadao Takaoka Department of Computer Science University of Canterbury Christchurch, New Zealand
Genera framework and motivation • If the given problem is partially solved, how much more time is needed to solve the problem completely. Time =Time spent + Time to be spent Output Input data Algorithm Save Recover Saved data A possible scenario: Suppose computer is stopped by power cut and partially solved data are saved by battery. After a while power is back on , data are recovered, and computation resumes. How much more time ? Estimate from the partially solved data.
Definition of entropy • Let X be the data set and X be decomposed like S(X)=(X1, …, Xk). Each Xi is solved. S(X) is a state of data, and abbreviated as S. Let pi=|Xi|/|X|, and |X|=n. The entropy H(S) is defined by • H(S) =[i=1, k]|Xi|log(|X|/|Xi|)=-n[i=1,k]pilog(pi) • pi = 1,0 H(S) nlog(k), maximum when all |Xi| are equal to 1/k.
Amortized analysis • The accounting equation at the i-th operation becomes • ai = ti - ΔH(Si), actual time – decrease of entropy where ΔH(Si) = H(Si-1) - H(Si). • Let T and A be the actual total time and the amortized total time. • Summing up ai for i=1, ..., N, we have • A = T + H(SN) - H(S0), • or T = A + H(S0) - H(SN). • For some applications, ai = ti - cΔH(Si) for some costant c
Problems analyzed by entropy • Minimal Mergesort : merging shortest ascending runs • Shortest Path Problem: Solid parts solved. Make a single shortest path spanning tree • Minimum Spanning Tree : solid part solved. Make a single tree souurce
Three examples with k=3 • (1) Ｍｉｎｉｍａｌ mergesort Time =O(H(S)) • Xi are ascending runs • S(X) = (2 5 6 1 4 7 3 8 9) • X1=(2 5 6), X2 =(1 4 7), X3=(3 8 9) • (2) Shortest paths for nearly acyclic graphs Time=O(m+H(S)) • G=(V, E) is a graph. S(V)=(V1, V2, V3) • Vi is an acyclic graph dominated by vi • (3) Minimum spanning tree Time = O(m+H(S)) • S(V)=(V1, V2, V3), subgraph Gi=(Vi, Ei) is the induced graph from Vi. We assume minimum spanning tree Ti for Gi is already obtained
Time complexities of the three problems • Minimal mergesort O(H(S)) • worst case O(nlog(n)) • Single source shortest paths O(m+H(S)) • worst case time O(m+nlog(n)) • data structures: Fibonacci heap or 2-3 heap • Minimum cost spanning trees O(m+H(S)) • Presort of edges (mlog(n)) excluded • worst case time O(m+nlog(n))
Minimal mergesort picture • Metasort S(X) S’(X) M L ．．. . . . W1 W2 merge W
Minimal MergesortM L : first list of L is moved to the last of M • Meta-sort S(X) into S’(X) by length of Xi • Let L = S’(X); /* L : list of lists */ • M=φ; M L; /* M : list of lists */ • for i=1 to k-1 do begin • W1 M; W2 M; • W=merge(W1, W2); • While L φ and |W|>first(L) do M L • M L • End
Lemma. L and M are meta-sorted. If W2 is a merged list, I.e., not original Xi, |W1| ≦|W2| ≦2|W1| • Lemma. Amortized time for i-th merge ai≦0 • Proof. Let |W1|=n1 and |W2|=n2 • ΔH = n1log(n/n1)+n2log(n/n2)-(n1+n2)log(n/(n1+n2) • n1log(1+n2/n1)+n2log(1+n1/n2) • n1log2+n2(log(3/2)=log(3/2)(n1+n2) • ai = ti – cΔH ≦n1+n2-1-ΔH /log(3/2) ≦ 0
Main results in minimal mergesort • Theorem. Minimal mergesort sorts sequence S(X) in O(H(S)) time • Theorem. Any sorting algorithm takes • (H(S)) time if |Xi| 2 for all i. • If |Xi|=1 for all I, S(X) is reverse-sorted, and it can be sorted in ascending order, in O(n) ttime.
Dijkstra’s algorithm for shortest paths with a priority queue • d[s]=0 • S={s}; F={w|(s,w) in out(s)} • while |S|<n do • delete v from F such that d[v] is minimum //delete-min • add v to S O(log n) • for w in out(v) do • if w is not in S then • if w is if F then d[w]=min{d[v], d[v]+c[v,w] //decrease-key O(1) • else {d[w]=d[v]+c[v,w]; add w to F} //insert O(1) • end do
Expansion of solution set • S: solution set of vertices to which shortest distances are known • F: frontier set of vertices connected from solution set by single edges Time = O(m + nlogn) S : solution set F : frontier w s w v
Efficient shortest path algorithm for nearly acyclic graphs • d[s]=0 • S={s}; Q={w|(s,w) in out(s)}; // F is organized as priority queue Q • while |S|<n do • if there is a vertex in F with no incoming edge from V-S • then choose v // easy vertex • else choose v from F such that d[v] is minimum //difficult • add v to S • Delete v from F • for w in out(v) do • if w is not in S then • if w is in F then d[w]=min{d[v], d[v]+c[v,w] //decrease-key • else {d[w]=d[v]+c[v,w]; add w to F} //insert O(1) • end do
Easy vertices and difficult vertices F S w s w difficult v easy
Entropy analysis of the shortest path algorithm • There are t difficult vertices u1, …, ut. • Each ui and the following easy vertices form an acyclic sub-graph. Let {v1,…, vk}, where v1=ui for some i, be one of the above acyclic sub-graphs. Let the number of descendants of vi be ni. • Delete v1, …, vk. Time = log(ni)+…+log(nk) ≦O(klog(n/k). • Let us index k by i for ui. Then the total time for deletes is O(k1log(n/k1)+…+ktlog(n/kt) = O(H(S)) where S(V)=(V1, …, Vt). • The rest of the time is O(m). Thus total time is O(m+O(H(S))
Nearly acyclic graph • Acyclic components are regarded assolved There are three acyclic components in this graph. Vertices in the component Vi can be deleted from the queue once the distance to the trigger vi is finalized vi Vi Acyclic graph topologically sorted 7 1 3 2 6
Algorithm 3 on the reduced graph vi Vi V1 After V1 is finalized, the reduced graph becomes acyclic
Minimum spanning trees • Blue fonts : vertices 1 8 7 5 9 3 8 2 2 T3 5 10 3 4 3 4 4 4 T1 2 2 11 6 1 9 6 7 5 T2 L=(1 2 2 2 3 3 4 4 4 5 5 6 7 8 9) L=(4 5 5 6 7 8 9) name =(1 1 1 1 2 2 2 3 3 3 3) (1 1 1 1 1 1 1 3 3 3 3)
Kruskal’s completion algorithm • 1 Let the sorted edge list L be partially scanned • 2 Minimum spanning trees for G1, ..., Gk have been obtained • 3 for i=1 to k do for v in Vkdo name[v]:=k • 4 while k > 1 do begin • 5Remove the first edge (u, v) from L • 6 if u and v belong to different sub-trees T1and T2 • 7 then begin • 8Connect T1 and T2 by (u, v) • 9 Change the names of the nodes in the smaller tree • to that of the larger tree; • 10k:=k - 1; • 11 end • 12 end.
Ｅｎｔｒｏｐｙ ａｎａｌｙｓｉｓ for name changes • the decrease of entropy is • ΔH = n1log(n/n1) + n2log(n/n2) • -(n1+n2)log(n/(n1+n2)) • = n1log(1+n2/n1) + n2log(1+n1/n2) • ≥ min{n1, n2} • Noting that ti <= min{n1, n2}, amortized time becomes • ai = ti - ΔH(Si) ≦ 0 • Scanning L takes O(m) time. Thus • T=(m+H(S0)), where H(S0) is the initial entropy.
