1 / 28

# - PowerPoint PPT Presentation

Support-Graph Preconditioning. John R. Gilbert MIT and U. C. Santa Barbara coauthors: Marshall Bern, Bruce Hendrickson, Nhat Nguyen, Sivan Toledo authors of other work surveyed: Erik Boman, Doron Chen, Keith Gremban, Bruce Hendrickson, Gary Miller, Sivan Toledo, Pravin Vaidya, Marco Zagha.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about '' - enye

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Support-Graph Preconditioning

John R. Gilbert

MIT and U. C. Santa Barbara

coauthors:

Marshall Bern, Bruce Hendrickson, Nhat Nguyen, Sivan Toledo

authors of other work surveyed:

Erik Boman, Doron Chen, Keith Gremban, Bruce Hendrickson, Gary Miller, Sivan Toledo, Pravin Vaidya, Marco Zagha

• A is large & sparse

• say n = 105 to 108, # nonzeros = O(n)

• Physical setting sometimes implies G(A) has separators

• O(n1/2) in 2D, O(n2/3) in 3D

• Here: A is symmetric and positive (semi)definite

• Direct methods: A=LU

• Iterative methods:y(k+1) = Ay(k)

7

1

3

7

1

6

8

6

8

4

10

4

10

9

2

9

2

5

5

Graphs and Sparse Matrices[Parter, … ]

Fill:new nonzeros in factor

Symmetric Gaussian elimination:

for j = 1 to n add edges between j’s higher-numbered neighbors

G+(A)[chordal]

G(A)

• Theory: approx optimal separators => approx optimal fill and flop count

• Orderings: nested dissection, minimum degree, hybrids

• Graph partitioning: spectral, geometric, multilevel

x0 = 0, r0 = b, p0 = r0

for k = 1, 2, 3, . . .

αk = (rTk-1rk-1) / (pTk-1Apk-1) step length

xk = xk-1 + αk pk-1 approx solution

rk = rk-1 – αk Apk-1 residual

βk = (rTk rk) / (rTk-1rk-1) improvement

pk = rk + βk pk-1 search direction

• One matrix-vector multiplication per iteration

• Two vector dot products per iteration

• Four n-vectors of working storage

• In exact arithmetic, CG converges in n steps (completely unrealistic!!)

• Accuracy after k steps of CG is related to:

• consider polynomials of degree k that are equal to 1 at 0.

• how small can such a polynomial be at all the eigenvalues of A?

• Thus, eigenvalues close together are good.

• Condition number:κ(A) = ||A||2 ||A-1||2 = λmax(A) / λmin(A)

• Residual is reduced by a constant factor by O(κ1/2(A)) iterations of CG.

• Suppose you had a matrix B such that:

(1) condition number κ(B-1A) is small

(2) By = z is easy to solve

• Then you could solve (B-1A)x = B-1b instead of Ax = b(actually (B-1/2AB-1/2) B1/2 x = B-1/2 b, but never mind)

• B = A is great for (1), not for (2)

• B = I is great for (2), not for (1)

+: New analytic tools, some new preconditioners

+: Can use existing direct-methods software

-: Current theory and techniques limited

• Define a preconditioner B for matrix A

• Explicitly compute the factorization B = LU

• Choose nonzero structure of B to make factoring cheap (using combinatorial tools from direct methods)

• Prove bounds on condition number using both algebraic and combinatorial tools

Spanning Tree Preconditioner [Vaidya]

• A is symmetric positive definite with negative off-diagonal nzs

• B is a maximum-weight spanning tree for A (with diagonal modified to preserve row sums)

• factor B in O(n) space and O(n) time

• applying the preconditioner costs O(n) time per iteration

G(A)

G(B)

Spanning Tree Preconditioner [Vaidya]

• support each edge of A by a path in B

• dilation(A edge) = length of supporting path in B

• congestion(B edge) = # of supported A edges

• p = max congestion, q = max dilation

• condition number κ(B-1A) bounded by p·q (at most O(n2))

G(A)

G(B)

Spanning Tree Preconditioner [Vaidya]

• can improve congestion and dilation by adding a few strategically chosen edges to B

• cost of factor+solve is O(n1.75), or O(n1.2) if A is planar

• in experiments by Chen & Toledo, often better than drop-tolerance MIC for 2D problems, but not for 3D.

G(A)

G(B)

Support Graphs [after Gremban/Miller/Zagha]

Intuition from resistive networks:How much must you amplify B to provide the conductivity of A?

• The support of B for A is σ(A, B) = min{τ : xT(tB– A)x  0 for all x, all t  τ}

• In the SPD case, σ(A, B) = max{λ : Ax = λBx} = λmax(A, B)

• Theorem:If A, B are SPD then κ(B-1A) = σ(A, B) · σ(B, A)

• Split A = A1+ A2 + ··· + Ak and B = B1+ B2 + ··· + Bk

• Ai and Bi are positive semidefinite

• Typically they correspond to pieces of the graphs of A and B (edge, path, small subgraph)

• Lemma: σ(A, B)  maxi {σ(Ai , Bi)}

• Lemma: σ(edge, path)  (worst weight ratio) · (path length)

• In the MST case:

• Ai is an edge and Bi is a path, to give σ(A, B)  p·q

• Bi is an edge and Ai is the same edge, to give σ(B, A)  1

B has same row sums as A

Strategy: Use the negative edges of B to support both the negative edges of A and the positive edges of B.

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

-1

B

A

.5

.5

.5

.5

.5

.5

.5

.5

.5

A = 2D model Poisson problem

B = MIC preconditioner for A

Support-graph analysis of modified incomplete Cholesky

Each solid edge of B supports one or two dotted edges

Tune fractions to support each dotted edge exactly

1/(2n – 2) of each solid edge is left over to support an edge of A

Supporting positive edges of B

• Each edge of A is supported by the leftover 1/(2n – 2) fraction of the same edge of B.

• Therefore σ(A, B)  2n – 2

• Easy to show σ(B, A)  1

• For this 2D model problem, condition number is O(n1/2)

• Similar argument in 3D gives condition number O(n1/3) or O(n2/3) (depending on boundary conditions)

Support-graph analysis for in Bbetter preconditioners?

• For model problems, the preconditioners analyzed so far are not as efficient as multigrid, domain decomposition, etc.

• Gremban/Miller: a hierarchical support-graph preconditioner, but condition number still not polylogarithmic.

• We analyze a multilevel-diagonal-scaling-like preconditioner in 1D . . .

• . . . but we haven’t proved tight bounds in higher dimensions.

- in B1

-1

-2

B

A

-1

-1

-1

-1

-1

-1

Hierarchical preconditioner (1D model problem)

• Good support in both directions: σ(A, B) = σ(B, A) = O(1)

• But, B is a mesh => expensive to factor and to apply

.5 in B

.5

-1

-1

-2

B

A

-1

-1

-1

-1

-1

-1

Hierarchical preconditioner continued

• Drop fill in factor of B (i.e. add positive edges) => factoring and preconditioning are cheap

• But B cannot support both A and its own positive edges;σ(A, B) is infinite

-.5 in B

-.5

-1

.5

.5

-1

-1

-2

B

A

-1

-1

-1

-1

-1

-1

Hierarchical preconditioner continued

• Solution: add a coarse mesh to support positive edges

• Now σ(A, B)  log2(n+1)

• Elimination/splitting analysis gives σ(B, A) = 1

• Therefore condition number = O(log n)

• Idea: mimic the 1D construction

• Generate a coarsening hierarchy with overlapping subdomains

• Use σ(B, A) = 1 as a constraint to choose weights

• Defines a preconditioner for regular or irregular problems

• Sublinear bounds on σ(A, B) for some 2D model problems.

• (Very preliminary) experiments show slow growth in κ(B-1A)

• The support of B for A is σ(A, B) = min{τ : xT(tB– A)x  0 for all x, all t  τ}

• In the SPD case, σ(A, B) = max{λ : Ax = λBx} = λmax(A, B)

• If A, B are SPD then κ(B-1A) = σ(A, B) · σ(B, A)

• [Boman/Hendrickson] If V·W=U, then σ(U·UT, V·VT)  ||W||22

Algebraic framework in B[Boman/Hendrickson]

Lemma: If V·W=U, then σ(U·UT, V·VT)  ||W||22

Proof:

• take t  ||W||22 = λmax(W·WT) = max x0 { xTW·WTx / xTx }

• then xT (tI - W·WT) x  0 for all x

• letting x = VTy gives yT (tV·VT - U·UT) y  0 for all y

• recall σ(A, B) = min{τ : xT(tB– A)x  0 for all x, all t  τ}

• thus σ(U·UT, V·VT)  t

[ in B

]

a2 +b2-a2 -b2 -a2 a2 +c2 -c2-b2 -c2 b2 +c2

[

]

a2 +b2-a2 -b2 -a2 a2 -b2 b2

B

=VVT

A

=UUT

]

[

]

]

[

[

1-c/a1 c/b/b

ab-a c-b

ab-a c-b-c

=

x

U

V

W

-a2

-c2

-a2

-b2

-b2

σ(A, B)  ||W||22 ||W||x ||W||1 = (max row sum) x (max col sum) (max congestion) x (max dilation)

Open problems I in B

• Other subgraph constructions for better bounds on||W||22?

• For example [Boman],

||W||22 ||W||F2= sum(wij2) = sum of (weighted) dilations,

and [Alon, Karp, Peleg, West]show there exists a spanning tree with average weighted dilation exp(O((log n loglog n)1/2)) = o(n );

this gives condition number O(n1+) and solution time O(n1.5+),

compared to Vaidya O(n1.75) with augmented spanning tree

• Is there a construction that minimizes ||W||22directly?

Open problems II in B

• Make spanning tree methods more effective in 3D?

• Vaidya gives O(n1.75) in general, O(n1.2) in 2D

• Issue: 2D uses bounded excluded minors, not just separators

• Spanning tree methods for more general matrices?

• All SPD matrices? ([Boman, Chen, Hendrickson, Toledo]: different matroid for all diagonally dominant SPD matrices)

• Finite element problems? ([Boman]: Element-by-element preconditioner for bilinear quadrilateral elements)

• Analyze a multilevel method in general?

n in B1/2

n1/3

Complexity of linear solvers

Time to solve model problem (Poisson’s equation) on regular mesh

References in B

• M. Bern, J. Gilbert, B. Hendrickson, N. Nguyen, S. Toledo. Support-graph preconditioners. (Submitted for publication, 2001.) ftp://parcftp.xerox.com/gilbert/support-graph.ps

• K. Gremban, G. Miller, M. Zagha. Performance evaluation of a parallel preconditioner. IPPS 1995.

• D. Chen, S. Toledo. Implementation and evaluation of Vaidya’s preconditioners. Preconditioning 2001. (Submitted for publication, 2001.)

• E. Boman, B. Hendrickson. Support theory for preconditioning. (Submitted for publication, 2001.)

• E. Boman, D. Chen, B. Hendrickson, S. Toledo. Maximum-weight-basis preconditioners. (Submitted for publication, 2001.)

• Bruce Hendrickson’s support theory web page: http://www.cs.sandia.gov/~bahendr/support.html