- 59 Views
- Uploaded on
- Presentation posted in: General

Solving [Specific Classes of] Linear Equations using Random Walks

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Solving [Specific Classes of] Linear Equations using Random Walks

Haifeng Qian

Sachin Sapatnekar

x1

x2

:

xn

b1

b2

:

bn

aii

Row i

=

- Matrix A is diagonally dominant if
aiiij|aij|

- Consider a system of linear equations
- Can solve Ax = b rapidly using a random walk analogy if A is diagonally dominant (but not singular, of course!)

(abbreviated as Ax = b)

- Diagonally dominant systems arise in several contexts in CAD (and in other fields) – for example,
- Power grid analysis
- VLSI Placement
- ESD analysis
- Thermal analysis
- FEM/FDM analyses
- Potential theory

- The idea for random walk-based linear solvers has been around even in the popular literature
- R. Hersh and R. J. Griego, “Brownian motion and potential theory,” Scientific American, pp. 67-74, March 1969.
- P. G. Doyle and J. L. Snell, Random walks and electrical networks, Mathematical Association of America, Washington DC, 1984.
http://math.dartmouth.edu/~doyle/docs/walks/walks.pdf

- Dirichlet problem: an example – thermal analysis
- Given a body of arbitrary shape, and complete information about temperature on the surface: find termperature at an internal point
- Temperature is a harmonic function: temperature at a point depends on average temperature of surrounding points

- Shizuo Kakutani (1944) – solution of Dirichlet problem
- Brownian motion starting from a point (say, a)
- Take an award T = temperature of first boundary point hit
- Find E[T]

- Brownian motion starting from a point (say, a)

a

b

A Direct Solver

Equations

Random event(s)

Variables

Expectation of random variables

Approximate solution

Average of random samples

Stochastic solver methodology

2

g2

g1

g3

x

3

1

g4

Ix

4

- Network of resistors and current sources
- The equation at node x is

Alternatively

∑ 1

- Solving a grid amounts to solving, at each node:or

$

Home

Home

Home

Random walk overview

- Given:
- A network of roads
- A motel at eachintersection
- A set of homes

- Random walk
- Walk one (randomlychosen) road every day
- Stay the night at a motel
(pay for it!)

- Keep going until reaching home
- Win a reward for reaching home!

- Problem: find the expected amount of money in the end as a function of the starting node x

4

px,4

x

1

3

px,1

px,3

px,2

2

Random walk overview

- For every node

Linear equation set

Random walk game

M walks from i-th node

Take average

This yields xi

$0

$1

0.33

0.33

+$0.13

-$0.2

-$ 0.05

+$ 0.45

A

A

0.44

0.44

0.67

0.67

$1

$0

$0

$1

B

B

D

D

0.2

0.2

0.44

0.44

0.5

0.5

0.8

0.8

-$0.04

+$0.76

0.5

0.5

0.12

0.12

-$0.022

-$0.022

C

C

Example

- Solution: xA=0.6 xB=0.8
xC=0.7 xD=0.9

Average

$0

-$0.284

$0

-$0.222

$0

-$0.222

-$0.422

-$0.2

$0.8

$0.578

$0.382

$0.444

$0.738

-$0.618

-$0.556

-$0.262

$0

$0

$0

-$0.516

-$0.222

-$0.222

-$0.578

-$0.262

-$0.2

-$0.2

-$0.494

-$0.422

-$0.2

-$0.556

-$0.2

-$0.484

-$0.2

-$0.506

-$0.222

-$0.444

$0.728

-$0.272

0.6

0.33

A

0.44

0.67

B

D

0.2

0.44

0.5

0.8

0.5

0.12

C

Play the game

$1

Pocket:

-$0.2

$1

$1

-$0.05

-$0.04

-$0.022

- Walk results:

$0.578

$0.444

$0.8

$0.728

……

$0.738

$0.382

Start

Start

New home

Previously calculated node

Benefit: more and more homes

shorter and shorter walks

one walk = average of multiple walks

Qian et al., DAC2003

Keep a record: motel/award list

New RHS: Ax = b2

Update motel prices, award values

Use the record: pay motels, receive awards

New solution

Benefit: no more walks

only feasible after trick#1

Qian et al., DAC2003

- Industrial circuit
- 70729 nodes, 31501 bottom-layer nodes
- VDD net true voltage range 1.1324—1.1917

- Advantage
- Locality: solve single entry

- Weakness
- Error ~ M-0.5
- 3% error to be faster than direct/iterative solvers

- Exact DC analysis: Solve GX = E
- Very expensive to solve for millions of nodes, eventually prohibitive
- Simple observations:
- VDD and GND pins all over chip surface (C4 connections)
- Most current drawn from “nearby” connections

A preconditioned iterative solver

- Preconditioning
- Popular choice: Incomplete LU
- Placement/thermal matrices
- Symmetric positive definite
- Popular choice: ICCG with
- Different ordering
- Different dropping rules

done

done

- Rules
- Pattern
- Min value
- Size limit

stochastic precondi-tioning

adopt an iterative scheme

Iterative

Solver

The Hybrid

Solver

Stochastic

Solver

- Special case today
- Symmetric
- Positive diagonal entries
- Negative off-diagonal entries
- Irreducibly diagonally dominant
- These are sufficient, NOT necessary, conditions

Stochastic

Solver

New solution:

Stochastic

Solver

approx. solve

error & residual

approx. solve

Benefit: ||r||2<<||b||2 ||y||2<<||x||2

same relative error = lower absolute error

A. W. Marshall 1956, J. H. Halton 1962

Keep a record: motel/award list

Stochastic

Solver

New RHS: Ax = b2

Update motel prices, award values

Stochastic

Solver

Use the record: pay motels, receive awards

New solution

New solution:

Can show that this amounts to preconditioned Gauss-Jacobi

So - why G-J? Why not CG/BiCG/MINRES/GMRES

Solver using

the record

Row i

These are “UL” factors.

Relation to LU factors? Just a matrix reordering!

Recall

- What we need for symmetric A
- What we have
- How to find ?
- Easy – details omitted here

- Easy extension to asymmetric matrices exists

- Existing ILU
- Gaussian elimination
- Drop edges by pattern, value, size

- Error propagation
- A missing edge affects subsequent computation
- Exacerbated for larger and denser matrices

b2

b2

b1

b1

a

b3

b3

b5

b5

b4

b4

- Each row of L is independently calculated
- No knowledge of other rows
- Responsible for its own accuracy
- No debt from other steps

- Quadratic placement instances
- Set #1: matrices and rhs’s by an industrial placer
- Set #2: matrices by UWaterloo placer on ISPD02 benchmarks, unit rhs’s

- LASPACK: ICCG with ILU(0)
- MATLAB: ICCG with ILUT
- Approx. Min. Degree ordering
- Tuned to similar factorization size

- Same accuracy:
- Complexity metric: # double-precision multiplications
- Solving stage only

- Reasonable preconditioning overhead
- Less than 3X solving time
- One-time cost, amortized over multiple solves

- Examples generated from Sparskit
- Finite difference discretization of 3D Laplace’s equation under Dirichlet boundary conditions
- S: #nonzeros in preconditioner (similar in all cases)
- C1: Condition # of original matrix
- C2: Condition # after split preconditioning

Ex1

Ex2

Ex3

Ex4

Ex5

Ex6

- Direct solver
- Locality property
- Reasonable for approximate solutions

- Hybrid solver
- Stochastically preconditioned iterative solver
- Independent row/column estimates for LU factors
- Better quality than same-size traditional ILU

- Extension to non-diagonally dominant matrices?
- Some pointers exist (see Haifeng Qian’s thesis)
- Needs further work…

Thank you!

- Downloads
- Solver package
http://www.ece.umn.edu/users/qianhf/hybridsolver

- Thesis
http://www-mount.ece.umn.edu/~sachin/Theses/HaifengQian.pdf

- Solver package

Haifeng Qian

(He will pick up the 2006 ACM Oustanding Dissertation Award in Electronic Design Automation in San Diego next week)