Solving specific classes of linear equations using random walks
Download
1 / 36

Solving [Specific Classes of] Linear Equations using Random Walks - PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on

Solving [Specific Classes of] Linear Equations using Random Walks. Haifeng Qian Sachin Sapatnekar. x 1 x 2 : x n. b 1 b 2 : b n. a ii. Row i. =. Definition of diagonal dominance. Matrix A is diagonally dominant if a ii   ij |a ij | Consider a system of linear equations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Solving [Specific Classes of] Linear Equations using Random Walks' - janae


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Definition of diagonal dominance

x Walks1

x2

:

xn

b1

b2

:

bn

aii

Row i

=

Definition of diagonal dominance

  • Matrix A is diagonally dominant if

    aiiij|aij|

  • Consider a system of linear equations

  • Can solve Ax = b rapidly using a random walk analogy if A is diagonally dominant (but not singular, of course!)

(abbreviated as Ax = b)


Why do i care
Why do I care? Walks

  • Diagonally dominant systems arise in several contexts in CAD (and in other fields) – for example,

    • Power grid analysis

    • VLSI Placement

    • ESD analysis

    • Thermal analysis

    • FEM/FDM analyses

    • Potential theory

  • The idea for random walk-based linear solvers has been around even in the popular literature

    • R. Hersh and R. J. Griego, “Brownian motion and potential theory,” Scientific American, pp. 67-74, March 1969.

    • P. G. Doyle and J. L. Snell, Random walks and electrical networks, Mathematical Association of America, Washington DC, 1984.

      http://math.dartmouth.edu/~doyle/docs/walks/walks.pdf


Dirichlet s problem
Dirichlet’s problem Walks

  • Dirichlet problem: an example – thermal analysis

    • Given a body of arbitrary shape, and complete information about temperature on the surface: find termperature at an internal point

    • Temperature is a harmonic function: temperature at a point depends on average temperature of surrounding points

  • Shizuo Kakutani (1944) – solution of Dirichlet problem

    • Brownian motion starting from a point (say, a)

      • Take an award T = temperature of first boundary point hit

      • Find E[T]

a

b



Stochastic solvers

Equations Walks

Random event(s)

Variables

Expectation of random variables

Approximate solution

Average of random samples

Stochastic solvers

Stochastic solver methodology


Example power grid analysis

2 Walks

g2

g1

g3

x

3

1

g4

Ix

4

Example: Power grid analysis

  • Network of resistors and current sources

  • The equation at node x is

Alternatively


Mapping this to the random walk game

∑ 1 Walks

Mapping this to the random walk “game”

  • Solving a grid amounts to solving, at each node:or


$ Walks

Home

Home

Home

Random walk overview

  • Given:

    • A network of roads

    • A motel at eachintersection

    • A set of homes

  • Random walk

    • Walk one (randomlychosen) road every day

    • Stay the night at a motel

      (pay for it!)

    • Keep going until reaching home

    • Win a reward for reaching home!

  • Problem: find the expected amount of money in the end as a function of the starting node x


4 Walks

px,4

x

1

3

px,1

px,3

px,2

2

Random walk overview

  • For every node


Random walk overview

Linear equation set Walks

Random walk game

M walks from i-th node

Take average

This yields xi

Random walk overview


$ Walks0

$1

0.33

0.33

+$0.13

-$0.2

-$ 0.05

+$ 0.45

A

A

0.44

0.44

0.67

0.67

$1

$0

$0

$1

B

B

D

D

0.2

0.2

0.44

0.44

0.5

0.5

0.8

0.8

-$0.04

+$0.76

0.5

0.5

0.12

0.12

-$0.022

-$0.022

C

C

Example

  • Solution: xA=0.6 xB=0.8

    xC=0.7 xD=0.9


Average Walks

$0

-$0.284

$0

-$0.222

$0

-$0.222

-$0.422

-$0.2

$0.8

$0.578

$0.382

$0.444

$0.738

-$0.618

-$0.556

-$0.262

$0

$0

$0

-$0.516

-$0.222

-$0.222

-$0.578

-$0.262

-$0.2

-$0.2

-$0.494

-$0.422

-$0.2

-$0.556

-$0.2

-$0.484

-$0.2

-$0.506

-$0.222

-$0.444

$0.728

-$0.272

0.6

0.33

A

0.44

0.67

B

D

0.2

0.44

0.5

0.8

0.5

0.12

C

Play the game

$1

Pocket:

-$0.2

$1

$1

-$0.05

-$0.04

-$0.022

  • Walk results:

$0.578

$0.444

$0.8

$0.728

……

$0.738

$0.382


Reusing computations avoiding repeated walks when solving for all x i values

Start Walks

Start

Reusing computations: avoiding repeated walks(When solving for all xi values)

New home

Previously calculated node

Benefit: more and more homes

shorter and shorter walks

one walk = average of multiple walks

Qian et al., DAC2003


Reusing computations journey record w hen solving for multiple right hand sides
Reusing computations: “Journey record” Walks(when solving for multiple right hand sides)

Keep a record: motel/award list

New RHS: Ax = b2

Update motel prices, award values

Use the record: pay motels, receive awards

New solution

Benefit: no more walks

only feasible after trick#1

Qian et al., DAC2003


Error vs runtime tradeoffs
Error vs. runtime tradeoffs Walks

  • Industrial circuit

    • 70729 nodes, 31501 bottom-layer nodes

    • VDD net true voltage range 1.1324—1.1917


Random walk overview1
Random walk overview Walks

  • Advantage

    • Locality: solve single entry

  • Weakness

    • Error ~ M-0.5

    • 3% error to be faster than direct/iterative solvers


Example application power grid analysis
Example application: Power grid analysis Walks

  • Exact DC analysis: Solve GX = E

  • Very expensive to solve for millions of nodes, eventually prohibitive

  • Simple observations:

    • VDD and GND pins all over chip surface (C4 connections)

    • Most current drawn from “nearby” connections



Current techniques
Current techniques Walks

  • Preconditioning

  • Popular choice: Incomplete LU

  • Placement/thermal matrices

    • Symmetric positive definite

    • Popular choice: ICCG with

      • Different ordering

      • Different dropping rules


Current techniques1
Current techniques Walks

done

done

  • Rules

    • Pattern

    • Min value

    • Size limit


Stochastic preconditioning

stochastic precondi-tioning Walks

adopt an iterative scheme

Iterative

Solver

The Hybrid

Solver

Stochastic

Solver

Stochastic preconditioning

  • Special case today

    • Symmetric

    • Positive diagonal entries

    • Negative off-diagonal entries

    • Irreducibly diagonally dominant

    • These are sufficient, NOT necessary, conditions


Sequential monte carlo

Stochastic Walks

Solver

New solution:

Sequential Monte Carlo

Stochastic

Solver

approx. solve

error & residual

approx. solve

Benefit: ||r||2<<||b||2 ||y||2<<||x||2

same relative error = lower absolute error

A. W. Marshall 1956, J. H. Halton 1962


This is computationally easy

Keep a record: motel/award list Walks

Stochastic

Solver

New RHS: Ax = b2

Update motel prices, award values

Stochastic

Solver

Use the record: pay motels, receive awards

New solution

New solution:

This is computationally easy!

Can show that this amounts to preconditioned Gauss-Jacobi

So - why G-J? Why not CG/BiCG/MINRES/GMRES


What s on the journey record

Solver using Walks

the record

What’s on the journey record?

Row i

These are “UL” factors.

Relation to LU factors? Just a matrix reordering!


Ldl factorization

Recall Walks

LDL factorization

  • What we need for symmetric A

  • What we have

  • How to find ?

    • Easy – details omitted here

  • Easy extension to asymmetric matrices exists


Compare to existing ilu
Compare to existing ILU Walks

  • Existing ILU

    • Gaussian elimination

    • Drop edges by pattern, value, size

  • Error propagation

    • A missing edge affects subsequent computation

    • Exacerbated for larger and denser matrices

b2

b2

b1

b1

a

b3

b3

b5

b5

b4

b4


Superior because
Superior because … Walks

  • Each row of L is independently calculated

    • No knowledge of other rows

    • Responsible for its own accuracy

    • No debt from other steps


Test setup
Test setup Walks

  • Quadratic placement instances

    • Set #1: matrices and rhs’s by an industrial placer

    • Set #2: matrices by UWaterloo placer on ISPD02 benchmarks, unit rhs’s

  • LASPACK: ICCG with ILU(0)

  • MATLAB: ICCG with ILUT

    • Approx. Min. Degree ordering

    • Tuned to similar factorization size

  • Same accuracy:

  • Complexity metric: # double-precision multiplications

    • Solving stage only


Comparison
Comparison Walks


Physical runtimes on p4 2 8ghz
Physical runtimes on P4-2.8GHz Walks

  • Reasonable preconditioning overhead

    • Less than 3X solving time

    • One-time cost, amortized over multiple solves



Newer results
Newer results Walks

  • Examples generated from Sparskit

    • Finite difference discretization of 3D Laplace’s equation under Dirichlet boundary conditions

    • S: #nonzeros in preconditioner (similar in all cases)

    • C1: Condition # of original matrix

    • C2: Condition # after split preconditioning

Ex1

Ex2

Ex3

Ex4

Ex5

Ex6


Conclusion
Conclusion Walks

  • Direct solver

    • Locality property

    • Reasonable for approximate solutions

  • Hybrid solver

    • Stochastically preconditioned iterative solver

    • Independent row/column estimates for LU factors

    • Better quality than same-size traditional ILU

  • Extension to non-diagonally dominant matrices?

    • Some pointers exist (see Haifeng Qian’s thesis)

    • Needs further work…


Thank you! Walks

  • Downloads

    • Solver package

      http://www.ece.umn.edu/users/qianhf/hybridsolver

    • Thesis

      http://www-mount.ece.umn.edu/~sachin/Theses/HaifengQian.pdf


Thanks also and especially to
Thanks also (and especially) to… Walks

Haifeng Qian

(He will pick up the 2006 ACM Oustanding Dissertation Award in Electronic Design Automation in San Diego next week)


ad