Numerical Simulation
Download
1 / 38

Solution of Sparse Linear Systems - PowerPoint PPT Presentation


  • 234 Views
  • Updated On :

Numerical Simulation CSE245 Spring 2010. Solution of Sparse Linear Systems. Thanks to: Kin Sou, Deepak Ramaswamy, Michal Rewienski, Jacob White, Shihhsien Kuo and Karen Veroy and especially Luca Daniel. Outline of today’s lecture. Solution of Sparse Linear Systems

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Solution of Sparse Linear Systems' - marinel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Numerical Simulation

CSE245 Spring 2010

Solution of Sparse Linear Systems

Thanks to: Kin Sou, Deepak Ramaswamy, Michal Rewienski, Jacob White, Shihhsien Kuo and Karen Veroy and

especially Luca Daniel


Slide2 l.jpg

Outline of today’s lecture

  • Solution of Sparse Linear Systems

    • Examples of Problems with Sparse Matrices

      • Struts and joints, resistor grids, 3-D heat flow

    • Tridiagonal Matrix Factorization

    • General Sparse Factorization

      • Fill-in and Reordering

      • Graph Based Approach

    • Sparse Matrix Data Structures

      • Scattering


Slide3 l.jpg

5

7

9

3

4

6

8

2

1

Applications

Sparse Matrices

Space Frame

Nodal Matrix

Space Frame

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Unknowns : Joint positions

Equations : forces = 0


Slide4 l.jpg

Applications

Sparse Matrices

Resistor Grid

Unknowns : Node Voltages

Equations : currents = 0


Slide5 l.jpg

Applications

Sparse Matrices

Nodal Formulation

Resistor Grid

Matrix non-zero locations for 100 x 10 Resistor Grid


Slide6 l.jpg

Applications

Sparse Matrices

Nodal Formulation

Temperature in a cube

Temperature known on surface, determine interior temperature

Circuit

Model


Slide7 l.jpg

Outline

  • Solution of Dense Linear Systems

  • Solution of Sparse Linear Systems

    • LU Factorization Reminder.

    • Example of Problems with Sparse Matrices

      • Struts and joints, resistor grids, 3-d heat flow

    • Tridiagonal Matrix Factorization

    • General Sparse Factorization

      • Fill-in and Reordering

      • Graph Based Approach

    • Sparse Matrix Data Structures

      • Scattering


Slide8 l.jpg

Tridiagonal Example

Sparse Matrices

Nodal Formulation

Matrix Form

How many operations to do LU factorization?

m


Slide9 l.jpg

Pivot

Multiplier

Tridiagonal Example

Sparse Matrices

GE Algorithm

For i = 1 to n-1 {“For each Row”

For j = i+1 to n {“For each target Row below the source”

For k = i+1 to n {“For each Row element beyond Pivot”

}

}

}

Order n Operations!


Slide10 l.jpg

Outline

  • Solution of Dense Linear Systems

  • Solution of Sparse Linear Systems

    • LU Factorization Reminder.

    • Example of Problems with Sparse Matrices

      • Struts and joints, resistor grids, 3-d heat flow

    • Tridiagonal Matrix Factorization

    • General Sparse Factorization

      • Fill-in and Reordering

      • Graph Based Approach

    • Sparse Matrix Data Structures

      • Scattering


Slide11 l.jpg

Sparse Matrix Fill-In

Sparse Matrices

Example

Resistor Example

Nodal Matrix

0

Symmetric

Diagonally Dominant


Slide12 l.jpg

X

X

X = fill in where 0 became non-zero

Fill-In

Sparse Matrices

Example

Matrix Non zero structure

Fill Ins

X

X

X

X

X= Non zero


Slide13 l.jpg

X

X

X

X

X

Fill-In

Sparse Matrices

Second Example

Fill-ins Propagate

X

X

X

X

X

Fill-ins from Step 1 result in Fill-ins in step 2


Slide14 l.jpg

0

Fill-In

Sparse Matrices

Reordering

Fill-ins

0

No Fill-ins

Node Reordering

- Can reduce fill-in

- Preserves properties (Symmetry, Diagonal Dominance)

- Equivalent to swapping rows and columns


Slide15 l.jpg

Possible Fill-in

Locations

Fill-in Estimate = (Non zeros in unfactored part of Row -i)

(Non zeros in unfactored part of Col -i)

Markowitz product

Fill-In

Sparse Matrices

Reordering

Where can fill-in occur ?

Already Factored

Multipliers


Slide16 l.jpg

Fill-In

Sparse Matrices

Reordering

Markowitz Reordering

Greedy Algorithm !


Slide17 l.jpg

1

2

3

1

0

0

3

2

Fill-In

Sparse Matrices

Reordering

Why only try diagonals ?

  • Corresponds to node reordering in Nodal formulation

  • Reduces search cost

  • Preserves Matrix Properties - Diagonal Dominance - Symmetry


Slide18 l.jpg

Fill-In

Sparse Matrices

Pattern of a Filled-in Matrix

Very Sparse

Very Sparse

Dense


Slide19 l.jpg

Fill-In

Sparse Matrices

Unfactored Random Matrix

Symmetric


Slide20 l.jpg

Fill-In

Sparse Matrices

Factored Random Matrix

Symmetric


Slide21 l.jpg

Outline

  • Solution of Dense Linear Systems

  • Solution of Sparse Linear Systems

    • LU Factorization Reminder.

    • Example of Problems with Sparse Matrices

      • Struts and joints, resistor grids, 3-d heat flow

    • Tridiagonal Matrix Factorization

    • General Sparse Factorization

      • Fill-in and Reordering

      • Graph Based Approach

    • Sparse Matrix Data Structures

      • Scattering


Slide22 l.jpg

Matrix Graphs

Sparse Matrices

Construction

Structurally Symmetric Matrices and Graphs

1

X

X

X

X

X

X

X

2

X

X

X

X

3

X

X

X

4

X

X

X

5

  • One Node Per Matrix Row

  • One Edge Per Off-diagonal Pair


Slide23 l.jpg

1

2

X

X

X

X

3

X

X

X

4

X

X

X

X

5

X

X

X

X

X

X

Matrix Graphs

Sparse Matrices

Markowitz Products

Markowitz Products =

(Node Degree)2


Slide24 l.jpg

Matrix Graphs

Sparse Matrices

Factorization

One Step of LU Factorization

1

X

X

X

X

X

X

X

X

2

X

X

X

X

X

3

X

X

X

4

X

X

X

X

X

5

  • Delete the node associated with pivot row

  • “Tie together” the graph edges


Slide25 l.jpg

1

2

3

4

5

1

2

3

4

5

Matrix Graphs

Sparse Matrices

Example

Graph

Markowitz products

( = Node degree)


Slide26 l.jpg

5

5

4

3

2

3

4

Graph

Matrix Graphs

Sparse Matrices

Example

Swap 2 with 1

X

1

2

2

1

X

Eliminate 1


Slide27 l.jpg

Graphs

Sparse Matrices

Resistor Grid Example

Unknowns : Node Voltages

Equations : currents = 0


Slide28 l.jpg

Matrix Graphs

Sparse Matrices

Grid Example


Slide29 l.jpg

What should you pivot for?

Sparse Matrices

For growth control?

Or to avoid fill-ins?

A) LU factorization applied to a strictly diagonally dominant matrix will never produce a zero pivot

B) The matrix entries produced by LU factorization applied to a strictly diagonally dominant matrix will never increase by more than a factor 2(n-1)

[which is anyway the best you can do by pivoting for growth control]

Bottom line:

if your matrix is strictly diagonally dominant

no need for numerical pivot for growth control!


Slide30 l.jpg

Sparse Factorization Approach

  • Assume matrix requires NO numerical pivoting.

    Diagonally dominant or symmetric positive definite.

  • Pre-Process:

    • Use Graphs to Determine Matrix Ordering

      • Many graph manipulation tricks used.

    • Form Data Structure for Storing Filled-in Matrix

      • Lots of additional non-zero added

  • Put numerical values in Data Structure and factor

    Computation must be organized carefully!


Slide31 l.jpg

Summary of Sparse Systems

  • LU Factorization and Diagonal Dominance.

    • Factor without numerical pivoting

  • Sparse Matrices

    • Struts, resistor grids, 3-d heat flow -> O(N) non-zeros

  • Tridiagonal Matrix Factorization

    • Factor in O(N) operations

  • General Sparse Factorization

    • Markowitz Reordering to minimize fill

  • Graph Based Approach

    • Factorization and Fill-in

    • Useful for estimating Sparse GE complexity


Slide32 l.jpg

Outline

  • Solution of Sparse Linear Systems

    • Example of Problems with Sparse Matrices

    • Tridiagonal Matrix Factorization

    • General Sparse Factorization

      • Fill-in and Reordering

      • Graph Based Approach

      • Summary of the Algorithm

    • Sparse Matrix Data Structures

      • Scattering and symbolic fill in


Slide33 l.jpg

Sparse Data Structure

Sparse Matrices

Vector of row

pointers

Arrays of Data in a Row

Matrix entries

Val 11

Val 12

Val 1K

1

Column index

Col 11

Col 12

Col 1K

Val 21

Val 22

Val 2L

Col 21

Col 22

Col 2L

Row pointers

Column Indices

Goal:

Never store a 0

Never multiply by 0

Val N1

Val N2

Val Nj

N

Col N1

Col N2

Col Nj


Slide34 l.jpg

Sparse Data Structure

Sparse Matrices

Problem of Misses

Eliminating Source Row i from Target row j

Row i

Row j

Must read all the row j entries to find the 3 that match row i


Slide35 l.jpg

Sparse Data Structure

Sparse Matrices

Data on Misses

More misses

than

ops!

Every Miss is an unneeded Memory Reference!


Slide36 l.jpg

Sparse Data Structure

Sparse Matrices

Scattering for Miss Avoidance

Row j

Use target row approach – Row j is the target.

1) Read all the elements in Row j, and scatter them in an n-length vector

2) Access only the needed elements using array indexing!


Slide37 l.jpg

Sparse Matrices – Another Data Structure

Orthogonal linked list

But if fill in occurs, pointer structures change. What do we do?

Pre-compute pivoting order and sparse fill in structure symbolically


Slide38 l.jpg

Summary of Sparse Systems

  • LU Factorization and Diagonal Dominance.

    • Factor without numerical pivoting

    • hard problems (ill-conditioned)

  • Sparse Matrices

    • Struts, resistor grids, 3-d heat flow -> O(N) nonzeros

  • Tridiagonal Matrix Factorization

    • Factor in O(N) operations

  • General Sparse Factorization

    • Markowitz Reordering to minimize fill

  • Graph Based Approach

    • Factorization and Fill-in

    • Useful for estimating Sparse GE complexity

  • Sparse Data Structures

    • Scattering and symbolic fill in


ad