slide1
Download
Skip this Video
Download Presentation
Solution of Sparse Linear Systems

Loading in 2 Seconds...

play fullscreen
1 / 38

Solution of Sparse Linear Systems - PowerPoint PPT Presentation


  • 235 Views
  • Uploaded on

Numerical Simulation CSE245 Spring 2010. Solution of Sparse Linear Systems. Thanks to: Kin Sou, Deepak Ramaswamy, Michal Rewienski, Jacob White, Shihhsien Kuo and Karen Veroy and especially Luca Daniel. Outline of today’s lecture. Solution of Sparse Linear Systems

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Solution of Sparse Linear Systems' - marinel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1
Numerical Simulation

CSE245 Spring 2010

Solution of Sparse Linear Systems

Thanks to: Kin Sou, Deepak Ramaswamy, Michal Rewienski, Jacob White, Shihhsien Kuo and Karen Veroy and

especially Luca Daniel

slide2
Outline of today’s lecture
  • Solution of Sparse Linear Systems
    • Examples of Problems with Sparse Matrices
      • Struts and joints, resistor grids, 3-D heat flow
    • Tridiagonal Matrix Factorization
    • General Sparse Factorization
      • Fill-in and Reordering
      • Graph Based Approach
    • Sparse Matrix Data Structures
      • Scattering
slide3
5

7

9

3

4

6

8

2

1

Applications

Sparse Matrices

Space Frame

Nodal Matrix

Space Frame

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Unknowns : Joint positions

Equations : forces = 0

slide4
Applications

Sparse Matrices

Resistor Grid

Unknowns : Node Voltages

Equations : currents = 0

slide5
Applications

Sparse Matrices

Nodal Formulation

Resistor Grid

Matrix non-zero locations for 100 x 10 Resistor Grid

slide6
Applications

Sparse Matrices

Nodal Formulation

Temperature in a cube

Temperature known on surface, determine interior temperature

Circuit

Model

slide7
Outline
  • Solution of Dense Linear Systems
  • Solution of Sparse Linear Systems
    • LU Factorization Reminder.
    • Example of Problems with Sparse Matrices
      • Struts and joints, resistor grids, 3-d heat flow
    • Tridiagonal Matrix Factorization
    • General Sparse Factorization
      • Fill-in and Reordering
      • Graph Based Approach
    • Sparse Matrix Data Structures
      • Scattering
slide8
Tridiagonal Example

Sparse Matrices

Nodal Formulation

Matrix Form

How many operations to do LU factorization?

m

slide9
Pivot

Multiplier

Tridiagonal Example

Sparse Matrices

GE Algorithm

For i = 1 to n-1 {“For each Row”

For j = i+1 to n {“For each target Row below the source”

For k = i+1 to n {“For each Row element beyond Pivot”

}

}

}

Order n Operations!

slide10
Outline
  • Solution of Dense Linear Systems
  • Solution of Sparse Linear Systems
    • LU Factorization Reminder.
    • Example of Problems with Sparse Matrices
      • Struts and joints, resistor grids, 3-d heat flow
    • Tridiagonal Matrix Factorization
    • General Sparse Factorization
      • Fill-in and Reordering
      • Graph Based Approach
    • Sparse Matrix Data Structures
      • Scattering
slide11
Sparse Matrix Fill-In

Sparse Matrices

Example

Resistor Example

Nodal Matrix

0

Symmetric

Diagonally Dominant

slide12
X

X

X = fill in where 0 became non-zero

Fill-In

Sparse Matrices

Example

Matrix Non zero structure

Fill Ins

X

X

X

X

X= Non zero

slide13
X

X

X

X

X

Fill-In

Sparse Matrices

Second Example

Fill-ins Propagate

X

X

X

X

X

Fill-ins from Step 1 result in Fill-ins in step 2

slide14
0

Fill-In

Sparse Matrices

Reordering

Fill-ins

0

No Fill-ins

Node Reordering

- Can reduce fill-in

- Preserves properties (Symmetry, Diagonal Dominance)

- Equivalent to swapping rows and columns

slide15
Possible Fill-in

Locations

Fill-in Estimate = (Non zeros in unfactored part of Row -i)

(Non zeros in unfactored part of Col -i)

Markowitz product

Fill-In

Sparse Matrices

Reordering

Where can fill-in occur ?

Already Factored

Multipliers

slide16
Fill-In

Sparse Matrices

Reordering

Markowitz Reordering

Greedy Algorithm !

slide17
1

2

3

1

0

0

3

2

Fill-In

Sparse Matrices

Reordering

Why only try diagonals ?

  • Corresponds to node reordering in Nodal formulation
  • Reduces search cost
  • Preserves Matrix Properties - Diagonal Dominance - Symmetry
slide18
Fill-In

Sparse Matrices

Pattern of a Filled-in Matrix

Very Sparse

Very Sparse

Dense

slide19
Fill-In

Sparse Matrices

Unfactored Random Matrix

Symmetric

slide20
Fill-In

Sparse Matrices

Factored Random Matrix

Symmetric

slide21
Outline
  • Solution of Dense Linear Systems
  • Solution of Sparse Linear Systems
    • LU Factorization Reminder.
    • Example of Problems with Sparse Matrices
      • Struts and joints, resistor grids, 3-d heat flow
    • Tridiagonal Matrix Factorization
    • General Sparse Factorization
      • Fill-in and Reordering
      • Graph Based Approach
    • Sparse Matrix Data Structures
      • Scattering
slide22
Matrix Graphs

Sparse Matrices

Construction

Structurally Symmetric Matrices and Graphs

1

X

X

X

X

X

X

X

2

X

X

X

X

3

X

X

X

4

X

X

X

5

  • One Node Per Matrix Row
  • One Edge Per Off-diagonal Pair
slide23
1

2

X

X

X

X

3

X

X

X

4

X

X

X

X

5

X

X

X

X

X

X

Matrix Graphs

Sparse Matrices

Markowitz Products

Markowitz Products =

(Node Degree)2

slide24
Matrix Graphs

Sparse Matrices

Factorization

One Step of LU Factorization

1

X

X

X

X

X

X

X

X

2

X

X

X

X

X

3

X

X

X

4

X

X

X

X

X

5

  • Delete the node associated with pivot row
  • “Tie together” the graph edges
slide25
1

2

3

4

5

1

2

3

4

5

Matrix Graphs

Sparse Matrices

Example

Graph

Markowitz products

( = Node degree)

slide26
5

5

4

3

2

3

4

Graph

Matrix Graphs

Sparse Matrices

Example

Swap 2 with 1

X

1

2

2

1

X

Eliminate 1

slide27
Graphs

Sparse Matrices

Resistor Grid Example

Unknowns : Node Voltages

Equations : currents = 0

slide28
Matrix Graphs

Sparse Matrices

Grid Example

slide29
What should you pivot for?

Sparse Matrices

For growth control?

Or to avoid fill-ins?

A) LU factorization applied to a strictly diagonally dominant matrix will never produce a zero pivot

B) The matrix entries produced by LU factorization applied to a strictly diagonally dominant matrix will never increase by more than a factor 2(n-1)

[which is anyway the best you can do by pivoting for growth control]

Bottom line:

if your matrix is strictly diagonally dominant

no need for numerical pivot for growth control!

slide30
Sparse Factorization Approach
  • Assume matrix requires NO numerical pivoting.

Diagonally dominant or symmetric positive definite.

  • Pre-Process:
    • Use Graphs to Determine Matrix Ordering
      • Many graph manipulation tricks used.
    • Form Data Structure for Storing Filled-in Matrix
      • Lots of additional non-zero added
  • Put numerical values in Data Structure and factor

Computation must be organized carefully!

slide31
Summary of Sparse Systems
  • LU Factorization and Diagonal Dominance.
    • Factor without numerical pivoting
  • Sparse Matrices
    • Struts, resistor grids, 3-d heat flow -> O(N) non-zeros
  • Tridiagonal Matrix Factorization
    • Factor in O(N) operations
  • General Sparse Factorization
    • Markowitz Reordering to minimize fill
  • Graph Based Approach
    • Factorization and Fill-in
    • Useful for estimating Sparse GE complexity
slide32
Outline
  • Solution of Sparse Linear Systems
    • Example of Problems with Sparse Matrices
    • Tridiagonal Matrix Factorization
    • General Sparse Factorization
      • Fill-in and Reordering
      • Graph Based Approach
      • Summary of the Algorithm
    • Sparse Matrix Data Structures
      • Scattering and symbolic fill in
slide33
Sparse Data Structure

Sparse Matrices

Vector of row

pointers

Arrays of Data in a Row

Matrix entries

Val 11

Val 12

Val 1K

1

Column index

Col 11

Col 12

Col 1K

Val 21

Val 22

Val 2L

Col 21

Col 22

Col 2L

Row pointers

Column Indices

Goal:

Never store a 0

Never multiply by 0

Val N1

Val N2

Val Nj

N

Col N1

Col N2

Col Nj

slide34
Sparse Data Structure

Sparse Matrices

Problem of Misses

Eliminating Source Row i from Target row j

Row i

Row j

Must read all the row j entries to find the 3 that match row i

slide35
Sparse Data Structure

Sparse Matrices

Data on Misses

More misses

than

ops!

Every Miss is an unneeded Memory Reference!

slide36
Sparse Data Structure

Sparse Matrices

Scattering for Miss Avoidance

Row j

Use target row approach – Row j is the target.

1) Read all the elements in Row j, and scatter them in an n-length vector

2) Access only the needed elements using array indexing!

slide37
Sparse Matrices – Another Data Structure

Orthogonal linked list

But if fill in occurs, pointer structures change. What do we do?

Pre-compute pivoting order and sparse fill in structure symbolically

slide38
Summary of Sparse Systems
  • LU Factorization and Diagonal Dominance.
    • Factor without numerical pivoting
    • hard problems (ill-conditioned)
  • Sparse Matrices
    • Struts, resistor grids, 3-d heat flow -> O(N) nonzeros
  • Tridiagonal Matrix Factorization
    • Factor in O(N) operations
  • General Sparse Factorization
    • Markowitz Reordering to minimize fill
  • Graph Based Approach
    • Factorization and Fill-in
    • Useful for estimating Sparse GE complexity
  • Sparse Data Structures
    • Scattering and symbolic fill in
ad