- 235 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Solution of Sparse Linear Systems' - marinel

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Numerical Simulation

CSE245 Spring 2010

Solution of Sparse Linear Systems

Thanks to: Kin Sou, Deepak Ramaswamy, Michal Rewienski, Jacob White, Shihhsien Kuo and Karen Veroy and

especially Luca Daniel

Outline of today’s lecture

- Solution of Sparse Linear Systems
- Examples of Problems with Sparse Matrices
- Struts and joints, resistor grids, 3-D heat flow
- Tridiagonal Matrix Factorization
- General Sparse Factorization
- Fill-in and Reordering
- Graph Based Approach
- Sparse Matrix Data Structures
- Scattering

5

7

9

3

4

6

8

2

1

Applications

Sparse Matrices

Space Frame

Nodal Matrix

Space Frame

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Unknowns : Joint positions

Equations : forces = 0

Applications

Sparse Matrices

Nodal Formulation

Resistor Grid

Matrix non-zero locations for 100 x 10 Resistor Grid

Applications

Sparse Matrices

Nodal Formulation

Temperature in a cube

Temperature known on surface, determine interior temperature

Circuit

Model

Outline

- Solution of Dense Linear Systems
- Solution of Sparse Linear Systems
- LU Factorization Reminder.
- Example of Problems with Sparse Matrices
- Struts and joints, resistor grids, 3-d heat flow
- Tridiagonal Matrix Factorization
- General Sparse Factorization
- Fill-in and Reordering
- Graph Based Approach
- Sparse Matrix Data Structures
- Scattering

Tridiagonal Example

Sparse Matrices

Nodal Formulation

Matrix Form

How many operations to do LU factorization?

m

Pivot

Multiplier

Tridiagonal Example

Sparse Matrices

GE Algorithm

For i = 1 to n-1 {“For each Row”

For j = i+1 to n {“For each target Row below the source”

For k = i+1 to n {“For each Row element beyond Pivot”

}

}

}

Order n Operations!

Outline

- Solution of Dense Linear Systems
- Solution of Sparse Linear Systems
- LU Factorization Reminder.
- Example of Problems with Sparse Matrices
- Struts and joints, resistor grids, 3-d heat flow
- Tridiagonal Matrix Factorization
- General Sparse Factorization
- Fill-in and Reordering
- Graph Based Approach
- Sparse Matrix Data Structures
- Scattering

Sparse Matrix Fill-In

Sparse Matrices

Example

Resistor Example

Nodal Matrix

0

Symmetric

Diagonally Dominant

X

X

X = fill in where 0 became non-zero

Fill-In

Sparse Matrices

Example

Matrix Non zero structure

Fill Ins

X

X

X

X

X= Non zero

X

X

X

X

X

Fill-In

Sparse Matrices

Second Example

Fill-ins Propagate

X

X

X

X

X

Fill-ins from Step 1 result in Fill-ins in step 2

0

Fill-In

Sparse Matrices

Reordering

Fill-ins

0

No Fill-ins

Node Reordering

- Can reduce fill-in

- Preserves properties (Symmetry, Diagonal Dominance)

- Equivalent to swapping rows and columns

Possible Fill-in

Locations

Fill-in Estimate = (Non zeros in unfactored part of Row -i)

(Non zeros in unfactored part of Col -i)

Markowitz product

Fill-In

Sparse Matrices

Reordering

Where can fill-in occur ?

Already Factored

Multipliers

1

2

3

1

0

0

3

2

Fill-In

Sparse Matrices

Reordering

Why only try diagonals ?

- Corresponds to node reordering in Nodal formulation

- Reduces search cost

- Preserves Matrix Properties - Diagonal Dominance - Symmetry

Outline

- Solution of Dense Linear Systems
- Solution of Sparse Linear Systems
- LU Factorization Reminder.
- Example of Problems with Sparse Matrices
- Struts and joints, resistor grids, 3-d heat flow
- Tridiagonal Matrix Factorization
- General Sparse Factorization
- Fill-in and Reordering
- Graph Based Approach
- Sparse Matrix Data Structures
- Scattering

Matrix Graphs

Sparse Matrices

Construction

Structurally Symmetric Matrices and Graphs

1

X

X

X

X

X

X

X

2

X

X

X

X

3

X

X

X

4

X

X

X

5

- One Node Per Matrix Row
- One Edge Per Off-diagonal Pair

1

2

X

X

X

X

3

X

X

X

4

X

X

X

X

5

X

X

X

X

X

X

Matrix Graphs

Sparse Matrices

Markowitz Products

Markowitz Products =

(Node Degree)2

Matrix Graphs

Sparse Matrices

Factorization

One Step of LU Factorization

1

X

X

X

X

X

X

X

X

2

X

X

X

X

X

3

X

X

X

4

X

X

X

X

X

5

- Delete the node associated with pivot row
- “Tie together” the graph edges

What should you pivot for?

Sparse Matrices

For growth control?

Or to avoid fill-ins?

A) LU factorization applied to a strictly diagonally dominant matrix will never produce a zero pivot

B) The matrix entries produced by LU factorization applied to a strictly diagonally dominant matrix will never increase by more than a factor 2(n-1)

[which is anyway the best you can do by pivoting for growth control]

Bottom line:

if your matrix is strictly diagonally dominant

no need for numerical pivot for growth control!

Sparse Factorization Approach

- Assume matrix requires NO numerical pivoting.

Diagonally dominant or symmetric positive definite.

- Pre-Process:
- Use Graphs to Determine Matrix Ordering
- Many graph manipulation tricks used.
- Form Data Structure for Storing Filled-in Matrix
- Lots of additional non-zero added
- Put numerical values in Data Structure and factor

Computation must be organized carefully!

Summary of Sparse Systems

- LU Factorization and Diagonal Dominance.
- Factor without numerical pivoting
- Sparse Matrices
- Struts, resistor grids, 3-d heat flow -> O(N) non-zeros
- Tridiagonal Matrix Factorization
- Factor in O(N) operations
- General Sparse Factorization
- Markowitz Reordering to minimize fill
- Graph Based Approach
- Factorization and Fill-in
- Useful for estimating Sparse GE complexity

Outline

- Solution of Sparse Linear Systems
- Example of Problems with Sparse Matrices
- Tridiagonal Matrix Factorization
- General Sparse Factorization
- Fill-in and Reordering
- Graph Based Approach
- Summary of the Algorithm
- Sparse Matrix Data Structures
- Scattering and symbolic fill in

Sparse Data Structure

Sparse Matrices

Vector of row

pointers

Arrays of Data in a Row

Matrix entries

Val 11

Val 12

Val 1K

1

Column index

Col 11

Col 12

Col 1K

Val 21

Val 22

Val 2L

Col 21

Col 22

Col 2L

Row pointers

Column Indices

Goal:

Never store a 0

Never multiply by 0

Val N1

Val N2

Val Nj

N

Col N1

Col N2

Col Nj

Sparse Data Structure

Sparse Matrices

Problem of Misses

Eliminating Source Row i from Target row j

Row i

Row j

Must read all the row j entries to find the 3 that match row i

Sparse Data Structure

Sparse Matrices

Data on Misses

More misses

than

ops!

Every Miss is an unneeded Memory Reference!

Sparse Data Structure

Sparse Matrices

Scattering for Miss Avoidance

Row j

Use target row approach – Row j is the target.

1) Read all the elements in Row j, and scatter them in an n-length vector

2) Access only the needed elements using array indexing!

Sparse Matrices – Another Data Structure

Orthogonal linked list

But if fill in occurs, pointer structures change. What do we do?

Pre-compute pivoting order and sparse fill in structure symbolically

Summary of Sparse Systems

- LU Factorization and Diagonal Dominance.
- Factor without numerical pivoting
- hard problems (ill-conditioned)
- Sparse Matrices
- Struts, resistor grids, 3-d heat flow -> O(N) nonzeros
- Tridiagonal Matrix Factorization
- Factor in O(N) operations
- General Sparse Factorization
- Markowitz Reordering to minimize fill
- Graph Based Approach
- Factorization and Fill-in
- Useful for estimating Sparse GE complexity
- Sparse Data Structures
- Scattering and symbolic fill in

Download Presentation

Connecting to Server..