Loading in 5 sec....

Steepest Decent and Conjugate Gradients (CG)PowerPoint Presentation

Steepest Decent and Conjugate Gradients (CG)

- 101 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Steepest Decent and Conjugate Gradients (CG)' - serina-stephenson

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Steepest Decent

Steepest Decent

Steepest Decent

Steepest Decent and Conjugate Gradients (CG)

- Solving of the linear equation system

Steepest Decent and Conjugate Gradients (CG)

- Solving of the linear equation system
- Problem: dimension n too big, or not enough time for gauss elimination
Iterative methods are used to get an approximate solution.

Steepest Decent and Conjugate Gradients (CG)

- Solving of the linear equation system
- Problem: dimension n too big, or not enough time for gauss elimination
Iterative methods are used to get an approximate solution.

- Definition Iterative method: given starting point , do steps
hopefully converge to the right solution

starting issues

- Solving is equivalent to minimizing

starting issues

- Solving is equivalent to minimizing
- A has to be symmetric positive definite:

starting issues

- If A is also positive definite the solution of is the minimum

starting issues

- If A is also positive definite the solution of is the minimum

starting issues

- error:
The norm of the error shows how far we are away from the exact solution, but can’t be computed without knowing of the exact solution .

starting issues

- error:
The norm of the error shows how far we are away from the exact solution, but can’t be computed without knowing of the exact solution .

- residual:
can be calculated

Steepest Decent

- We are at the point . How do we reach ?

Steepest Decent

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )

Steepest Decent

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )
- how far should we go?

Steepest Decent

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )
- how far should we go?
Choose so that is minimized:

Steepest Decent

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )
- how far should we go?
Choose so that is minimized:

Steepest Decent

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )
- how far should we go?
Choose so that is minimized:

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )
- how far should we go?
Choose so that is minimized:

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )
- how far should we go?
Choose so that is minimized:

- We are at the point . How do we reach ?
- Idea: go into the direction in which decreases most quickly ( )
- how far should we go?
Choose so that is minimized:

Steepest Decent

one step of steepest decent can be calculated as follows:

Steepest Decent

one step of steepest decent can be calculated as follows:

- stopping criterion: or with an given small
It would be better to use the error instead of the residual, but you can’t calculate the error.

Steepest Decent

Method of steepest decent:

Steepest Decent

- As you can see the starting point is important!

Steepest Decent

- As you can see the starting point is important!
When you know anything about the solution use it to guess a good starting point. Otherwise you can choose a starting point you want e.g. .

Steepest Decent - Convergence

- Definition energy norm:

Steepest Decent - Convergence

- Definition energy norm:
- Definition condition:
( is the largest and the smallest eigenvalue of A)

Steepest Decent - Convergence

- Definition energy norm:
- Definition condition:
( is the largest and the smallest eigenvalue of A)

convergence gets worse when the condition gets larger

Conjugate Gradients

- is there a better direction?

Conjugate Gradients

- is there a better direction?
- Idea: orthogonal search directions

Conjugate Gradients

- is there a better direction?
- Idea: orthogonal search directions

Conjugate Gradients

- is there a better direction?
- Idea: orthogonal search directions
- only walk once in each direction and minimize

Conjugate Gradients

- is there a better direction?
- Idea: orthogonal search directions
- only walk once in each direction and minimize
maximal n steps are needed to reach the exact solution

Conjugate Gradients

- is there a better direction?
- Idea: orthogonal search directions
- only walk once in each direction and minimize
maximal n steps are needed to reach the exact solution

has to be orthogonal to

Conjugate Gradients

- example with the coordinate axes as orthogonal search directions:

Conjugate Gradients

- example with the coordinate axes as orthogonal search directions:
Problem: can’t be computed because (you don’t know !)

Conjugate Gradients

- new idea: A-orthogonal

Conjugate Gradients

- new idea: A-orthogonal
- Definition A-orthogonal: A-orthogonal
(reminder: orthogonal: )

Conjugate Gradients

- new idea: A-orthogonal
- Definition A-orthogonal: A-orthogonal
(reminder: orthogonal: )

- now has to be A-orthogonal to

Conjugate Gradients

- new idea: A-orthogonal
- Definition A-orthogonal: A-orthogonal
(reminder: orthogonal: )

- now has to be A-orthogonal to

Conjugate Gradients

- new idea: A-orthogonal
- Definition A-orthogonal: A-orthogonal
(reminder: orthogonal: )

- now has to be A-orthogonal to
can be computed!

Conjugate Gradients

- A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt).

Conjugate Gradients

- Gram-Schmidt:
linearly independent vectors

Conjugate Gradients

- Gram-Schmidt:
linearly independent vectors

Conjugate Gradients

- Gram-Schmidt:
linearly independent vectors

- conjugate Gram-Schmidt:

Conjugate Gradients

- A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt).
- CG works by setting (makes conjugate Gram-Schmidt easy)

Conjugate Gradients

- A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt).
- CG works by setting (makes conjugate Gram-Schmidt easy)
with

Conjugate Gradients - Convergence

- for steepest decent for CG
Convergence of CG is much better!

Download Presentation

Connecting to Server..