Steepest decent and conjugate gradients cg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 68

Steepest Decent and Conjugate Gradients (CG) PowerPoint PPT Presentation


  • 67 Views
  • Uploaded on
  • Presentation posted in: General

Steepest Decent and Conjugate Gradients (CG). Steepest Decent and Conjugate Gradients (CG). Solving of the linear equation system. Steepest Decent and Conjugate Gradients (CG). Solving of the linear equation system Problem : dimension n too big, or not enough time for gauss elimination

Download Presentation

Steepest Decent and Conjugate Gradients (CG)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Steepest decent and conjugate gradients cg

Steepest Decent and Conjugate Gradients (CG)


Steepest decent and conjugate gradients cg1

Steepest Decent and Conjugate Gradients (CG)

  • Solving of the linear equation system


Steepest decent and conjugate gradients cg2

Steepest Decent and Conjugate Gradients (CG)

  • Solving of the linear equation system

  • Problem: dimension n too big, or not enough time for gauss elimination

    Iterative methods are used to get an approximate solution.


Steepest decent and conjugate gradients cg3

Steepest Decent and Conjugate Gradients (CG)

  • Solving of the linear equation system

  • Problem: dimension n too big, or not enough time for gauss elimination

    Iterative methods are used to get an approximate solution.

  • Definition Iterative method: given starting point , do steps

    hopefully converge to the right solution


Starting issues

starting issues


Starting issues1

starting issues

  • Solving is equivalent to minimizing


Starting issues2

starting issues

  • Solving is equivalent to minimizing

  • A has to be symmetric positive definite:


Starting issues3

starting issues


Starting issues4

starting issues

  • If A is also positive definite the solution of is the minimum


Starting issues5

starting issues

  • If A is also positive definite the solution of is the minimum


Starting issues6

starting issues

  • error:

    The norm of the error shows how far we are away from the exact solution, but can’t be computed without knowing of the exact solution .


Starting issues7

starting issues

  • error:

    The norm of the error shows how far we are away from the exact solution, but can’t be computed without knowing of the exact solution .

  • residual:

    can be calculated


Steepest decent

Steepest Decent


Steepest decent1

Steepest Decent

  • We are at the point . How do we reach ?


Steepest decent2

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )


Steepest decent3

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )

  • how far should we go?


Steepest decent4

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )

  • how far should we go?

    Choose so that is minimized:


Steepest decent5

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )

  • how far should we go?

    Choose so that is minimized:


Steepest decent6

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )

  • how far should we go?

    Choose so that is minimized:


Steepest decent7

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )

  • how far should we go?

    Choose so that is minimized:


Steepest decent8

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )

  • how far should we go?

    Choose so that is minimized:


Steepest decent9

Steepest Decent

  • We are at the point . How do we reach ?

  • Idea: go into the direction in which decreases most quickly ( )

  • how far should we go?

    Choose so that is minimized:


Steepest decent10

Steepest Decent

one step of steepest decent can be calculated as follows:


Steepest decent11

Steepest Decent

one step of steepest decent can be calculated as follows:

  • stopping criterion: or with an given small

    It would be better to use the error instead of the residual, but you can’t calculate the error.


Steepest decent12

Steepest Decent

Method of steepest decent:


Steepest decent13

Steepest Decent

  • As you can see the starting point is important!


Steepest decent14

Steepest Decent

  • As you can see the starting point is important!

    When you know anything about the solution use it to guess a good starting point. Otherwise you can choose a starting point you want e.g. .


Steepest decent convergence

Steepest Decent - Convergence


Steepest decent convergence1

Steepest Decent - Convergence

  • Definition energy norm:


Steepest decent convergence2

Steepest Decent - Convergence

  • Definition energy norm:

  • Definition condition:

    ( is the largest and the smallest eigenvalue of A)


Steepest decent convergence3

Steepest Decent - Convergence

  • Definition energy norm:

  • Definition condition:

    ( is the largest and the smallest eigenvalue of A)

    convergence gets worse when the condition gets larger


Conjugate gradients

Conjugate Gradients


Conjugate gradients1

Conjugate Gradients

  • is there a better direction?


Conjugate gradients2

Conjugate Gradients

  • is there a better direction?

  • Idea: orthogonal search directions


Conjugate gradients3

Conjugate Gradients

  • is there a better direction?

  • Idea: orthogonal search directions


Conjugate gradients4

Conjugate Gradients

  • is there a better direction?

  • Idea: orthogonal search directions

  • only walk once in each direction and minimize


Conjugate gradients5

Conjugate Gradients

  • is there a better direction?

  • Idea: orthogonal search directions

  • only walk once in each direction and minimize

    maximal n steps are needed to reach the exact solution


Conjugate gradients6

Conjugate Gradients

  • is there a better direction?

  • Idea: orthogonal search directions

  • only walk once in each direction and minimize

    maximal n steps are needed to reach the exact solution

    has to be orthogonal to


Conjugate gradients7

Conjugate Gradients

  • example with the coordinate axes as orthogonal search directions:


Conjugate gradients8

Conjugate Gradients

  • example with the coordinate axes as orthogonal search directions:

    Problem: can’t be computed because (you don’t know !)


Conjugate gradients9

Conjugate Gradients

  • new idea: A-orthogonal


Conjugate gradients10

Conjugate Gradients

  • new idea: A-orthogonal

  • Definition A-orthogonal: A-orthogonal

    (reminder: orthogonal: )


Conjugate gradients11

Conjugate Gradients

  • new idea: A-orthogonal

  • Definition A-orthogonal: A-orthogonal

    (reminder: orthogonal: )

  • now has to be A-orthogonal to


Conjugate gradients12

Conjugate Gradients

  • new idea: A-orthogonal

  • Definition A-orthogonal: A-orthogonal

    (reminder: orthogonal: )

  • now has to be A-orthogonal to


Conjugate gradients13

Conjugate Gradients

  • new idea: A-orthogonal

  • Definition A-orthogonal: A-orthogonal

    (reminder: orthogonal: )

  • now has to be A-orthogonal to

    can be computed!


Conjugate gradients14

Conjugate Gradients

  • A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt).


Conjugate gradients15

Conjugate Gradients

  • Gram-Schmidt:

    linearly independent vectors


Conjugate gradients16

Conjugate Gradients

  • Gram-Schmidt:

    linearly independent vectors


Conjugate gradients17

Conjugate Gradients

  • Gram-Schmidt:

    linearly independent vectors

  • conjugate Gram-Schmidt:


Conjugate gradients18

Conjugate Gradients

  • A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt).

  • CG works by setting (makes conjugate Gram-Schmidt easy)


Conjugate gradients19

Conjugate Gradients

  • A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt).

  • CG works by setting (makes conjugate Gram-Schmidt easy)

    with


Conjugate gradients20

Conjugate Gradients


Conjugate gradients21

Conjugate Gradients


Conjugate gradients22

Conjugate Gradients


Conjugate gradients23

Conjugate Gradients


Conjugate gradients24

Conjugate Gradients


Conjugate gradients25

Conjugate Gradients


Conjugate gradients26

Conjugate Gradients


Conjugate gradients27

Conjugate Gradients


Conjugate gradients28

Conjugate Gradients


Conjugate gradients29

Conjugate Gradients


Conjugate gradients30

Conjugate Gradients


Conjugate gradients31

Conjugate Gradients


Conjugate gradients32

Conjugate Gradients


Method of conjugate gradients

Method of Conjugate Gradients:


Conjugate gradients convergence

Conjugate Gradients - Convergence


Conjugate gradients convergence1

Conjugate Gradients - Convergence


Conjugate gradients convergence2

Conjugate Gradients - Convergence

  • for steepest decent for CG

    Convergence of CG is much better!


  • Login