1 / 68

Steepest Decent and Conjugate Gradients (CG)

Steepest Decent and Conjugate Gradients (CG). Steepest Decent and Conjugate Gradients (CG). Solving of the linear equation system. Steepest Decent and Conjugate Gradients (CG). Solving of the linear equation system Problem : dimension n too big, or not enough time for gauss elimination

jalen
Download Presentation

Steepest Decent and Conjugate Gradients (CG)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Steepest Decent and Conjugate Gradients (CG)

  2. Steepest Decent and Conjugate Gradients (CG) • Solving of the linear equation system

  3. Steepest Decent and Conjugate Gradients (CG) • Solving of the linear equation system • Problem: dimension n too big, or not enough time for gauss elimination Iterative methods are used to get an approximate solution.

  4. Steepest Decent and Conjugate Gradients (CG) • Solving of the linear equation system • Problem: dimension n too big, or not enough time for gauss elimination Iterative methods are used to get an approximate solution. • Definition Iterative method: given starting point , do steps hopefully converge to the right solution

  5. starting issues

  6. starting issues • Solving is equivalent to minimizing

  7. starting issues • Solving is equivalent to minimizing • A has to be symmetric positive definite:

  8. starting issues

  9. starting issues • If A is also positive definite the solution of is the minimum

  10. starting issues • If A is also positive definite the solution of is the minimum

  11. starting issues • error: The norm of the error shows how far we are away from the exact solution, but can’t be computed without knowing of the exact solution .

  12. starting issues • error: The norm of the error shows how far we are away from the exact solution, but can’t be computed without knowing of the exact solution . • residual: can be calculated

  13. Steepest Decent

  14. Steepest Decent • We are at the point . How do we reach ?

  15. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( )

  16. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( ) • how far should we go?

  17. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( ) • how far should we go? Choose so that is minimized:

  18. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( ) • how far should we go? Choose so that is minimized:

  19. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( ) • how far should we go? Choose so that is minimized:

  20. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( ) • how far should we go? Choose so that is minimized:

  21. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( ) • how far should we go? Choose so that is minimized:

  22. Steepest Decent • We are at the point . How do we reach ? • Idea: go into the direction in which decreases most quickly ( ) • how far should we go? Choose so that is minimized:

  23. Steepest Decent one step of steepest decent can be calculated as follows:

  24. Steepest Decent one step of steepest decent can be calculated as follows: • stopping criterion: or with an given small It would be better to use the error instead of the residual, but you can’t calculate the error.

  25. Steepest Decent Method of steepest decent:

  26. Steepest Decent • As you can see the starting point is important!

  27. Steepest Decent • As you can see the starting point is important! When you know anything about the solution use it to guess a good starting point. Otherwise you can choose a starting point you want e.g. .

  28. Steepest Decent - Convergence

  29. Steepest Decent - Convergence • Definition energy norm:

  30. Steepest Decent - Convergence • Definition energy norm: • Definition condition: ( is the largest and the smallest eigenvalue of A)

  31. Steepest Decent - Convergence • Definition energy norm: • Definition condition: ( is the largest and the smallest eigenvalue of A) convergence gets worse when the condition gets larger

  32. Conjugate Gradients

  33. Conjugate Gradients • is there a better direction?

  34. Conjugate Gradients • is there a better direction? • Idea: orthogonal search directions

  35. Conjugate Gradients • is there a better direction? • Idea: orthogonal search directions

  36. Conjugate Gradients • is there a better direction? • Idea: orthogonal search directions • only walk once in each direction and minimize

  37. Conjugate Gradients • is there a better direction? • Idea: orthogonal search directions • only walk once in each direction and minimize maximal n steps are needed to reach the exact solution

  38. Conjugate Gradients • is there a better direction? • Idea: orthogonal search directions • only walk once in each direction and minimize maximal n steps are needed to reach the exact solution has to be orthogonal to

  39. Conjugate Gradients • example with the coordinate axes as orthogonal search directions:

  40. Conjugate Gradients • example with the coordinate axes as orthogonal search directions: Problem: can’t be computed because (you don’t know !)

  41. Conjugate Gradients • new idea: A-orthogonal

  42. Conjugate Gradients • new idea: A-orthogonal • Definition A-orthogonal: A-orthogonal (reminder: orthogonal: )

  43. Conjugate Gradients • new idea: A-orthogonal • Definition A-orthogonal: A-orthogonal (reminder: orthogonal: ) • now has to be A-orthogonal to

  44. Conjugate Gradients • new idea: A-orthogonal • Definition A-orthogonal: A-orthogonal (reminder: orthogonal: ) • now has to be A-orthogonal to

  45. Conjugate Gradients • new idea: A-orthogonal • Definition A-orthogonal: A-orthogonal (reminder: orthogonal: ) • now has to be A-orthogonal to can be computed!

  46. Conjugate Gradients • A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt).

  47. Conjugate Gradients • Gram-Schmidt: linearly independent vectors

  48. Conjugate Gradients • Gram-Schmidt: linearly independent vectors

  49. Conjugate Gradients • Gram-Schmidt: linearly independent vectors • conjugate Gram-Schmidt:

  50. Conjugate Gradients • A set of A-orthogonal directions can be found with n linearly independent vectors and conjugate Gram-Schmidt (same idea as Gram-Schmidt). • CG works by setting (makes conjugate Gram-Schmidt easy)

More Related