1 / 39

Non-linear optimization

Non-linear optimization. An overview, problems and a guide. Optimization. Unconstraint non-linear optimization. E( w ). w 2. w 1. Classes of Methods. Linear optimization Constraint <-> unconstraint Gradient based 1 st order, 2 nd order Genetic Algorithms, Evolutionary Strategies

lynsey
Download Presentation

Non-linear optimization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Non-linear optimization An overview, problems and a guide

  2. Optimization • Unconstraint non-linear optimization E(w) w2 w1

  3. Classes of Methods • Linear optimization • Constraint <-> unconstraint • Gradient based • 1st order, 2nd order • Genetic Algorithms,Evolutionary Strategies • Stochastic methods (Simulated Annealing, Tabu Search, …)

  4. Ellipsoid

  5. Rosenbrock-function

  6. Cross-Function

  7. Canyon-function

  8. Step-function

  9. Performance criteria • Number of function evaluations • Number of gradient calculation • Time • Number of fails • Number of method params. • Sensitivity of method params. • Accuracy

  10. Methods • Direct methods • Successive variation • Hooke-Jeeves • Gradient based methods • Gradient decent • Back-propagation • Polak-Ribiere • Second order methods • Newton-Raphson • BFGS

  11. Successive Variation

  12. Successive Variation

  13. Successive Variation

  14. Successive Variation

  15. Hooke-Jeeves

  16. Hooke-Jeeves

  17. Hooke-Jeeves

  18. Gradient descent

  19. Gradient descent

  20. Gradient descent

  21. Gradient Decent

  22. Gradient descent

  23. Gradient descent

  24. Back-propagation Gradient decent Momentum

  25. Back-propagation

  26. Back-propagation Error E Cycle

  27. Conjugated gradients Qn property

  28. Beam search

  29. Polak-Ribiere Beam search

  30. Polak-Ribiere

  31. Newton-method Q1 property

  32. BFGS

  33. BFGS

  34. Comparison: Ellipsoid

  35. Comparison: Cross-Function

  36. Comparison: Rosenbrock-Function

  37. Comparison: Canyon-Function n(E)=8983

  38. Comparison: Step-Function n(E)=2487 n(E)=2448

  39. Decision tree Complexity #minima many some few one Knowledge MC / SA GA / ES Multi-start differentiable no yes elliptic? aligned? no yes no yes yes #parameters coordinate axis channels? G / PR/ BFGS many few no yes curved along axes flat NM / LBFGS HJ / ROS ROS SV #parameters QP / RPROP BP many few PR / LBFGS BFGS

More Related