Optimization methods review
This presentation is the property of its rightful owner.
Sponsored Links
1 / 42

Optimization methods Review PowerPoint PPT Presentation


  • 110 Views
  • Uploaded on
  • Presentation posted in: General

Optimization methods Review. Mateusz Sztangret. Faculty of Metal Engineering and Industrial Computer Science Department of Applied Computer Science and Modelling Krakow, 03-11-2010 r. Outline of the presentation. Basic concepts of optimization Review of optimization methods

Download Presentation

Optimization methods Review

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Optimization methods review

Optimization methodsReview

Mateusz Sztangret

Faculty of Metal Engineering and Industrial Computer Science

Department of Applied Computer Science and Modelling

Krakow, 03-11-2010 r.


Outline of the presentation

Outlineof the presentation

Basic concepts of optimization

Review of optimization methods

  • gradientless methods,

  • gradient methods,

  • linear programming methods,

  • non-deterministic methods

    Characteristics of selected methods

  • method of steepestdescent

  • genetic algorithm


Basic concepts of optimization

Basic concepts of optimization

Man’s longing for perfection finds expression in the theory of optimization. It studies how to describe and attain what is Best, once one knows how to measure and alter what is Good and Bad… Optimization theory encompasses the quantitative study of optima and methods for finding them.

Beightler, Phillips, Wilde

Foundations of Optimization


Basic concepts of optimization1

Basic concepts of optimization

Optimization /optimum/ - process of finding the best solution

Usually the aim of the optimization is to find better solution than previous attached


Basic concepts of optimization2

Basic concepts of optimization

Specification of the optimization problem:

  • definition of the objective function,

  • selection of optimization variables,

  • identification of constraints.


Mathematical definition

Mathematical definition

where:

  • x is the vector of variables, also called unknowns or parameters;

  • f is the objective function, a (scalar) function of x that we want to maximize or minimize;

  • giand hi are constraint functions, which are scalar functions of x that define certain equations and inequalities that the unknown vector x must satisfy.


Set of allowed solutions

Set of allowed solutions

Constrain functions define the set of allowed solution that is a set of points which we consider in the optimization process.

X

Xd


Obtained solution

Obtained solution

Solution is called global minimum if,

for all

Solution is called local minimum if there is a neighbourhood N of such that

for all

Global minimum as well as local minimum is never exact due to limited accuracy of numerical methods and round off error


Local and global solutions

Local and global solutions

f(x)

local minimum

global minimum

x


Problems with multimodal objective function

Problems with multimodal objective function

f(x)

start

start

x


Discontinuous objective function

Discontinuous objective function

f(x)

Discontinuous function

x

3


Minimum or maximum

Minimum or maximum

f(x)

f

c

x*

x

– c

– f


General optimization flowchart

General optimization flowchart

Start

Set starting point x(0)

i = 0

i = i + 1

Calculatef(x(i))

NO

Stop condition

x(i+1) = x(i) + Δx(i)

YES

Stop


Stop conditions

Stop conditions

Commonly used stop conditions are as follows:

  • obtain sufficient solution,

  • lack of progress,

  • reach the maximum number of iterations


Classification of optimization methods

Classification of optimization methods


Optimization methods

Optimization methods

The are several type of optimization algorithms:

  • gradientless methods,

    • line search methods,

    • multidimensional methods,

  • gradient methods,

  • linear programming methods,

  • non-deterministic methods


Gradientless methods

Gradientless methods

  • Line search methods

    • Expansion method

    • Golden ratio method

  • Multidimensional methods

    • Fibonacci method

    • Method based on Lagrange interpolation

    • Hooke-Jeeves method

    • Rosenbrock method

    • Nelder-Mead simplex method

    • Powell method


Features of gradientless methods

Features of gradientless methods

Advantages:

  • simplicity,

  • they do not require computing derivatives of the objective function.

    Disadvantages:

  • they find first obtained minimum

  • they demand unimodality and continuity of objective function


Gradient methods

Gradient methods

  • Method of steepest descent

  • Conjugate gradients method

  • Newton method

  • Davidon-Fletcher-Powell method

  • Broyden-Fletcher-Goldfarb-Shanno method


Features of gradient methods

Features of gradient methods

Advantages:

  • simplicity,

  • greater effciency in comparsion with gradientless methods.

    Disadvantages:

  • they find first obtained minimum

  • they demand unimodality, continuity and differentiability of objective function


Linear programming

Linear programming

If both the objective function and constraints are linear we can use one of the linear programming method:

  • Graphical method

  • Simplex method


Non deterministic method

Non-deterministic method

  • Monte Carlo method

  • Genetic algorithms

  • Evolutionary algorithms

    • strategy (1 + 1)

    • strategy (μ + λ)

    • strategy (μ, λ)

  • Particle swarm optimization

  • Simulated annealing method

  • Ant colony optimization

  • Artificial immune system


Features of non deterministic methods

Features of non-deterministic methods

Advantages:

  • any nature of optimised objective function,

  • they do not require computing derivatives of the objective function.

    Disadvantages:

  • high number of objective function calls


Optimization with constraints

Optimization with constraints

Ways of integrating constrains

  • External penalty function method

  • Internal penalty function method


Multicriteria optimization

Multicriteria optimization

In some cases solved problem is defined by few objective function. Usually when we improve one the others get whose.

  • weighted criteria method

  • ideal point method


W eighted criteria method

Weighted criteria method

Method involves the transformation

multicriterial problem into

one-criterial problem by adding

particular objective functions.


I deal point method

Ideal point method

In this method we choose

an ideal solution which is

outside the set of allowed

solution and the searching

optimal solution inside

the set of allowed solution

which is closest the

the ideal point. Distance we can

measure using various metrics

Ideal point

Allowed solution


Method of steepest descent

Method of steepest descent

Algorithm consists of following steps:

  • Substitute data:

    • u0 – starting point

    • maxit – maximum number of iterations

    • e – require accuracy of solution

    • i = 0 – iteration number

  • Compute gradient in ui


Method of steepest descent1

Method of steepest descent

  • Choose the search direction

  • Find optimal solution along the chosen direction (using any line search method).

  • If stop conditions are not satisfied increased i and go to step 2.


Zigzag effect

Zigzag effect

Let’s consider a problem

of finding minimum

of function:

f(u)=u12+3u22

Starting point:

u0=[-2 3]

Isolines


Genetic algorithm

Genetic algorithm

Algorithm consists of following steps:

  • Creation of a baseline population.

  • Compute fitness of whole population

  • Selection.

  • Crossing.

  • Mutation.

  • If stop conditions are not satisfied go to step 2.


Creation of a baseline population

Genotype

1 0 1 0 1 0 1 0

0 1 0 1 0 1 0 1

1 1 0 1 0 1 0 0

1 0 1 1 0 1 1 0

0 0 1 0 1 0 1 1

1 1 1 0 0 1 0 0

Objective function value (f(x)=x2)

28900

7225

44944

33124

1849

51984

Creation of a baseline population


Selection

Baseline population

1 0 1 0 1 0 1 0

0 1 0 1 0 1 0 1

1 1 0 1 0 1 0 0

1 0 1 1 0 1 1 0

0 0 1 0 1 0 1 1

1 1 1 0 0 1 0 0

Parents’ population

1 1 1 0 0 1 0 0

1 1 0 1 0 1 0 0

1 1 1 0 0 1 0 0

0 1 0 1 0 1 0 1

1 0 1 1 0 1 1 0

1 0 1 0 1 0 1 0

Selection


Roulette wheel method

Roulette wheel method


Crossing

Parent individual no 1

1 0 1 0 1

Parent individual no 2

0 1 0 1 0

crossing point

Descendant individual no 1

0 1 0

Descendant individual no 2

1 0 1

Crossing


Mutation

Mutation

Parent individual1 0 1 0 1 0 1 0


Mutation1

Mutation1 0 1 0 1 0 1 0

Mutation

r>pm

r>pm

r<pm


Mutation2

Mutation 1 0 0 0 1 0 1 0

Mutation

r<pm

r>pm

r>pm

r>pm

r<pm


Mutation3

Mutation1 0 0 0 1 0 0 0

Mutation

r>pm

r<pm


Mutation4

Mutation

Parent individual1 0 1 0 1 0 1 0

Descendant individual1 0 0 0 1 0 0 0


Genetic algorithm1

Genetic algorithm

After mutation, completion individuals are recorded in the descendant population, which becomes the baseline population for the next algorithm iteration.

If obtained solution satisfies stop condition procedure is terminated. Otherwise selection, crossing and mutation are repeated.


Thank you for your attention

Thank you for your attention!


  • Login