Fminsearch uses Nelder Mead • Fminsearch finds minimum of a function of several variables starting from an initial value. • Unconstraint nonlinear optimization method, meaning we cannot give upper or lower bounds for parameters • Downhill simplex method • Global optimization method (finds global minimum)
Nelder Mead • The Nelder–Mead technique is a heuristic search method that can converge to non-stationary points. But it is easy to use and will converge for a large class of problems. • The Nelder–Mead technique was proposed by John Nelder & Roger Mead (1965). • The method uses the concept of a simplex, which is a special polynomium type with N + 1 vertices in N dimensions. • Examples of simplexes include a line segment on a line, a triangle on a plane, a tetrahedron in three-dimensional space and so forth.
The Nelder-Mead Algorithm • Given n+1 vertices xi, i=1… n+1 and associated function values f(xi). • Define the following coefficients • R=1 (reflection) • K=0.5 (contraction) • E=2 (expansion) • S=0.5 (shrinkage)
The Nelder-Mead Algorithm • Sort by function value: Order the vertices to satisfy f1 < f2 < … < fn+1 • Calculate xm = sum xi (the average of all the points except the worst) • Reflection. Compute xr = xm + R(xm-xn+1) and evaluate f(xr). If f1 < fr< fn accept xr and terminate the iteration.
The Nelder-Mead Algorithm • Expansion. If fr< f1 calculate xe = xm+ K (xr- xm) and evaluate f(xe). If fe< fr, accept xe; otherwise accept xr. Terminate the iteration.
The Nelder-Mead Algorithm • Contraction. If fr> fn, perform a contraction between xm and the better of xr and xn+1. • Outside. If fn < fr< fn+1 calculate xoc= xm+ K (xr- xm) and evaluate f(xoc). If foc< fr, accept xoc and terminate the iteration; otherwise do a shrink. • Inside. If fr> fn+1 calculate xic= xm– K (xm- xn+1) and evaluate f(xic). If fic< fn+1 accept xic and terminate the iteration; otherwise do a shrink.
The Nelder-Mead Algorithm • Shrink. Evaluate f at the n points vi = xi + S (xi-x1), i = 2,….,n+1. The vertices of the simplex at the next iteration are x1, v2, …, vn+1.
Example Nelder-Mead Algorithm Parameter 2 Parameter 1