APPLICATION OF ORDER STATISTICS TO TERMINATION OF STOCHASTIC ALGORITHMS

1. APPLICATION OF ORDER STATISTICS TO TERMINATION OF STOCHASTIC ALGORITHMS Vaida Bartkutė, Leonidas Sakalauskas

2. Outline • Introduction; • Application of order statistics to optimality testing and termination of the algorithm: • Stochastic Approximation algorithms; • Simulated Annealing algorithm; • Experimental results; • Conclusions.

3. Introduction; Application of order statistics to optimality testing and termination of the algorithm: Stochastic Approximation algorithms; Simulated Annealing algorithm; Experimental results; Conclusions. Outline

4. Introduction Termination of the algorithm is a topical problem in stochastic and heuristic optimization. We consider the application of order statistics to establish the optimality in Markov type optimization algorithms. We build a method for the estimation of minima confidence intervals using order statistics, which is implemented for optimality testing and termination.

5. Statement of the problem The optimization problem is (minimization) as follows: where is a bounded from below locally Lipshitz function. Denotethe generalized gradient of this functionby Let be the sequenceconstructed by stochastic search algorithm, where ηt=f(xt), t= 0, 1, …..

6. The Markovian algorithms for optimization The Markovian algorithm of random searching represents a Markov chain in which the distribution of probabilities of a point xt+1 depends on a location of the previous point xt and value of function ηt=f(xt) in it, that Examples: Stochastic Approximation; Simulated Annealing; Random Search (Rastrigin method) and etc.

7. Order statistics and target values for optimality testing and termination • Beginning of the problem: Mockus (1968) • Theoretical background: Zilinskas, Zhigljavsky (1991) • Application to maximum location: Chen (1996) • Time-to-target-solution value: Aiex, Resende, & Ribeiro, (2002), Pardalos (2005).

8. Method for optimality testing by order statistics We build a method for estimation of minimum M of the objective function using values of the function provided in optimization: Let only k+1 order statistics from the sample H to be chosen: , where .

9. Let apply linear estimators forestimation of the minimum: where . We examine a simple set (Hall (1982)): Let apply linear estimators forestimation of the minimum: where . We examine a simple set (Hall (1982)): Let apply linear estimators forestimation of the minimum: where . We examine a simple set (Hall (1982)):

10. The one side confidence interval for the minimum value of the objective function is: [ ] where , where  is a confidence level. - the parameter of extreme values distribution; n – dimension; - the parameter of homogeneity of the function f(x) (Zilinskas & Zhigliavsky (1991)).

11. Stochastic Approximation The smoothing is the standard way for the nondifferentiable optimization. We consider a functionsmoothed by Lipshitz perturbationoperator: where is the value of the perturbation parameter, • is a randomvector distributed with density p(.). If density p(.) is locally Lipshitz then functions smoothed by this operator are twice continuously differentiable (Rubinstein & Shapiro (1993), Bartkute & Sakalauskas (2004)).

12. where stochastic gradient, is a scalar multiplier. This scheme is the same for different Stochastic Approximationalgorithms whose distinguish only by approach to stochastic gradient estimation.The minimizing sequence converges a.s. to solution of the optimization problem under conditions typical for SA algorithms (Ermolyev (1976),Mikhalevitch et at (1987), Spall (1992), Bartkute & Sakalauskas (2004)).

13. -smoothing parameter.

14. Rate of Convergence Let consider that the function f(x)has a sharp minimum in the point , in which the algorithm converges when Then where A>0, H>0, K>0 are certain constants, is minimum point of the smoothed function (Sakalauskas, Bartkute (2007)).

15. Experimental results Unimodal testing functions (SPSAL, SPSAU, FDSA) • Generated funkcions with sharp minimum- • CB3- • Rozen Suzuki- Multiextremal testing functions (Simulated Annealing (SA)) • Branin- • Beale- • Rastrigin-

16. The samples of T=500 test functions were generated, when and minimized by SPSA with Lipshitz perturbation. The coefficients of the optimizing sequence were chosen according to convergence conditions (Bartkute & Sakalauskas (2006)):

17. Testing hypothesis about Pareto distribution If order statistics follows from Weibull distribution, then distributed with respect to Pareto distribution (Žilinskas, Zhigljavsky (1991)): Thus, statistical hypothesis tested: . H0:

18. Testing hypothesis about Pareto distribution The hypothesis tested by criteria 2 ( ) for various stochastic algorithms (critical value 0,46)

19. V. Bartkute, L. Sakalauskas. Application of Order Statistics to termination of Stochastic lgorithms. CWU Workshop, December 10-12, 2007 One side confidence interval [ ], =0.95

20. Confidence bounds of the minimum

21. Confidence bounds of the hitting probability

22. Termination criterion of the algorithms To stop the algorithm when minima confidence interval becomes less admissible value :

23. Number of iterations after the termination of the algorithm

24. Simulated Annealing Algorithm I.Choose temperature updating function neighborhood size function solution generation density function and initial solution x0(Yang (2000)). II. Construct the optimizing sequence:

25. Experimental results Let consider results ofoptimality testing withBeale testing function: F(x,y)= (1.5-x+xy)2 + (2.25-x+xy2)2 + (2.625-x+xy3)2, where search domain: -4.5 ≤x,y≤ 4.5. It is known that this function has few local minima and global minimum is 0 at the point (3; 0.5).

26. Confidence bounds of the minimum

27. Confidence bounds of the hitting probability

28. Number of iterations after the termination of the algorithm

29. Conclusions • Linear estimator for minimum has been proposed using theory of order statistics, which was studied by experimental way; • Developed procedures are simple and depend only on theparameter of extreme values distribution ; • Parameter is easily estimated using a homogeneity of the objective function or by statistical way; • Theoretical considerations and computer examples have shown that we can estimate the confidence interval of a function extremum with an admissible accuracy, when the number of iterations increased; • Termination rule using the minimum confidence interval was proposed and implemented to Stochastic Approximation and Simulated Annealing.