SIMULATION_ALGORITHM_FOR_ANNEALING_group3

1. SIMULATION ALGORITHM FOR ANNEALING (Based on applied Operations Research) Group - 03

2. Outline of the Presentation Introduction • Purpose • Boltzmann Distribution • Metropolis Criterion • Applying the proposed algorithm to a selected unimodal function • Introducing to a new cooling schedule • Introducing to Benchmark functions • Checking the SA algorithm with Benchmark functions • For uni-modal function • For Multi-modal function • Limitations • References •

3. INTRODUCTION

4. Purpose : Constructing a SA algorithm according to cooling process of Metropolis criterion Simulation annealing is very useful in many practical situations. • Basically, Simulation process means finding the equilibrium. • Within the industry or when we conduct a research or experiment, we may have to decrease some function values • according to some controlling factor. Here we are trying to anneal the corresponding function according to the Boltzmann cooling process, • For that here we use Metropolis criteria & build a pseudo code • Based on this pseudo code we implemented the simulation annealing algorithm •

5. Python implementation of the Boltzmann curve

6. PSEUDOCODE FOR THE METROPOLIS CRITERION COOLING PROCESS START Set initial temperature T Compute the initial state (i) Compute random neighboring state Compute the energy difference new state (n) If ( Δ E) Δ E = E(i) – E(n) If Δ E < 0 Accept and continue with new state Else Compute the Boltzmann factor (p) Generate Uniform Random Number (r) END If r < p Else Accept the new State Do not accept the new state Here, P – Boltzmann factor = -∆? ?

7. DRY RUN OF THE ABOVE PSEUDO CODE Energy function f(x) = xi2 Initial state = x1 Current state = x2 Energy difference (ΔE) = (x2 – x1) Boltzmann factor = P Iteration 2 let , Then Current Temperature (C) = 250K x1 = 0.43 x2 = 0.70 Iteration 1 let , Then Current Temperature (C) = 300K x1 = 0.62 x2 = 0.43 ΔE = (0.70)2 - (0.43)2 = 0.3051 since ΔE > 0, Then update Calculate P, ΔE = (0.43)2 - (0.62)2 = -0.1995 P = e –ΔE/C = e–0.3051/250 = 0.9988 Generate a random number x3 = 0.32 Since P > x3 , Accept x3 since ΔE < 0, Accept x2 Then update Current state (x2) = 0.43 Initial state (x 1) x3 = 0.32 Initial state (x1)

8. APPLYING THE PROPOSED ALGORITHM

9. APPLYING THE PROPOSED ALGORITHM TO A TWO VARIABLE FUNCTION • After the dry run of the pseudo code, proposed algorithm was applied to the selected unimodal function. • Selected unimodal function : • F( x1, x2 ) = x12 + x22– 2 . x1 . x2 •With Initial Temperature = 300 K •Selected colling schedule : Initial Temperature * 0.85 •By using this cooling schedule, we can reduce the temperature by 15 % .

10. • GRAPH OF STATES VS ITERATIONS WITHIN ( -1 , 1) GRAPH OF ACCEPTED STATES VS TEMPERATURE •

11. GRAPH OF FUNCTION VALUES VS ITERATIONS

12. INTRODUCING TO NEW COOLING SCHEDULE

13. THE NEW COOLING SCHEDULE : • From Research article which, was conducted by Alex Kwaku Peprah, Simon Kojo Appiah, Samuel Kwame Amponsah, found a suitable cooling schedule to the developed SA algorithm. https://www.scirp.org/html/11-7403355_78834.htm • Research article link : https://www.scirp.org/html/11-7403355_78834.htm • THE SELECTED COOLING SCHEDULE : Tk • Tk+1 = (1+(Tk∗c)) Where , Tk+1– New temperature Tk – Initial temperature C - Cooling Rate • This cooling schedule decrease the temperature geometrically which gives us a smoother curve than a discriminant of 15% of current temperature.

14. IMPLEMENTED RESULTS As a result, we can reach optimal function value : 6.012677840061542e-07 Optimal xivalues :(-1.742625763838141, -1.7412875975887587) GRAPH OF ITERATION VS TEMPERATURE COMPARISON OF GRAPHS •

15. ITERATION VS X1 VALUES ITERATION VS X2 VALUES • •

16. GRAPH OF FUNCTION VALUES VS ITERATIONS

17. USING DOUBLE LOOPING METHOD FOR THE PROPOSED SIMULATED ANNEALING ALGORITHM • From here we apply double looping method to decrease the temperature, • Which we can iterate certain function values for a constant temperature and obtain best function value for considered temperature. • And then decrease the temperature according to the selected cooling schedule.

18. IMPLEMENTED RESULTS As a result, we can reach optimal function value : 7.649745905301264e-08 Optimal xivalues : (-0.08358883791930172, -0.08331225617902938) GRAPH OF ITERATION VS TEMPERATURE

19. ITERATION VS X1 VALUES ITERATION VS X2 VALUES • •

20. HISTOGRAM OF X1 VALUES HISTOGRAM OF X2 VALUES • •

21. GRAPH OF FUNCTION VALUES VS ITERATIONS

22. INTRODUCING TO BENCHMARK FUNCTIONS

23. WHY WE USE BENCHMARK FUNCTIONS • The test for reliability, efficiency and validation of optimization algorithms are frequently carried out by using a chosen set of common standard benchmarks or test functions. • So, we can use BM for future validation and comparison of optimization algorithms. • For any new optimization, it is essential to validate its performance and compare with other existing algorithms over a good set of test functions. • Basically, we are going to validate our SA algorithm using BM function • For that we are going to use, • Uni-Modal • Multi-Modal

24. VALIDATION OF SIMULATED ANNEALING ALGORITHM WITH MULTI-MODAL FUNCTION • Schwefel 2.36 Function [77] • (Continuous, Differentiable, Separable, Scalable, Multimodal) • f(x1, x2) = (− x1 . x2 (72 − 2x1− 2x2) ) • subject to 0 ≤ xi≤ 500. • The global minimum is located at x ∗ = f(12, , 12) • f(x ∗ ) = −3456.

25. IMPLEMENTED RESULTS Optimal solution Optimal function value Error difference : (11.974643554433936, 12.01599026024709) : -3455.9881560466615 : 0.011843953338484425 SURFACE PLOT FOR THE FUNCTION SCATTER PLOT FOR THE FUNCTION • •

26. GRAPH OF ITERATION VS TEMPERATURE • THE SELECTED COOLING SCHEDULE : Tk+1 = (1+(Tk ∗c)) Where , Tk+1– New temperature Tk – Initial temperature C - Cooling Rate Tk

27. ITERATION VS X1 VALUES ITERATION VS X2 VALUES • •

28. HISTOGRAM OF X1 VALUES HISTOGRAM OF X2 VALUES • •

29. ITERATIONS VS ∆ VALUES HISTOGRAM OF ∆ VALUES • • Markov chain length = 200 •

30. ITERATIONS VS FUNCTION VALUES ITERATIONS VS ENERGY VALUES • •

31. VALIDATION OF SAA WITH UNIMODAL FUNCTION • Rotated Ellipse 2 Function [66] • (Continuous, Differentiable, Non-Separable, Non-Scalable, Unimodal) • f(x1, x2) = x12− x1x2 + x22 • subject to −500 ≤ xi ≤ 500. • The global minimum is located at x ∗ = f(0, 0), • f(x ∗ ) = 0

32. IMPLEMENTED RESULTS Optimal solution: (-0.0744927062980878, 0.01320235313788487) Optimal function value: 0.006706944434734683 Error difference: 0.006706944434734683 SURFACE PLOT FOR THE FUNCTION • SCATTER PLOT FOR THE FUNCTION •

33. GRAPH OF ITERATION VS TEMPERATURE • THE SELECTED COOLING SCHEDULE : Tk+1 = (1+(Tk∗c)) Where , Tk+1– New temperature Tk – Initial temperature C - Cooling Rate Tk

34. ITERATION VS X1 VALUES • ITERATION VS X2 VALUES •

35. HISTOGRAM OF X1 VALUES • HISTOGRAM OF X2 VALUES •

36. ITERATIONS VS ∆ VALUES HISTOGRAM OF ∆ VALUES • •

37. ITERATIONS VS FUNCTION VALUES • ITERATIONS VS ENERGY VALUES •

38. LIMITATIONS Testing of a unfitted function for the proposed algorithm For example, consider; • Goldstein Price Function [38] • This algorithm is not fit for all modals.(According to various reasons) • (Continuous, Differentiable, Non-separable, Non Scalable, Multimodal) • f(x) = [1 + (x1 + x2 + 1)2 (19 − 14x1 +3x12− 14x2 + 6x1x2 + 3x 22 )] ×[30 + (2x1− 3x2) 2 (18 − 32x1 + 12x12 + 48x2− 36x1x2 + 27x22 )] Since we used while loop as the outer-loop(related to the double looping part) of this algorithm, it is difficult to control the temperature values. • subject to −2 ≤ xi≤ 2. The global minimum is located at x∗ = f(0, −1), f(x∗) = 3 Implemented results: Optimal solution: (-1.7086528494306141, -1.9995808255883722) Optimal function value: -593708.3231964876 Error difference: -593711.3231964876

39. REFERENCES https://www.scirp.org/html/11-7403355_78834.htm Benchmark_Functions.pdf IEEE_engine_SA.pdf Basics_of_simulated_annealing_algorithm.pdf

40. Thank you!