OR II GSLM 52800

1. OR IIGSLM 52800 1

2. Outline • separable programming • quadratic programming 2

3. Separable Programs • a separable NLP if • f and all gjare separable functions • 0  xi i, a finite number 3

4. Idea of Separable Program • min f(x), • s.t. • gj(x)  0 for j = 1, …, m. • hard NLP but simple LP problems • approximating a separable NL program by a LP • a non-linear function by a piecewise linear one 4

5. A Fact About Convex Functions • f: a convex function • for any  > 0, possible to find a sequence of piecewise linear convex functions fnsuch that |f  fn|   5

6. Example 6.1 • a separable program 6

7. 20.25 C 16 B 9 4 A 1 O 3 4 1 2 Example 6.1 • approximating by a piecewise linear function • two representations, -form and -form 7

8. 20.25 C 16 B 9 4 A 1 O 3 4 1 2 Form • a piecewise linear function with (segment) break points • any point = the convex combination of the two break points of the linear segment • i ( 0) = the weight of break point i 8

9. Example 6.1 • the program becomes the last but one type of constraints is non-linear 9

10. Fact • nonlinear constraint: at most two adjacent itaking non-zero values • possible to have only one i = 1 • for convex f and gj: no need to have the non-linear constraint • non-optimal to have more than two non-zero i, or two i not adjacent 10

11. Fact • non-optimal to have more than two non-zero i, or two i not adjacent • e.g., f being an objective function • any convex combination between two non-adjacent break points being above the piecewise non-linear function • similarly, the point for three or more non-zero I’s lying above the piecewise non-linear function • think about A = 0.3, B = 0.4, and C = 0.3 20.25 C 16 B 9 4 A 1 O 11 3 4 1 2

12. Fact • non-optimal to have more than two non-zero i, or two i not adjacent • e.g., gj being a constraint • gj(0.3A+0.7B)  gj(0.3A+0.7C)  bj the feasible set of {0.3A+0.7B} is larger than that by {0.3A+0.7C}  the solution from {0.3A+0.7C} cannot be minimum • similar argument for three or more non-zero i’s lying above the piecewise non-linear function C B 12 A O

13. Example 6.1 • the program becomes a linear program 13

14. Example 6.2: Non-Convex Problem • min f(x), • s.t. 1 x 3. • approximating f(x) by a piecewise linear function • y = 0 + 10A + 6B • x = 0 + 2A + 3B 14

15. Example 6.2: Non-Convex Problem • adding slack variable s, surplus variable u, and artificial variable a1 and a2: 15

16. Example 6.2: Non-Convex Problem 16

17. Example 6.2: Non-Convex Problem 17

18. Example 6.2: Non-Convex Problem • most negative 0 • B in basis  only A qualified to enter, not O 18

19. Example 6.2: Non-Convex Problem 19

20. Example 6.2: Non-Convex Problem 20

21. Example 6.2: Non-Convex Problem 21

22. 20.25 C 16 B 9 4 A 1 O 3 4 1 2 Form • again, the last constraint is unnecessary for a convex program 22

23. Quadratic Programming 23

24. Quadratic Objective Function & Linear Constraints • Langrangian function 24

25. KKT Conditions • positive definite Q • a convex program • a unique global minimum • the KKT sufficient • otherwise, KKT necessary 25

26. KKT Conditions • cT + xTQ + TA  0  Qx + A y = c • Ax  b  0  Ax + v = b • xT(c + Qx + A) = 0  xTy = 0 • T(Ax  b) = 0  Tv = 0 • x  0,  0, y  0, v  0 • solving the set of equations  phase-1 of a linear program 26

27. Example 7.1(Example 10.14 of JB) 27

28. Example 7.1(Example 10.14 of JB) • KKT conditions • 2x1 + 1 + 2y1 = 8, • 8x2+ 1y2 = 16, • x1+ x2 + v1 = 5, • x1 + v2 = 3. • x1y1 = x2y2 = 1v1 = 2v2 = 0 • x1, y1, x2, y2, 1, v1, 2, v2 0 28

29. Example 7.1(Example 10.14 of JB) 29

30. Example 7.1(Example 10.14 of JB) 30