Convex Programming. Brookes Vision Reading Group. Huh?. What is convex ??? What is programming ??? What is convex programming ???. Huh?. What is convex ??? What is programming ??? What is convex programming ???. Convex Function. f(t x + (1-t) y) <= t f(x) + (1-t) f(y). Convex Function.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Convex Programming
Brookes Vision Reading Group
f(t x + (1-t) y) <= t f(x) + (1-t) f(y)
Is a linear function convex ???
Region above a convex function is a convex set.
Is the set of all positive semidefinite matrices convex??
Objective function
Example
Constraints
Optimal solution
Vertices
Objective function
Feasible region
2 out of 3 is not bad !!!
x2 + y2 <= z2
Hmmm
ICE CREAM !!
Linear Objective Function
Affine mapping of SOC
Minimize fTx
Subject to || Ai x+ bi || <= ciT x + di
i = 1, … , L
Constraints are SOC of ni dimensions
Feasible regions are intersections of conic regions
Minimize xT P0 x + 2 q0T x + r0
Subject to xT Pi x + 2 qiT x + ri
Pi >= 0
|| P01/2 x + P0-1/2 x ||2 + r0 -q0TP0-1p0
Minimize xT P0 x + 2 q0T x + r0
Subject to xT Pi x + 2 qiT x + ri
Minimize t
Subject to || P01/2 x + P0-1/2 x || < = t
|| P01/2 x + P0-1/2 x || < = (r0 -q0TP0-1p0)1/2
Minimize || Fi x + gi ||
Minimize ti
Subject to || Fi x + gi || <= ti
Special Case: L-1 norm minimization
Minimize max || Fi x + gi ||
Minimize t
Subject to || Fi x + gi || <= t
Special Case: L-inf norm minimization
x >= 0 , y >= 0
w2 <= xy
|| [2w; x-y] || <= x+y
Linear Objective Function
Linear Constraints
Minimize C X
Subject to Ai X = bi
X >= 0
Linear Programming on Semidefinite Matrices
Matrix M(z)
To find vector z* such that max is minimized.
Let max(M(z)) <= n
max(M(z)-nI) <= 0
min(nI - M(z)) >= 0
nI - M(z) >= 0
Matrix M(z)
To find vector z* such that max is minimized.
Max -n
nI - M(z) >= 0
Minimize xTQ0x + 2q0Tx + r0
Subject to xTQix + 2qiTx + ri < = 0
Qi >= 0
=> Convex
Non-Convex Quadratic Programming Problem !!!
Redefine x in homogenous coordinates.
y = (1; x)
Let’s solve this now !!!
Minimize xTQ0x + 2q0Tx + r0
Subject to xTQix + 2qiTx + ri < = 0
Minimize yTM0y
Subject to yTMiy < = 0
Mi = [ ri qiT; qi Qi]
Bad Constraint !!!!
No donut for you !!!
Minimize yTM0y
Subject to yTMiy < = 0
Minimize M0 Y
Subject to Mi Y < = 0
Y = yyT
SDP Problem
Minimize yTM0y
Subject to yTMiy < = 0
Minimize M0 Y
Subject to Mi Y < = 0
Y >= 0
Nothing left to do ….
but Pray
Note that we have squared the number of variables.
- xi = -1
- xi = +1
Alright !!! So it’s an integer programming problem !!!
Doesn’t look like quadratic programming to me !!!
Max Cut problem can be written as
Naah !! Let’s get it into the standard quadratic form.
Max Cut problem can be written as
Naah !! Let’s get it into the standard quadratic form.
To the white board.
(You didn’t think I’ll prepare slides for this, did you??)
X - xxT >= 0
Minimize yTM0y
Subject to yTMiy < = 0
Remember
Y = [1 xT; x X]
Minimize M0 Y
Subject to Mi Y < = 0
Y >= 0
Say you’re given C = { C1, C2, … Cn} such that Cj >= 0
Cj (X - xxT) >= 0
(Ux)T (Ux) <= Cj X
Wait .. Isn’t this a hyperbolic constraint
Therefore, it’s SOCP-able.
Minimize yTM0y
Subject to yTMiy < = 0
Minimize Q0 X + 2q0Tx + r0
Subject to Qi X + 2qiTx + ri < = 0
Cj (X - xxT) >= 0
Cj C
If C is the infinite set of all semidefinite matrices
SOCP Relaxation = SDP Relaxation
If C is finite,
SOCP relaxation is ‘looser’ than SDP relaxation.
Then why SOCP relaxation ???
Efficiency - Accuracy Tradeoff
Remember we had squared the number of variables.
Let’s try to reduce them with our choice of C.
For a general problem - Kim and Kojima
Using the structure of a specific problem -
e.g. Muramatsu and Suzuki for Max Cut
Minimize cT x
Subject to Qi X + 2qiTx + ri < = 0
Q X + 2qTx + r <= 0
Q = n i uiuiT
Let1 >= 2 >= …. k >=0 >= k+1 >=n
Q+ = k i uiuiT
C =
Q X + 2qTx + r <= 0
xT Q+ x - Q+ X <= 0
xT Q+ x + k+1i uiuiT X + 2qTx + r <= 0
zi
Q+ = k i uiuiT
C =
uiuiT i = k+1, k+2, … n
Q X + 2qTx + r <= 0
xT Q+ x + k+1i zi+ 2qTx + r <= 0
xTuiuiTx - uiuiT X <= 0
Q+ = k i uiuiT
C =
uiuiT i = k+1, k+2, … n
Q X + 2qTx + r <= 0
xT Q+ x + k+1i zi+ 2qTx + r <= 0
xTuiuiTx - zi <= 0
ei eiT
i = 1, … , |V|
C =
uij uijT
(i,j) E
vij vijT
(i,j) E
ei = [0 0 …. 1 0 …0]
uij = ei + ej
vij = ei - ej
Warning: Scary equations to follow.