1 / 16

# Quadratic Programming and Duality - PowerPoint PPT Presentation

Quadratic Programming and Duality. Sivaraman Balakrishnan. Outline. Quadratic Programs General Lagrangian Duality Lagrangian Duality in QPs. Norm approximation . Problem Interpretation Geometric – try to find projection of b into ran(A) Statistical – try to find solution to b = Ax + v

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Sivaraman Balakrishnan

Outline
• General Lagrangian Duality
• Lagrangian Duality in QPs
Norm approximation
• Problem
• Interpretation
• Geometric – try to find projection of b into ran(A)
• Statistical – try to find solution to b = Ax + v
• v is a measurement noise (choose norm so that v is small in that norm)
• Several others
Examples
• -- Least Squares Regression
• -- Chebyshev
• -- Least Median Regression
• More generally can use *any* convex penalty function
Least norm
• Perfect measurements 
• Not enough of them 
• Heart of something known as compressed sensing
• Related to regularized regression in the noisy case
Smooth signal reconstruction
• S(x) is a smoothness penalty
• Least squares penalty
• Smooths out noise and sharp transitions
• Total variation (peak to valley intuition)
• Smooths out noise but preserves sharp transitions
Euclidean Projection
• Very fundamental idea in constrained minimization
• Efficient algorithms to project onto many many convex sets (norm balls, special polyhedra etc)
• More generally finding minimum distance between polyhedra is a QP
General recipe
• Form Lagrangian
• How to figure out signs?
Primal & Dual Programs
• Primal Programs
• Constraints are now implicit in the primal
• Dual Program
Lagrangian Properties
• Can extract primal and dual problem
• Dual problem is always concave
• Proof
• Dual problem is always a lower bound on primal
• Proof
• Strong duality gives complementary slackness
• Proof
Some examples of QP duality
• Consider the example from class
• Lets try to derive dual using Lagrangian
General PSD QP
• Primal
• Dual
SVM – Lagrange Dual
• Primal SVM
• Dual SVM
• Recovering Primal Variables and Complementary Slackness