1 / 16

# Quadratic Programming and Duality - PowerPoint PPT Presentation

Quadratic Programming and Duality. Sivaraman Balakrishnan. Outline. Quadratic Programs General Lagrangian Duality Lagrangian Duality in QPs. Norm approximation . Problem Interpretation Geometric – try to find projection of b into ran(A) Statistical – try to find solution to b = Ax + v

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Sivaraman Balakrishnan

• General Lagrangian Duality

• Lagrangian Duality in QPs

• Problem

• Interpretation

• Geometric – try to find projection of b into ran(A)

• Statistical – try to find solution to b = Ax + v

• v is a measurement noise (choose norm so that v is small in that norm)

• Several others

• -- Least Squares Regression

• -- Chebyshev

• -- Least Median Regression

• More generally can use *any* convex penalty function

• Perfect measurements 

• Not enough of them 

• Heart of something known as compressed sensing

• Related to regularized regression in the noisy case

• S(x) is a smoothness penalty

• Least squares penalty

• Smooths out noise and sharp transitions

• Total variation (peak to valley intuition)

• Smooths out noise but preserves sharp transitions

• Very fundamental idea in constrained minimization

• Efficient algorithms to project onto many many convex sets (norm balls, special polyhedra etc)

• More generally finding minimum distance between polyhedra is a QP

• Form Lagrangian

• How to figure out signs?

• Primal

• Dual

• Primal Programs

• Constraints are now implicit in the primal

• Dual Program

• Can extract primal and dual problem

• Dual problem is always concave

• Proof

• Dual problem is always a lower bound on primal

• Proof

• Strong duality gives complementary slackness

• Proof

• Consider the example from class

• Lets try to derive dual using Lagrangian

• Primal

• Dual

• Primal SVM

• Dual SVM

• Recovering Primal Variables and Complementary Slackness