Quadratic programming and duality
Download
1 / 16

Quadratic Programming and Duality - PowerPoint PPT Presentation


  • 335 Views
  • Uploaded on

Quadratic Programming and Duality. Sivaraman Balakrishnan. Outline. Quadratic Programs General Lagrangian Duality Lagrangian Duality in QPs. Norm approximation . Problem Interpretation Geometric – try to find projection of b into ran(A) Statistical – try to find solution to b = Ax + v

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Quadratic Programming and Duality' - thyra


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Quadratic programming and duality

Quadratic Programming and Duality

Sivaraman Balakrishnan


Outline
Outline

  • Quadratic Programs

  • General Lagrangian Duality

  • Lagrangian Duality in QPs


Norm approximation
Norm approximation

  • Problem

  • Interpretation

    • Geometric – try to find projection of b into ran(A)

    • Statistical – try to find solution to b = Ax + v

      • v is a measurement noise (choose norm so that v is small in that norm)

    • Several others


Examples
Examples

  • -- Least Squares Regression

  • -- Chebyshev

  • -- Least Median Regression

  • More generally can use *any* convex penalty function



Least norm
Least norm

  • Perfect measurements 

  • Not enough of them 

  • Heart of something known as compressed sensing

  • Related to regularized regression in the noisy case


Smooth signal reconstruction
Smooth signal reconstruction

  • S(x) is a smoothness penalty

  • Least squares penalty

    • Smooths out noise and sharp transitions

  • Total variation (peak to valley intuition)

    • Smooths out noise but preserves sharp transitions


Euclidean projection
Euclidean Projection

  • Very fundamental idea in constrained minimization

  • Efficient algorithms to project onto many many convex sets (norm balls, special polyhedra etc)

  • More generally finding minimum distance between polyhedra is a QP



General recipe
General recipe

  • Form Lagrangian

  • How to figure out signs?



Primal dual programs
Primal & Dual Programs

  • Primal Programs

  • Constraints are now implicit in the primal

  • Dual Program


Lagrangian properties
Lagrangian Properties

  • Can extract primal and dual problem

  • Dual problem is always concave

    • Proof

  • Dual problem is always a lower bound on primal

    • Proof

  • Strong duality gives complementary slackness

    • Proof


Some examples of qp duality
Some examples of QP duality

  • Consider the example from class

  • Lets try to derive dual using Lagrangian


General psd qp
General PSD QP

  • Primal

  • Dual


Svm lagrange dual
SVM – Lagrange Dual

  • Primal SVM

  • Dual SVM

  • Recovering Primal Variables and Complementary Slackness