quadratic programming and duality
Download
Skip this Video
Download Presentation
Quadratic Programming and Duality

Loading in 2 Seconds...

play fullscreen
1 / 16

Quadratic Programming and Duality - PowerPoint PPT Presentation


  • 335 Views
  • Uploaded on

Quadratic Programming and Duality. Sivaraman Balakrishnan. Outline. Quadratic Programs General Lagrangian Duality Lagrangian Duality in QPs. Norm approximation . Problem Interpretation Geometric – try to find projection of b into ran(A) Statistical – try to find solution to b = Ax + v

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Quadratic Programming and Duality' - thyra


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
quadratic programming and duality

Quadratic Programming and Duality

Sivaraman Balakrishnan

outline
Outline
  • Quadratic Programs
  • General Lagrangian Duality
  • Lagrangian Duality in QPs
norm approximation
Norm approximation
  • Problem
  • Interpretation
    • Geometric – try to find projection of b into ran(A)
    • Statistical – try to find solution to b = Ax + v
      • v is a measurement noise (choose norm so that v is small in that norm)
    • Several others
examples
Examples
  • -- Least Squares Regression
  • -- Chebyshev
  • -- Least Median Regression
  • More generally can use *any* convex penalty function
least norm
Least norm
  • Perfect measurements 
  • Not enough of them 
  • Heart of something known as compressed sensing
  • Related to regularized regression in the noisy case
smooth signal reconstruction
Smooth signal reconstruction
  • S(x) is a smoothness penalty
  • Least squares penalty
    • Smooths out noise and sharp transitions
  • Total variation (peak to valley intuition)
    • Smooths out noise but preserves sharp transitions
euclidean projection
Euclidean Projection
  • Very fundamental idea in constrained minimization
  • Efficient algorithms to project onto many many convex sets (norm balls, special polyhedra etc)
  • More generally finding minimum distance between polyhedra is a QP
general recipe
General recipe
  • Form Lagrangian
  • How to figure out signs?
primal dual programs
Primal & Dual Programs
  • Primal Programs
  • Constraints are now implicit in the primal
  • Dual Program
lagrangian properties
Lagrangian Properties
  • Can extract primal and dual problem
  • Dual problem is always concave
    • Proof
  • Dual problem is always a lower bound on primal
    • Proof
  • Strong duality gives complementary slackness
    • Proof
some examples of qp duality
Some examples of QP duality
  • Consider the example from class
  • Lets try to derive dual using Lagrangian
general psd qp
General PSD QP
  • Primal
  • Dual
svm lagrange dual
SVM – Lagrange Dual
  • Primal SVM
  • Dual SVM
  • Recovering Primal Variables and Complementary Slackness
ad