1 / 12

# Online convex optimization Gradient descent without a gradient - PowerPoint PPT Presentation

Online convex optimization Gradient descent without a gradient. Abie Flaxman CMU Adam Tauman Kalai TTI Brendan McMahan CMU. Goal: find x f(x) ¸ max z 2 S f(z) – . High-dimensional. } . Standard convex optimization. Convex feasible set S ½ < d Concave function f : S ! <.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Abie Flaxman CMU

Brendan McMahan CMU

Goal: find x

f(x) ¸ maxz2Sf(z) – 

High-dimensional

} 

Standard convex optimization

Convex feasible set S ½<d

Concave function f : S !<

Steepest ascent
• Move in the direction of steepest ascent
• Compute f’(x) (rf(x) in higher dimensions)
• Works for convex optimization
• (and many other problems)

x1

x2

x3

x4

Typical application
• Toyota produces certain numbers of cars per month
• Vector x 2<d (#Corollas, #Camrys, …)
• Profit of Toyota is concave function of production vector
• Maximize total (eq. average) profit

PROBLEMS

Problem definition and results
• Sequence of unknown (concave) functions
• On tth month, choose production xt2 S ½<d
• Maximize avg. profit
• No assumption on distribution of ft

8 z, z’ 2 S

|| z – z’ || · D

||rf(z)|| · G

First try

Zinkevich ’03:

If we could only compute gradients…

f4(x4)

f3(x3)

f2(x2)

f4

PROFIT

f1(x1)

f3

f2

f1

x4

x3

x2

x1

#CAMRYS

With probability ½, estimate = f(x + )/

With probability ½, estimate = –f(x – )/

PROFIT

E[ estimate ] ¼ f’(x)

x

x-

x+

#CAMRYS

For online optimization, use Zinkevich’s

analysis of online gradient ascent [Z03]

Analysis

PROFIT

x-

x+

#CAMRYS

d-dimensional online algorithm
• Choose u 2<d, ||u||=1
• Choose xt+1 = xt +  u ft(xt+u)/
• Repeat

x3

x4

x1

x2

S

Hidden complication…

Dealing with steps outside set is difficult!

S

Hidden complication…

Dealing with steps outside set is difficult!

S’

…reshape into

Isotropic position

Related work & conclusions
• Related work
• Online convex gradient descent [Zinkevich03]
• One point gradient estimates [Spall97,Granichin89]
• Same exact problem [Kleinberg04] (different solution)
• Conclusions
• Can estimate gradient of a function from single evaluation (using randomness)