the smoothed analysis of algorithms simplex methods and beyond l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Smoothed Analysis of Algorithms: Simplex Methods and Beyond PowerPoint Presentation
Download Presentation
The Smoothed Analysis of Algorithms: Simplex Methods and Beyond

Loading in 2 Seconds...

play fullscreen
1 / 45

The Smoothed Analysis of Algorithms: Simplex Methods and Beyond - PowerPoint PPT Presentation


  • 435 Views
  • Uploaded on

The Smoothed Analysis of Algorithms: Simplex Methods and Beyond. Shang-Hua Teng Boston University/Akamai. Joint work with Daniel Spielman (MIT). Outline. Why. What. Simplex Method. Numerical Analysis. Condition Numbers/Gaussian Elimination. Conjectures and Open Problems.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Smoothed Analysis of Algorithms: Simplex Methods and Beyond' - paul2


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the smoothed analysis of algorithms simplex methods and beyond

The Smoothed Analysis of Algorithms:Simplex Methods and Beyond

Shang-Hua Teng

Boston University/Akamai

Joint work with Daniel Spielman (MIT)

outline
Outline

Why

What

Simplex Method

Numerical Analysis

Condition Numbers/Gaussian Elimination

Conjectures and Open Problems

slide3

Motivation for Smoothed Analysis

Wonderful algorithms and heuristics that work well in practice, but whose performance cannot be understood through traditional analyses.

worst-case analysis:

if good, is wonderful.

But, often exponential for these heuristics

examines most contrived inputs

average-case analysis:

a very special class of inputs

may be good, but is it meaningful?

slide5

Analyses of Algorithms:

worst case

maxx T(x)

average case

Er T(r)

smoothed complexity

slide6

Instance of smoothed framework

x is Real n-vector

sr is Gaussian random vector,

variance s2

measure smoothed complexity

as function of n and s

slide7

Complexity Landscape

run time

input space

slide8

Complexity Landscape

worst case

run time

input space

slide9

average case

Complexity Landscape

worst case

run time

input space

slide11

Smoothed Complexity Landscape

run time

smoothed

complexity

input space

smoothed analysis of algorithms
Smoothed Analysis of Algorithms
  • Interpolate between Worst case and Average Case.
  • Consider neighborhood of every input instance
  • If low, have to be unlucky to find bad input instance
motivating example simplex method for linear programming
Motivating Example: Simplex Method for Linear Programming

max zT x

s.t. A x £ y

  • Worst-Case: exponential
  • Average-Case: polynomial
  • Widely used in practice
slide14

Carbs

Protein

Fat

Iron

Cost

1 slice bread

30

5

1.5

10

30¢

1 cup yogurt

10

9

2.5

0

80¢

2tsp Peanut Butter

6

8

18

6

20¢

US RDA Minimum

300

50

70

100

The Diet Problem

Minimize 30 x1 + 80 x2 + 20 x3

s.t. 30x1 + 10 x2 + 6 x3  300

5x1 + 9x2 + 8x3  50

1.5x1 + 2.5 x2 + 18 x3  70

10x1 + 6 x3  100

x1, x2, x3  0

history of linear programming
History of Linear Programming
  • Simplex Method (Dantzig, ‘47)
  • Exponential Worst-Case (Klee-Minty ‘72)
  • Avg-Case Analysis (Borgwardt ‘77, Smale ‘82, Haimovich, Adler, Megiddo, Shamir, Karp, Todd)
  • Ellipsoid Method (Khaciyan, ‘79)
  • Interior-Point Method (Karmarkar, ‘84)
  • Randomized Simplex Method (mO(d) )

(Kalai ‘92, Matousek-Sharir-Welzl ‘92)

smoothed analysis of simplex method

max zT x

s.t. A x £ y

max zT x

s.t.

G is Gaussian

Smoothed Analysis of Simplex Method

[Spielman-Teng 01]

Theorem: For all A, simplex method takes

expected time polynomial

slide22

Polar Linear Program

z

max 

zÎ ConvexHull(a1, a2, ..., am)

initial simplex
Initial Simplex

Opt Simplex

count pairs in different facets

[

]

Different

Facets

< c/N

Pr

Count pairs in different facets

So, expect c Facets

intuition for smoothed analysis of simplex method
Intuition for Smoothed Analysis of Simplex Method

After perturbation, “most” corners have angle bounded away from flat

opt

start

most: some appropriate measure

angle: measure by condition number

of defining matrix

condition number at corner
Condition number at corner

Corner is given by

Condition number is

  • sensitivity of x to change in C and b
  • distance of C to singular
condition number at corner31
Condition number at corner

Corner is given by

Condition number is

connection to numerical analysis
Connection to Numerical Analysis

Measure performance of algorithms

in terms of condition number of input

Average-case framework of Smale:

1. Bound the running time of an algorithm solving

a problem in terms of its condition number.

2. Prove it is unlikely that a random problem

instance has large condition number.

connection to numerical analysis33
Connection to Numerical Analysis

Measure performance of algorithms

in terms of condition number of input

Smoothed Suggestion:

1. Bound the running time of an algorithm solving

a problem in terms of its condition number.

2’. Prove it is unlikely that a perturbed problem

instance has large condition number.

condition number

Edelman ‘88:

for standard Gaussian random matrix

Condition Number

Theorem: for Gaussian random matrix

variance centered anywhere

[Sankar-Spielman-Teng 02]

condition number35

Edelman ‘88:

for standard Gaussian random matrix

Condition Number

Theorem: for Gaussian random matrix

variance centered anywhere

(conjecture)

[Sankar-Spielman-Teng 02]

gaussian elimination
Gaussian Elimination
  • A = LU
  • Growth factor:
  • With partial pivoting, can be 2n
  • Precision needed (n )bits
  • For every A,
slide37

Condition Number and Iterative LP Solvers

Renegar defined condition number for LP

maximize subject to

  • distance of (A, b, c) to ill-posed linear program
  • related to sensitivity of x to change in (A,b,c)

Number of iterations of many LP solvers

bounded by function of condition number:

Ellipsoid, Perceptron, Interior Point, von Neumann

slide38

Smoothed Analysis of Perceptron Algorithm

[Blum-Dunagan 01]

Theorem: For perceptron algorithm

Bound through “wiggle room”,

a condition number

Note: slightly weaker than a bound on expectation

slide39

Compare: worst-case complexity of

IPM is iterations, note

Smoothed Analysis of Renegar’s Cond Number

Theorem:

[Dunagan-Spielman-Teng 02]

Corollary: smoothed complexity of interior point method is

for accuracy e

slide40

Perturbations of Structured and Sparse Problems

Structured perturbations of structured inputs

perturb

Zero-preserving perturbations of sparse inputs

perturb non-zero entries

Or, perturb discrete structure…

goals of smoothed analysis
Goals of Smoothed Analysis

Relax worst-case analysis

Maintain mathematical rigor

Provide plausible explanation for

practical behavior of algorithms

Develop a theory closer to practice

http://math.mit.edu/~spielman/SmoothedAnalysis

slide43

Geometry of

should be d1/2

(union bound)

slide44

Improving bound on

For ,

Apply to random

conjecture

Lemma:

So

slide45

Compare: worst-case complexity of

IPM is iterations, note

Smoothed Analysis of Renegar’s Cond Number

Theorem:

[Dunagan-Spielman-Teng 02]

Corollary: smoothed complexity of interior point method is

for accuracy e

conjecture