Constructive Algorithms for Discrepancy Minimization

1 / 26

# Constructive Algorithms for Discrepancy Minimization - PowerPoint PPT Presentation

Constructive Algorithms for Discrepancy Minimization. Nikhil Bansal (IBM). S 3. S 4. S 1. S 2. Combinatorial Discrepancy. Universe: U= [1,…,n] Subsets: S 1 ,S 2 ,…,S m Color elements red/blue so each set is colored as evenly as possible. Applications.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Constructive Algorithms for Discrepancy Minimization' - keene

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Constructive Algorithms for Discrepancy Minimization

Nikhil Bansal (IBM)

S3

S4

S1

S2

Combinatorial Discrepancy

Universe: U= [1,…,n]

Subsets: S1,S2,…,Sm

Color elements red/blue so each

set is colored as evenly as possible.

Applications

CS: Computational Geometry, Comb. Optimization, Monte-Carlo simulation, Machine learning, Complexity, Pseudo-Randomness, …

Math: Dynamical Systems, Combinatorics, Mathematical Finance,

Number Theory, Ramsey Theory, Algebra, Measure Theory, …

General Set System

Universe: U= [1,…,n]

Subsets: S1,S2,…,Sm

Find : [n] ! {-1,+1} to

Minimize |(S)|1 = maxS | i 2 S(i) |

For simplicity consider m=n henceforth.

Best Known Algorithm

Random: Color each element i independently as

x(i) = +1 or -1 with probability ½ each.

Thm: Discrepancy = O (n log n)1/2

Pf: For each set, expect O(n1/2) discrepancy

Standard tail bounds: Pr[ | i 2 S x(i) | ¸c n1/2 ] ¼e-c2

Union bound + Choose c ¼ (log n)1/2

Analysis tight: Random actually incurs ((n log n)1/2).

Better Colorings Exist!

[Spencer 85]: (Six standard deviations suffice)

Always exists coloring with discrepancy ·6n1/2

(In general for arbitrary m, discrepancy = O(n1/2log(m/n)1/2)

Tight: For m=n, cannot beat 0.5 n1/2 (Hadamard Matrix, “orthogonal” sets)

Inherently non-constructive proof

(pigeonhole principle on exponentially large universe)

Challenge: Can we find it algorithmically ?

Certain algorithms do not work [Spencer]

Conjecture[Alon-Spencer]: May not be possible.

1 2 … n

1’ 2’ … n’

S1

S2

S’1

S’2

Approximating Discrepancy

Question: If a set system has low discrepancy (say << n1/2)

Can we find a good discrepancy coloring ?

[Charikar, Newman, Nikolov 11]:

Even 0 vs. O (n1/2) is NP-Hard

(Matousek): What if system has low Hereditary discrepancy?

herdisc (U,S) = maxU’ ½ U disc (U’, S|U’)

Robust measure of discrepancy

Widely used: TU set systems, Geometry, …

Our Results

Thm 1: Can get Spencer’s bound constructively.

That is, O(n1/2) discrepancy for m=n sets.

Thm 2: For any set system, can find

Discrepancy ·O(log (mn))Hereditary discrepancy.

General Technique: Constructive bounds for geometric problems, Beck Fiala setting, k-permutation problem, …

Relaxations: LPs and SDPs

Not clear how to use.

Linear Program is useless. Can color each element ½ red and ½ blue. Discrepancy of each set = 0!

SDPs(vector coloring)

| i 2 S vi |2· n 8 S

|vi|2 = 1

Intended solution vi = (+1,0,…,0) or (-1,0,…,0).

Trivially feasible: vi = ei (all vi’s orthogonal)

Yet, SDPs will be the major tool.

Key Point

The SDP gap example does not work if discrepancy << n1/2

As we will see, SDPs are very useful in that regime.

But seems useless for Spencer’s problem

Idea: “Tighter” bounds for some sets.

|i 2 S vi |2· 2 n

| i 2 S’ vi|2· n/log n

|vi|2 = 1

Why can one do this: Entropy Method.

Tighter bound for S’

Talk Outline

Introduction

The Method

Low Hereditary discrepancy -> Good coloring

Spencer’s O(n1/2) bound

start

finish

Algorithm (at high level)

Each dimension: An Element

Each vertex: A Coloring

Cube: {-1,+1}n

Algorithm: “Sticky” random walk

Each step generated by rounding a suitable SDP Move in various dimensions correlated, e.g. t1 + t2¼ 0

Analysis: Few steps to reach a vertex (walk has high variance)

Disc(Si) does a random walk (with low variance)

An SDP

Hereditary disc. ) the following SDP is feasible

SDP:

Low discrepancy: |i 2 Sj vi |2 ·2

|vi|2 = 1

Obtain vi2 Rn

Rounding:

Pick random Gaussian g = (g1,g2,…,gn)

each coordinate gi is iid N(0,1)

For each i, consider i = g¢ vi

Properties of Rounding

Lemma: If g 2 Rn is random Gaussian. For any v 2 Rn,

g ¢ v is distributed as N(0, |v|2)

Pf: N(0,a2) + N(0,b2) = N(0,a2+b2) g¢ v = i v(i) gi» N(0, i v(i)2)

Recall: i = g ¢ vi

• Each i» N(0,1)
• For each set S,

i 2 Si = g ¢ (i2 S vi) » N(0, ·2)

(std deviation ·)

SDP:

|vi|2 = 1

|i2S vi|2·2

’s mimics a low discrepancy coloring (but is not {-1,+1})

+1

time

-1

Algorithm Overview

Construct coloring iteratively.

Initially: Start with coloring x0 = (0,0,0, …,0) at t = 0.

At Time t: Update coloring as xt = xt-1 +  (t1,…,tn)

( tiny: 1/n suffices)

xt(i) = (1i + 2i + … + ti)

Color of element i: Does random walk

over time with step size ¼ N(0,1)

x(i)

Fixed if reaches -1 or +1.

Set S: xt(S) = i 2 S xt(i) does a random walk w/ step N(0,·2)

Analysis

Consider time T = O(1/2)

Claim 1: With prob. ½, at least n/2 elements reach -1 or +1.

Pf: Each element doing random walk with size ¼.

Recall: Random walk with step 1, is ¼ O(t1/2) away in t steps.

A Trouble: Walks for various elements are correlated

Consider basic walk x(t+1) = x(t) 1 with prob ½

Define Energy (t) = x(t)2 if never reached n1/2

= (t-1) + 1 otherwise

E[(t)] = ½ (x(t-1)+1)2 + ½ (x(t-1)-1)2 = x(t-1)2 + 1 = (t-1)+1

So, E[(10 n) ] = 10 n. Moreover, (10 n) · 11 n

So only a small fraction of walks can have  < n.

Analysis

Consider time T = O(1/2)

Claim 2: Each set has O() discrepancy in expectation.

Pf: For each S, xt(S) doing random walk with step size ¼

Define a round as T = O(1/2) time steps.

Claim 1:) Everything colored in O(log n) rounds.

Claim 2: ) Expected discrepancy of a set at end = O( log n)

+ By Chernoff Bounds, discrepancy = O( log mn) whp over all sets.

Recap

At each step of walk, formulate SDP on unfixed variables.

Use some (existential) property to argue SDP is feasible

Rounding SDP solution -> Step of walk

Properties of walk:

High Variance -> Quick convergence

Low variance for discrepancy on sets -> Low discrepancy

Refinements

Spencer’s six std deviations result:

Goal: Obtain O(n1/2) discrepancy for any set system on m = O(n) sets.

Random coloring has n1/2(log n)1/2 discrepancy

Previous approach seems useless:

Expected discrepancy for a set O(n1/2),

but some random walks will deviate by up to (log n)1/2 factor

Need an additional idea to prevent this.

Spencer’s O(n1/2) result

Partial Coloring Lemma: For any system with m sets, there exists a coloring on ¸ n/2 elements with discrepancy O(n1/2 log1/2 (2m/n))

[For m=n, disc = O(n1/2)]

Algorithm for total coloring:

Repeatedly apply partial coloring lemma

Total discrepancy

O( n1/2 log1/2 2 ) [Phase 1]

+ O( (n/2)1/2 log1/2 4 ) [Phase 2]

+ O((n/4)1/2 log1/2 8 ) [Phase 3]

+ … = O(n1/2)

X1 = ( 1,-1, 1 , …,1,-1,-1)

X2 = (-1,-1,-1, …,1, 1, 1)

X = ( 1, 0, 1 , …,0,-1,-1)

Proving Partial Coloring Lemma

Beautiful Counting argument (entropy method + pigeonhole)

Idea: Too many colorings (2n), but few “discrepancy profiles”

Key Lemma: There exist k=24n/5 colorings X1,…,Xk such that

every two Xi, Xj are “similar” for every set S1,…,Sn.

Some X1,X2 differ on ¸ n/2 positions

Consider X = (X1 – X2)/2

Pf: X(S) = (X1(S) – X2(S))/2 2 [-10 n1/2 , 10 n1/2]

A useful generalization

There exists a partial coloring with non-uniform discrepancy bounds S for set S

Provided S = ( n1/2) in some average sense

An SDP

Suppose there exists partial coloring X:

1. On ¸ n/2 elements

2. Each set S has |X(S)| ·S

SDP:

Low discrepancy: |i 2 Sj vi |2·S2

Many colors:i |vi|2¸ n/2

|vi|2· 1

Pick random Gaussian g = (g1,g2,…,gn)

each coordinate gi is iid N(0,1)

For each i, consider i = g ¢ vi

Obtain vi2 Rn

Algorithm

Initially write SDP with S = c n1/2

Each set S does random walk and expects to reach

discrepancy of O(DS) = O(n1/2)

Some sets will become problematic.

Reduce their S on the fly.

Not many problematic sets, and entropy penalty low.

Danger 3 …

Danger 1

Danger 2

35n1/2

0

30n1/2

20n1/2

Concluding Remarks

Probable right answer: O( (log m)1/2) for her. Disc. 

Can be derandomized[Bansal-Spencer] (add new constraints to SDP)

Gives derandomization for prob. inequalities stronger than Chernoff bounds.

(Exponential moment technique for derandomizing Chernoff loses (log n)1/2)

Other non-constructive problems:

Lattices (Minkowski Thm)

Fixed-Point Based (Nash, Sperner’s Lemma)

Topological (Hypergraph matching), …

Discrepancy problems:

Beck Fiala Conjecture (more generally Komlos conjecture)

Erdos Discrepancy problem

3-permutation conjecture, …