1 / 60

Which Boolean Functions Are Most Informative?

Which Boolean Functions Are Most Informative?. Gowtham R Kumar and Thomas Courtade. Stanford University, CSoI , AFOSR. Setup. i.i.d. ~ bern ( ). BSC( ). 1-. 1. 1. 0. 0. 1-. Setup. i.i.d. ~ bern ( ). BSC( ). 1-. 1. 1. 0. 0. 1-. Boolean function. Maximize over.

blondelle-kayson
Download Presentation

Which Boolean Functions Are Most Informative?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Which Boolean Functions Are Most Informative? Gowtham R Kumar and Thomas Courtade Stanford University, CSoI, AFOSR

  2. Setup i.i.d. ~bern() BSC() 1- 1 1 0 0 1-

  3. Setup i.i.d. ~bern() BSC() 1- 1 1 0 0 1- Boolean function Maximize over

  4. Guess for that maximizes . If , Can we do better?

  5. Conjecture 0 Trivially achieved by setting

  6. Loose Outer Bound Using Mrs. Gerber’s lemma, Loose outer bound.

  7. Standard Tools don’t work Conjecture 0: Standard Tools from Information Theory? New Tools? Combinatorics?

  8. Standard Tools don’t work Conjecture 0: Standard Tools from Information Theory? New Tools? Combinatorics? Weaker result:

  9. Related Results

  10. Related Results

  11. Related Results

  12. Related Results

  13. Related Results

  14. Related Results

  15. Discrete Isoperimetric Inequalities

  16. Lexicographic Ordering (Lex) defined

  17. Lexicographic Ordering (Lex) defined • A Boolean function is Lex if

  18. Lexicographic Ordering (Lex) defined • A Boolean function is Lex if • is not Lex if

  19. Discrete Hypercube defined • Vertex set: • Edge between 2 nodes if Hamming distance is 1. 110 111 100 101 010 011 000 001 Discrete Hypercube

  20. Discrete Hypercube defined • Vertex set: • Edge between 2 nodes if Hamming distance is 1. 110 111 100 101 = 0 = 1 010 011 000 001 “Volume”:

  21. Isoperimetric Inequality Of all closed surfaces with a fixed volume, the sphere has the smallest surface area.

  22. Isoperimetric Inequality Of all closed surfaces with a fixed volume, the sphere has the smallest surface area. What about edge boundary?

  23. Edge boundary illustrated • Vertex set: • Edge between 2 nodes if Hamming distance is 1. • : Edge between red (0) and blue (1). • Edge boundary is the number of edges between red and blue. 110 111 100 101 = 0 = 1 010 011 000 001 “Surface Area”: Edge boundary = 7

  24. Lex minimizes Edge boundary 110 110 111 111 100 100 101 101 010 011 010 011 000 001 000 001 Lex: Edge boundary = 5 Edge boundary = 7

  25. Lex is a Sphere Lemma 1: Of all Boolean functions with fixed, the edge boundary is minimized by the that is Lex.

  26. Conjecture 1 Conjecture 1: Of all Boolean functions with fixed, the conditional entropy is minimized by the that is Lex.

  27. Conjecture 1 Conjecture 1: Of all Boolean functions with fixed, the conditional entropy is minimized by the that is Lex. When ,

  28. Split Conjecture 0 split Conjecture 0 Maximize over Structure Inequality Conjecture 1 Maximize over Lex is optimal. Conjecture 2 For Lex,

  29. Split Conjecture 0 split Conjecture 0 Maximize over Structure Inequality Conjecture 1 Maximize over Lex is optimal. Conjecture 2 For Lex,

  30. Progress towards Conjecture 1

  31. Representing using a special tabular form. Let be a subset of indices. In the example below, , n=3. Table: Representation of an arbitrary Boolean function .

  32. Representing using a special tabular form Let be a subset of indices. In the example below, , n=3. Table: Representation of an arbitrary Boolean function . =0

  33. Compressions Illustrated Illustrating I-compression. I-compression reduces “Surface Area”! Table: Representation of an arbitrary Boolean function . =0

  34. Representing using a special tabular form Let be a subset of indices. In the example below, , n=5. Table: Representation of an arbitrary Boolean function

  35. Compressions Illustrated Illustrating I-compression. I-compression reduces “Surface Area”! Table: Representation of an arbitrary Boolean function =0

  36. -Compression reduces edge boundary. Lemma: For any , -Compression reduces the number of edges between and .

  37. -Compression reduces conditional entropy for Theorem: For any containing at most 2 elements, -Compression reduces the conditional entropy . Proof: Concavity of entropy and a little work. Unfortunately, we can find counter-examples for where I-Compression increases the conditional entropy.

  38. -Compression reduces conditional entropy for Theorem: For any containing at most 2 elements, I-Compression reduces the conditional entropy . Proof: Concavity of entropy and a little work. Unfortunately, we can find counter-examples for where I-Compression increases the conditional entropy.

  39. Can now verify Conjecture by simulation for up to . Able to reduce the number of candidate Boolean functions for which Conjectures 0,1 needs to be verified.

  40. Progress towards Conjecture 2

  41. Plot of For Lex,

  42. Defining

  43. in terms of f

  44. Stating Conjecture 2 in terms of

  45. Plot of

  46. Plot of Observation 1: For ,

  47. Plot of Observation 2 (Symmetry):

  48. Plot of Therefore, we need to prove only for .

  49. Plot of

  50. Plot of for How does vary with ?

More Related