Error Detection and Correction in Coding Theory: Algorithms and Sublinear Time Techniques

Local Error-Detection and Error-correction Madhu Sudan MIT Coding & Sublinear time

k k k k k ( ) ( ( ( ) ) ) d d d d d d d f f 9 9 h ± G F G E G C § C § § § § I E E § C § § § § § E E i i i i n n n n i i i i i n i n i i µ · t t t t t t 2 2 2 2 2 2 x v ² e - n o v v : x e e n n e x x a n c o ; a m s m s p e e o u a c c c g e e a e e m e m m : a s s m ; n x m z e m s m x ² ! = ! = , , , . . . . . ; . . k ( ( ( ) ) ) ( = ( ) ( ( ( ) ) ) ) ± k ± d d ± l d d G E R C § C E E i i i i · t t 2 v e n m m x p r n o c v o m e p u e n o r m m m a x z e ² s a n c e = = ; , , ; . . . Algorithmic Problems in Coding Theory • Code: • Encoding: • Error-detection ( Testing): • Error-correction (Decoding): Coding & Sublinear time

l x - o r a c e 0 ( ) f k x ( ( f ) ) g f g i f f k n i j 0 1 0 1 x o x : x n ! j ; ; ; 0 ( ) h f ¼ w e r e x x x i f Sublinear time algorithmics • Given can it be “computed” in time? • Answer 1: Clearly NO, since that is the time it takes to even read the input/write the output • Answer 2: YES, if we are willing to • Present input implicitly (by an oracle). • Represent output implicitly • Compute function on approximation to input. Extends to computing relations as well. Coding & Sublinear time

Sub-linear time algorithms • Initiated in late eighties in context of • Program checking • Interactive Proofs/PCPs • Now successful in many more contexts • Property testing/Graph-theoretic algorithms • Sorting/Searching • Statistics/Entropy computations • (High-dim.) Computational geometry • Many initial results are coding-theoretic! Coding & Sublinear time

Sub-linear time algorithms & Coding • Encoding: Not reasonable to expect in sub-linear time. • Testing? Decoding? – Can be done in sublinear time. • In fact many initial results do so! • Codes that admit efficient … • … testing: Locally Testable Codes (LTCs) • … decoding: Locally Decodable Codes (LDCs). Coding & Sublinear time

Rest of this talk • Definitions of LDCs and LTCs • Quick description of known results • Some basic constructions • (Time permitting) Yekhanin’s construction of LDCs. Coding & Sublinear time

Definitions Coding & Sublinear time

n k ( ) ( ( ) ( ) = ) h f d ± ? d h l l d d f b l W C D § § C M L D i n i i i i i 2 t t t t > : a r e a ² s q n s r q a n ² - o m o c g p a o s y n e o e n e c s o o o a w e ! w ; [ ] = f 9 d d l f ` l d k d D D i i i i 2 3 t t t t t t t t 2 r a e n p e o c r o o u a e p r u s s s o m u w g p v p o e n a c o e e a w s o r s i . . . . . . ( ( ) ) ( ) = d l 9 ± ± C C 2 · · t a n o r a c e w s m w m ² . . ; , Locally Decodable Code Code: Coding & Sublinear time

n ( ( ( ) ) ) ( ) ` ` l l d l l d l d d b d l d f 8 b l f 9 f d C D C § D D i i i i i i i i n i j t t t t 2 s q s ² ² r - e a - o c s s a - q y n e c o s r a - a n e c e o o m a p w o e s o n s o e c o w e r s w ; ; ; ; , . . [ ] ( ) [ ] ( ) = # d d k d d ` ± d l l ` C i i i j 2 3 · t t t t t t t t 2 2 2 g v c e a o n n e w o o u r p s u a c n s m s w a p n w a o c r a e c a s e ² w s s a m o s j i . . . . ; . . . . ( ( ) ) l l f ± C i i · t m m a r e a m e s s a g e s s a s y n g w m ² ` j 1 ; : : : ; ; Locally List-Decodable Code Code: Coding & Sublinear time

History of definitions • Constructions predate formal definitions • [Goldreich-Levin ’89]. • [Beaver-Feigenbaum ’90, Lipton ’91]. • [Blum-Luby-Rubinfeld ’90]. • Hints at definition (in particular, interpretation in the context of error-correcting codes): [Babai-Fortnow-Levin-Szegedy ’91]. • Formal definitions • [S.-Trevisan-Vadhan ’99] (local list-decoding). • [Katz-Trevisan ’00] Coding & Sublinear time

n ( ( ) ) d d l l b l C T § L T n i i i µ t t r e a s q s n q r ² a n - o o c m a p y o s e s o n a s : e w ; f 9 f T I T C i 1 t t t 2 ² e s w e r s a c c e p s w p . . . . . = f f f h I C i j 1 2 ¸ t t ² w s ² - a r r o m e n r e e c s w p , . . .  Locally Testable Codes Code: “Weak” definition: hinted at in [BFLS], explicit in [RS’96, Arora’94, Spielman’94, FS’95]. Coding & Sublinear time

n ( ( ) ) d d l l b l T C § L T n i i i µ t t r e a s q s n q r ² a n - o o c m a p y o s e s o n a s : e w ; f 9 f T I T C i 1 t t t 2 ² e s w e r s a c c e p s w p . . . . . F § n 2 ² o r e v e r y w , ( ( ) ) ± T C j ¸ t r e e c s w p w . . ; . Strong Locally Testable Codes Code: “Strong” Definition: [Goldreich-S. ’02] Coding & Sublinear time

Motivations Coding & Sublinear time

N ( ( ) ) l l d d f h f L A C i i i i i t t t t t t t t t l l l d d b l d f S C § N i n 2 µ f g ¼ ² ² b d f o c e a C r n a e c e o n n e g r p r e a c a o n n : c o m o p m u p u e c e x c x o r w e v e o r y u § x r e - i i n 0 1 ) t ² u p p o s e s o c a y - e c o a e c o e o r 2 = ² c c a n e v e w e a s u n c o n c : ! , . ; . ( 0 ) 0 [ ] l d f d l L I H R B F P i i i i ( t t t t t t h l l l d d b f h d d F i t t t o v e n a e c n a g n x c o m p e u a e s c x o n o s r a m n c o e s x n g e a e s a v e r r v a a g e e - u r e r a s s u m e c a n o c a y e c o e s o e c o e w o r . . , , [ ] [ ] f l l I R C G K S L S T V i i i i ) t t t t t t d b f h j i t t t t c n a s o e r c m o a m p o n e x e y r o e v w a o r s - c a s e p o n a n n o u s s o e m e s s a g e . . , . Motivations for Local decoding Coding & Sublinear time

Motivation for Local-testing • No generic applications known. • However, • Interesting phenomenon on its own. • Intangible connection to Probabilistically Checkable Proofs (PCPs). • Potentially good approach to understanding limitations of PCPs (though all resulting work has led to improvements). Coding & Sublinear time

Contrast between decoding and testing • Decoding: Property of words near codewords. • Testing: Property of words far from code. • Decoding: • Motivations happy with n = quasi-poly(k), and q = poly log n. • Lower bounds show q = O(1) and n = nearly-linear(k) impossible. • Testing: Better tradeoffs possible! Likely more useful in practice. • Even conceivable: n = O(k) with q = O(1)? Coding & Sublinear time

Some LDCs and LTCs Coding & Sublinear time

( = ) j j = j j k l ± f f d l l ± f l l F F F F F P P m i i m m i m i ¸ t t t t t t ¼ e c o v a e u a c e m o n n s s o o n e g o n a m - v o a r a e p o y n o m a = , , , . . ¯ ¯ l d F i t o v e r n e e P Codes via Multivariate Polynomials Message: Encoding: Parameters: (Reed Muller code) Coding & Sublinear time

k f f b l h d l h f d f f d d V P i i i i i i i i i t t t t t t t t t t t t t m e - c r v a y r s u a e s r p e p a s o c r e y c n o e r m o u a o g s o p p a o c e e n g r s e x e o o r e e n g s r e e r r e e c s e o . , 0 ( ) d d d d l ± b b l l S i i i i t < m a n a m e m e c c o - o m p e m o e x n a s y u n s e p a s c u e s p a c e s p o y n o m a . . . f d t o e g r e e 0 j j ( ) . l l F Q T i i m t u e r y c o m p e x y q ; m e p o y q = = . 0 b l ! i ¿ m m s u n e a r ) Basic insight to locality • Local Decoding: • Local Testing: Coding & Sublinear time

( ) k l k c ¢ o g = . = ( ) = ( ) l 1 1 1 ¡ ( ) ( ) ( ) l l l l d d b l b b h l l h h h k d k k L L L L T D D i i i i i i i i i i i q p o y q ² 3 t t t t t t t t t o o o o c c c c a a a a y e e e s q c c o o a w a a y n w y y w w e x p n q e x a p n n e x p = = = = ~ ( ) ( ) d k O O 1 q a n n = = Summary of Constructions • Polynomial Codes: (Locally decodable and testable) • Polynomial Codes + Composition/Concatenation: • Codes based on “Algebraic Designs” [Yekhanin] Coding & Sublinear time

[Yekhanin ’07]’s LDCs Coding & Sublinear time

( ( ( ( ) ) ) ) ( ) l l l l l k T L L S S £ i y m m a a p r r a a g g c e e a a n s w e r m = j ( j j j ) l f d d k f b ? ? S H S S A S T T T T i t \ \ o s w a a u r g n e e o c v c e a o n n n o m e k k i i i j 1 1 ; : : : ; , ; : : . . : ; . f g S T 1 µ m i i ; ; : : : ; . Recall: Combinatorial Designs • Families of Sets: • Restrictions on Intersections: • E.g., • Basic Question: i vs. i: i vs. j: Coding & Sublinear time

j j ¡ ¢ S 1 ¡ ( ) m l d l d ! B A S i i i p i t t p p a s s m m c a o - p s - e p m s m r e g s m n g e n » h h h h i i i i l 6 6 k b ? H S 0 0 0 0 1 ; ¡ 2 3 u p u u u u o w v v v v u a r v g e c a n v e = = = k k i i i i i i j j 1 1 ; : : : ; ; ; ; ; , ; : : : ; . h ? C i i t a n w e a c e v e F m 2 u v i i ; . p [Yekhanin]’s Algebraic Designs • Families of Vectors: • Restrictions on Inner Products: • Basic Question: Coding & Sublinear time

= 1 1 1 ¡ ¡ ( ) ( ) ( ) k l 9 b d k f h k ¤ F F L L S A B S S p i i i i i p i m 1 2 · µ t t t q e e p m m m m m a a - : : g q e a s r c a q p c n - p n e c s e e g n x n e p s w p s o s v e c o r s n = ) = = ; ; . . . p p ( ) ( ) d b h k k b b F S L D C i i i i i i i m m t t t t t p p - q - u e e r s y g n w n a r y v e c o s r s m a n p p n g - s o p s ) ; p k b b L D C i i i m t t t q - q u e r y s m a p p n g s o p s ) . [Y’07] Algebraic designs and LDCs (Matches some of the early constructions) Coding & Sublinear time

( ( ) ( ) ) ¯ f l 9 b l l b d ¯ l f l ¤ f F D L O S S S S i i i i i i i i i i i 2 · µ t t t t t q e e p m n e n m o a - o n a w : : g o e q e q r s u a q q c v - a a p n e g c n e e n r e a s e s c p a n o s y o n n s c e = ; ; . . . p ( ) ( ) [ ] = ( ) 9 d l h k l h F F S i i i i m p 1 t t t ¡ 2 p a q - - s p e a s r s g e n p w o y n o m v e a c o r s x n x x s 2 ; . . p ¯ f ( ) j g d l d b h ¯ k b b l L D C S i i i i i i i m t t t t t 2 e a q - q g u e e n r e y r a e s y m a p p x n g s o s n p o n - r s v a ) . . [Y’07] Algebraic designs and LDCs Coding & Sublinear time

= 7 1 7 ( ) ( ) ( ) f ( g ) l l l l b 9 b d l l k ! b f ¤ k b F S E L S L S D C S S S i i i i i i i i i i 3 3 2 1 2 7 1 2 4 8 1 6 3 2 6 4 · µ t t t t t m q e x p m a s m - m o q n - p u a a g - e e g r : a : y e p p g r e q a r - c a a e q c m s y ; p n a g n n p c p s e c n e e n e x g s s s p o s s o e x p s ) = = = ; ; ; ; ; . . ; ; ; . ; p ( ) d h k F S i i i m t t p - e s g n w v e c o r s n ; p k b b L D C i i i m t t t q - q u e r y s m a p p n g s o p s ) . [Y’07] Algebraic designs and LDCs Coding & Sublinear time

1 t t ¡ ( ) ( ) f g h l 9 9 b l l f b ¤ f F F T L L L S S S L D C S S i i i i i 2 3 4 3 2 1 1 2 4 2 · µ t t t ¡ q e e e p m m m e o m m m r e a a a m - a : : : : g p e q m - r q a u u q c e r p p n y c c e a n e s v s e p o s s u g r o u p o = = ) = p ; ; . . ; ; ; . : : : ; p j j S 0 0 0 0 0 0 0 1 ( ) ( ) ( ) 9 l d b k d b l l h k f l k h b F S S i i i i i i i i i i m 3 t t t t t t : m s p a p - a p - p g n e g e s r g a - n c e a w s s g y n o n o e c x e v p e e n c g o r s n m s ) » ; ; . . . p k b b L D C i i i m t t t q - q u e r y s m a p p n g s o p s ) . [Y’07] Algebraic designs and LDCs Coding & Sublinear time

Proofs? • Disclaimer: Proof of Lemma 2, Lemma 3 too long to fit here. (Many context switches, but elementary.) • Will only attempt to show Lemmas 1 and 4. Coding & Sublinear time

G i v e n u u ; v v k k 1 1 ; : : : ; ; : : : ; 2 3 u x i . . . 6 7 G 1 ¢ ¢ ¢ ¢ ¢ ¢ = h i 6 7 u x i ; 4 5 . . . Basic designs and LDCs Coding & Sublinear time

( ) 2 1 + + + ¡ y m y y v p v y m v k i i i 1 ; : : : ; h i 6 0 t s u y = i . . ; 2 3 . . . 6 7 1 ¢ ¢ ¢ ¢ ¢ ¢ h i 6 7 u x i ; 4 5 . . . Basic designs and LDCs message … codeword … Report parity of locations Coding & Sublinear time

h h i t t o e r r o w r o w s l l 1 ¡ - - p a o n e o s n e s . . 2 3 . . 2 3 . 6 7 . ? = . 6 7 . 4 5 6 7 1 . ¢ ¢ ¢ ¢ ¢ ¢ h i . 6 7 u x i . ; 4 5 . . . Basic designs and LDCs + + + … codeword Coding & Sublinear time

h i h = j j i j j f g k h h f b h l f f l f f U T S ~ ~ i i i i i i i j i i 0 1 1 1 t t t t t t t t t t t t t ¡ ¡ \ 2 v u u a s e e v u s c c e u a a v o r v r a a c s c ; a z e s p e e r r a u s e s x o a c v c v c e v v ; e e y c e c n p o o s r r o r o o p c o s o e w m e r p s e o m e u n p v o s e $ = = = = = i i i i i i i i i j i i ; ; ; ; . ; : : : ; ; . . Proof of Lemma 4 • Construction of Basic p-designs: • Construction of (p,S)-designs for S multiplicative: Coding & Sublinear time

Conclusions • Local algorithms in error-detection/correction lead to interesting new questions. • Non-trivial progress so far. • Limits largely unknown • O(1)-query LDCs must have R(C) = 0 [Katz-Trevisan] Coding & Sublinear time

Error Detection and Correction in Coding Theory: Algorithms and Sublinear Time Techniques