Asymptotically good binary code with efficient encoding & Justesen code

1 / 42

# Asymptotically good binary code with efficient encoding & Justesen code - PowerPoint PPT Presentation

Asymptotically good binary code with efficient encoding & Justesen code. Tomer Levinboim Error Correcting Codes Seminar (2008). Outline. Intro codes Singleton Bound Linear Codes Bounds Gilbert-Varshamov Hamming RS codes Code Concatention Examples Wozencraft Ensemble Justesen Codes.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Asymptotically good binary code with efficient encoding & Justesen code' - galvin-hatfield

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Asymptotically good binary code with efficient encoding& Justesen code

Tomer Levinboim

Error Correcting Codes Seminar (2008)

Outline
• Intro
• codes
• Singleton Bound
• Linear Codes
• Bounds
• Gilbert-Varshamov
• Hamming
• RS codes
• Code Concatention
• Examples
• Wozencraft Ensemble
• Justesen Codes
Hamming Distance
• Hamming Distance between
• The Hamming Distance is a metric
• Non negative
• Symmetric
• Triangle inequality

=

Weight
• The weight (wt) of
• Example (on board)
Code
• An (n,k,d)q code C is a function such that:
• For every
Code (parameters)
• (n,k,d)q
• Parameters
• n – block length
• k – information length
• d – minimum distance (actually, a lower bound)
• q – size of alphabet
• |C| = qk or k=logq|C|
Code (parameters div n)
• Asymptotic view of parameters as n∞:
• The rate
• Relative minimum distance
• Thus an (n,k,d)q can be written as (1,R,δ)q
• Notation: (n,k,d)q vs. [n,k,d]q – latter reserved for linear code (soon)
Trivial Code Example
• FEC3 = write each bit three time
• R = ?
• d = ?
• how many errors can we
• Detect ? (d-1)
• Correct ? t, where d=2t+1
Goal
• Would like to:
• Maximize δ – correct more

* conflicting goals - would like to be able to construct an [n,k,d]q code s.t. δ>0, R>0 and both are constant.

• Minimize q – for practical reasons
• Maximize number of codewords while minimizing n and keeping d large.
Singleton Bound
• Let C be an [n,k,d]q code then
• k ≤ n – d + 1

equivalently

• R ≤ 1 – δ + o(1)
• Proof: project C to first k-1 coordinates
• On Board
Visual intuition
• On board...
• Ballq(x,r)
• r:=d
• r:=t (where d=2t+1)
• Volq(n,r) = |Ballq(x,r)|
Linear Codes
• An [n,k,d]q code C:FqKFqn is linear when:
• Fq is a field
• C is linear function (e.g., matrix)
• Linearity implies:
• C(ax+by) = aC(x) + bC(y)
• 0n member of C
Linear Codes (example)
• FEC3
• [3,1,3]2
• Hadamard – longest linear code
• [n,logn, n/2]2
• e.g., - [8,3,4]2
• (H - Matrix representation on board)
• Dimensions
• Asymptotic behavior
Linear Codes – minimum distance
• Lemma: if C:FqKFqn is linear then

Note: for clarity Cx means C(x)

• Proof:
• ≤ - trivial
• ≥ - follows from linearity (on board)
Reed-Solomon code
• Idea: oversample a polynomial
• Let q be prime power and Fq a finite field of size q.
• Let k<n and fix n elements of Fq,
• x1,x2,..xn
• Given a message m=(c0..ck-1) interpret it has the coefficients of the polynomial p
RS Codes
• Thus (c0..ck-1) is mapped to (p(x1),..p(xn))
• Linear mapping (Vandermonde)
• Using linearity, can show for x≠0

 RS meet the Singleton bound

• Proof: on board
• (# of roots of a k-1 degree poly)
• Encoding time
Gilbert-Varshamov Bound Preliminaries
• Binary Entropy
• Stirling

Implying that:

Gilbert-Varshamov Bound Preliminaries
• Using the binary entropy we obtain
• On board
Gilbert-Varshamov Boundbound statement
• For every n and d<n/2 there is an (n,k,d)q (not necessarily linear) code such that:
• In terms of rate and relative min-distance:
Gilbert-Varshamov Bound Proof
• On Board
• Sketch of proof:
• if C is maximal then:
• And
• Now use union bound and entropy to obtain result (we show for q=2, using binary entropy)
GV-Bound
• Gilbert proved this with a greedy construction
• Varshamov proved for linear codes
• proved using random generator matrices – most matrices are good error correcting codes
Singleton / GV Plot

1

Singleton (upper)

Gilbert-Varshamov (lower)

0.5

1

Hamming Bound (Upper)
• With similar reasoning to GV bound but using
• For q=2 can show that
Bounds plot

Code Concatenation - Motivation
• RS codes imply we can construct good [n,k,d]q codes for any q=pk
• Practically would like to work with small q (2, 28)
• Consider the “obvious” idea for binary code generated from C – simply convert each symbol from Σn to log2q,
• What’s the problem with this approach ? (write the new code!)
Code Concatenation
• Due to Forney (1966)
• Two codes:
• Outer: Cout = [N,K,D]Q
• Inner: Cin = [n,k,d]q
• Inner code should encode each symbol of outer code  k = logqQ
Code Concatenation
• How does it work ?

* Luca Trevisan (Lecture 2)

Code Concatenation
• What is the new code ?
• dcon = dD Proof:
• On board
Code Concatenation (Examples)
• Asymptotically
• δ = ¼ 
• R=logn/2n  0 
Good Codes
• Can we “explicitly” build asymptotically good (linear) codes ?
• asymptotically good = constant R, δ> 0 as n∞
• Explicit = polytime constructable / logspace constructible
Asymptotically Good Codes
• GV tells us that most linear functions of a certain size are good error-correcting codes
• Can find a good code in brute-force
• Use brute force on inner-code, where the alphabet is exponentially smaller!
• Do we really need to search ?
Wozencraft Ensemble
• Consider the following set of codes:

such that (R=1/2) (

• Notice that (on board)
Wozencraft Ensemble
• Lemma: There exists an ensemble of codes c1,..cN of rate ½ where N = qk-1 such that for at least (1-ε)N value of i, the code Ci has distance dis.t.
• Proof (on board), outline:
• Different codes have only 0n in common
• Let y=Cα(x), then, If wt(y)<d

 y in Ball(0n, d)

 there are at most Vol(n,d) “bad” codes

• For large enough n=2k, we have Vol(n,d) ≤ εN
Wozencraft Ensemble
• Implications:
• Can construct entire ensemble in O(2k)=O(2n)
• There are many such good codes, but which one do we use ?
Justesen Code
• Concatenation of:
• Cout - RS code over
• a set of inner codes
• Justesen Code: C* = Cout(C1, C2, .. CN)
• Each symbol of Cout is encoded using a different inner code Cj
• If RS has rate R C* has rate R/2
Justesen Code - δ
• Denote the outer RS code [N,K,D]Q
• Claim: C* has relative distance
Justesen Code Proof
• Intuition: like regular concatenation, but εN bad codes.
• for x≠y, the outer code induces S={j | xj≠yj},
• |S| ≥D
• There are at most εN j’s such that Cj is bad and therefore at least |S|- εN ≥ D- εN ≥ (1-R- ε)N good codes
• since RS implies D=N-(K-1)
• Each good code has relative distance ≥ d
• d* ≥ (1-R- ε)Nd
Justesen Code
• The concatenated code C* is an asymptotically good code and has a “super” explicit construction
• Can take q=2 to get such a binary code