Asymptotically good binary code with efficient encoding justesen code
Download
1 / 42

Asymptotically good binary code with efficient encoding - PowerPoint PPT Presentation


  • 468 Views
  • Uploaded on

Asymptotically good binary code with efficient encoding & Justesen code. Tomer Levinboim Error Correcting Codes Seminar (2008). Outline. Intro codes Singleton Bound Linear Codes Bounds Gilbert-Varshamov Hamming RS codes Code Concatention Examples Wozencraft Ensemble Justesen Codes.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Asymptotically good binary code with efficient encoding' - omer


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Asymptotically good binary code with efficient encoding justesen code

Asymptotically good binary code with efficient encoding& Justesen code

Tomer Levinboim

Error Correcting Codes Seminar (2008)


Outline
Outline

  • Intro

    • codes

    • Singleton Bound

  • Linear Codes

  • Bounds

    • Gilbert-Varshamov

    • Hamming

  • RS codes

  • Code Concatention

    • Examples

  • Wozencraft Ensemble

  • Justesen Codes


Hamming distance
Hamming Distance

  • Hamming Distance between

  • The Hamming Distance is a metric

    • Non negative

    • Symmetric

    • Triangle inequality

=


Weight
Weight

  • The weight (wt) of

  • Example (on board)


Code

  • An (n,k,d)q code C is a function such that:

    • For every


Code parameters
Code (parameters)

  • (n,k,d)q

  • Parameters

    • n – block length

    • k – information length

    • d – minimum distance (actually, a lower bound)

    • q – size of alphabet

    • |C| = qk or k=logq|C|


Code parameters div n
Code (parameters div n)

  • Asymptotic view of parameters as n∞:

    • The rate

    • Relative minimum distance

  • Thus an (n,k,d)q can be written as (1,R,δ)q

  • Notation: (n,k,d)q vs. [n,k,d]q – latter reserved for linear code (soon)


Trivial code example
Trivial Code Example

  • FEC3 = write each bit three time

    • R = ?

    • d = ?

  • how many errors can we

    • Detect ? (d-1)

    • Correct ? t, where d=2t+1


Goal

  • Would like to:

    • Maximize δ – correct more

    • Maximize R – send more information

      * conflicting goals - would like to be able to construct an [n,k,d]q code s.t. δ>0, R>0 and both are constant.

    • Minimize q – for practical reasons

    • Maximize number of codewords while minimizing n and keeping d large.


Singleton bound
Singleton Bound

  • Let C be an [n,k,d]q code then

    • k ≤ n – d + 1

      equivalently

    • R ≤ 1 – δ + o(1)

  • Proof: project C to first k-1 coordinates

    • On Board


Visual intuition
Visual intuition

  • On board...

  • Ballq(x,r)

    • r:=d

    • r:=t (where d=2t+1)

  • Volq(n,r) = |Ballq(x,r)|



Linear codes1
Linear Codes

  • An [n,k,d]q code C:FqKFqn is linear when:

    • Fq is a field

    • C is linear function (e.g., matrix)

  • Linearity implies:

    • C(ax+by) = aC(x) + bC(y)

    • 0n member of C


Linear codes example
Linear Codes (example)

  • FEC3

    • [3,1,3]2

  • Hadamard – longest linear code

    • [n,logn, n/2]2

    • e.g., - [8,3,4]2

    • (H - Matrix representation on board)

      • Dimensions

  • Asymptotic behavior


Linear codes minimum distance
Linear Codes – minimum distance

  • Lemma: if C:FqKFqn is linear then

    Note: for clarity Cx means C(x)

  • Proof:

    • ≤ - trivial

    • ≥ - follows from linearity (on board)


Reed solomon code
Reed-Solomon code

  • Idea: oversample a polynomial

  • Let q be prime power and Fq a finite field of size q.

  • Let k<n and fix n elements of Fq,

    • x1,x2,..xn

  • Given a message m=(c0..ck-1) interpret it has the coefficients of the polynomial p


Rs codes
RS Codes

  • Thus (c0..ck-1) is mapped to (p(x1),..p(xn))

    • Linear mapping (Vandermonde)

  • Using linearity, can show for x≠0

     RS meet the Singleton bound

  • Proof: on board

    • (# of roots of a k-1 degree poly)

  • Encoding time



Gilbert varshamov bound preliminaries
Gilbert-Varshamov Bound Preliminaries

  • Binary Entropy

  • Stirling

    Implying that:


Gilbert varshamov bound preliminaries1
Gilbert-Varshamov Bound Preliminaries

  • Using the binary entropy we obtain

  • On board


Gilbert varshamov bound bound statement
Gilbert-Varshamov Boundbound statement

  • For every n and d<n/2 there is an (n,k,d)q (not necessarily linear) code such that:

  • In terms of rate and relative min-distance:


Gilbert varshamov bound proof
Gilbert-Varshamov Bound Proof

  • On Board

  • Sketch of proof:

    • if C is maximal then:

    • And

    • Now use union bound and entropy to obtain result (we show for q=2, using binary entropy)


Gv bound
GV-Bound

  • Gilbert proved this with a greedy construction

  • Varshamov proved for linear codes

    • proved using random generator matrices – most matrices are good error correcting codes


Singleton gv plot
Singleton / GV Plot

1

Singleton (upper)

Gilbert-Varshamov (lower)

0.5

1


Hamming bound upper
Hamming Bound (Upper)

  • With similar reasoning to GV bound but using

  • For q=2 can show that


Bounds plot
Bounds plot

*Madhu Sudan (Lecture 5, 2001)



Code concatenation motivation
Code Concatenation - Motivation

  • RS codes imply we can construct good [n,k,d]q codes for any q=pk

  • Practically would like to work with small q (2, 28)

  • Consider the “obvious” idea for binary code generated from C – simply convert each symbol from Σn to log2q,

  • What’s the problem with this approach ? (write the new code!)


Code concatenation
Code Concatenation

  • Due to Forney (1966)

  • Two codes:

    • Outer: Cout = [N,K,D]Q

    • Inner: Cin = [n,k,d]q

  • Inner code should encode each symbol of outer code  k = logqQ


Code concatenation1
Code Concatenation

  • How does it work ?

* Luca Trevisan (Lecture 2)


Code concatenation2
Code Concatenation

  • What is the new code ?

  • dcon = dD Proof:

    • On board


Code concatenation examples
Code Concatenation (Examples)

  • Asymptotically

    • δ = ¼ 

    • R=logn/2n  0 


Good codes
Good Codes

  • Can we “explicitly” build asymptotically good (linear) codes ?

    • asymptotically good = constant R, δ> 0 as n∞

    • Explicit = polytime constructable / logspace constructible



Asymptotically good codes1
Asymptotically Good Codes

  • GV tells us that most linear functions of a certain size are good error-correcting codes

    • Can find a good code in brute-force

      • Use brute force on inner-code, where the alphabet is exponentially smaller!

      • Do we really need to search ?


Wozencraft ensemble
Wozencraft Ensemble

  • Consider the following set of codes:

    such that (R=1/2) (

  • Notice that (on board)


Wozencraft ensemble1
Wozencraft Ensemble

  • Lemma: There exists an ensemble of codes c1,..cN of rate ½ where N = qk-1 such that for at least (1-ε)N value of i, the code Ci has distance dis.t.

  • Proof (on board), outline:

    • Different codes have only 0n in common

    • Let y=Cα(x), then, If wt(y)<d

       y in Ball(0n, d)

       there are at most Vol(n,d) “bad” codes

    • For large enough n=2k, we have Vol(n,d) ≤ εN


Wozencraft ensemble2
Wozencraft Ensemble

  • Implications:

    • Can construct entire ensemble in O(2k)=O(2n)

    • There are many such good codes, but which one do we use ?


Justesen code
Justesen Code

  • Concatenation of:

    • Cout - RS code over

    • a set of inner codes

  • Justesen Code: C* = Cout(C1, C2, .. CN)

    • Each symbol of Cout is encoded using a different inner code Cj

    • If RS has rate R C* has rate R/2


Justesen code1
Justesen Code - δ

  • Denote the outer RS code [N,K,D]Q

  • Claim: C* has relative distance


Justesen code proof
Justesen Code Proof

  • Intuition: like regular concatenation, but εN bad codes.

  • for x≠y, the outer code induces S={j | xj≠yj},

    • |S| ≥D

  • There are at most εN j’s such that Cj is bad and therefore at least |S|- εN ≥ D- εN ≥ (1-R- ε)N good codes

    • since RS implies D=N-(K-1)

  • Each good code has relative distance ≥ d

  • d* ≥ (1-R- ε)Nd


Justesen code2
Justesen Code

  • The concatenated code C* is an asymptotically good code and has a “super” explicit construction

  • Can take q=2 to get such a binary code


ad