Belief propagation with information correction near maximum likelihood decoding of ldpc codes
Download
1 / 27

Belief-Propagation with Information Correction: Near Maximum-Likelihood Decoding of LDPC Codes - PowerPoint PPT Presentation


  • 86 Views
  • Uploaded on

Belief-Propagation with Information Correction: Near Maximum-Likelihood Decoding of LDPC Codes. Ned Varnica + , Marc Fossorier # , Alek Kav čić + + Division of Engineering and Applied Sciences Harvard University # Department of Electrical Engineering University of Hawaii. Outline.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Belief-Propagation with Information Correction: Near Maximum-Likelihood Decoding of LDPC Codes' - ria-rose


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Belief propagation with information correction near maximum likelihood decoding of ldpc codes

Belief-Propagation with Information Correction: Near Maximum-Likelihood Decoding of LDPC Codes

Ned Varnica+, Marc Fossorier#, Alek Kavčić+

+Division of Engineering and Applied Sciences

Harvard University

#Department of Electrical Engineering

University of Hawaii


Outline
Outline Maximum-Likelihood Decoding of LDPC Codes

  • Motivation – BP vs ML decoding

  • Improved iterative decoder of LDPC codes

  • Types of BP decoding errors

  • Simulation results


Ldpc code graph

. . . Maximum-Likelihood Decoding of LDPC Codes

. . .

LDPC Code Graph

  • Parity check matrix

    • H

  • Bipartite Tanner code graph G = (V,E,C)

    • Variable (symbol) nodes vi  V, i = 0, 1, …, N-1

    • Parity check nodes cj C, j = 0 , 1, … , Nc-1

Nc x N

A non-zero entry in H

an edge in G

  • Code rate

    • R = k/N, k  N-Nc

  • Belief Propagation

    • Iterative propagation of conditional probabilities


Standard belief propagation on ldpc codes
Standard Belief-Propagation on LDPC Codes Maximum-Likelihood Decoding of LDPC Codes

  • Locally operating

    • optimal for cycle-free graphs

      • Optimized LDPC codes (Luby et al 98, Richardson, Shokrollahi & Urbanke 99, Hou, Siegel & Milstein 01, Varnica & Kavcic 02)

    • sub-optimal for graphs with cycles

  • Good finite LDPC have an exponential number of cycles in their Tanner graphs (Etzion, Trachtenberg and Vardy 99)

  • Encoder constructions

  • BP to ML performance gap due to convergence to pseudo-codewords (Wiberg 95, Forney et al 01, Koetter & Vontobel 03)


Examples
Examples Maximum-Likelihood Decoding of LDPC Codes

  • Short Codes

    - e.g. Tanner code with N = 155, k = 64, diam = 6, girth = 8,dmin = 20

  • Long Codes

  • - e.g. Margulis Code with N = 2640k = 1320


Goals
Goals Maximum-Likelihood Decoding of LDPC Codes

  • Construct decoder

    • Improved BP decoding performance

    • More flexibility in performance versus complexity

    • Can nearly achieve ML performance with much lower computational burden

      • Reduce or eliminate LDPC error floors

  • Applications

    • Can use with any “off-the-shelf” LDPC encoder

    • Can apply to any communication/data storage channel


Subgraph definitions
Subgraph Definitions Maximum-Likelihood Decoding of LDPC Codes

^

x(L) {0,1}N

Definition 1:SUC graphGS(L) = (VS(L), ES(L), CS(L))is graph induced by SUCCS(L)

rRN

x {0,1}N

BCJR detector

BP decoder

channel

transmitted binary

vector

received

vector

decoded vector

after L iterations

^

  • Syndrome s = H x(L)

  • CS(L) - Set of unsatisfied check nodes(SUC) CS(L) = {ci :(Hx(L))i 0}

  • VS(L) - Set of variable nodes incident to c  CS(L)

  • ES(L) - Set of edges connecting VS(L)and CS(L)

^

  • dGs(v) - Degree in SUC graphGS(L)for v  V

    • dGs(v)  dG(v)


Properties of suc graph
Properties of SUC graph Maximum-Likelihood Decoding of LDPC Codes

Observation 1: The higher the degree dGs(v) of a node v  Vs(L) the more likely is v to be in error

e.g. Statistics for Tanner (155,64) code blocksfor which BP failed on AWGN channel at SNR = 2.5 dB

  • Select v node

  • Perform information correction


Node selection strategy 1
Node Selection Strategy 1 Maximum-Likelihood Decoding of LDPC Codes

Strategy 1: Determine SUC graph and select the node with maximal degree dGs in SUC graphGS(L)

Select node v0 or v2 or v12


Properties of suc graph cntd
Properties of SUC graph, cntd Maximum-Likelihood Decoding of LDPC Codes

Definition 2: Nodes v1 and v2 are neighbors with respect to SUC if there exist cCS(L) incident to both v1 and v2

CS(L)

  • nv(m) - number of neighbors of v with degree dGs = m

. . .

nv(2) = 1 and nv(1) = 4

. . .

Observation 2: The smaller the number of neighbors (wrt to SUC graph)

with high degree, the more likely v is to be in error


Node selection strategy 2
Node Selection Strategy 2 Maximum-Likelihood Decoding of LDPC Codes

Strategy 2: Among nodes with maximal degree dGs select a node with minimal number of highest degree neighbors

nv0(2)= nv12(2)= 1; nv2(2)= 2

nv0(1)= 4; nv12(1)= 6

Select node v0


Alternatives to strategy 2
Alternatives to Maximum-Likelihood Decoding of LDPC CodesStrategy 2

max

  • dGs = max dGs(v)

  • Set of suspicious nodes Sv= {v : dGs(v) = dGs }

  • Edge penalty function r(v,c) =

    (Nc- set of v nodes incident to c)

  • Penalty function R(v) =  r(v,c) –  r(v,c)

  • Select vp Svas vp = argmin R(v)

  • Numerous related approaches possible

v  V

max

max

max dGs(vn); if Nc \ {v}

vn Nc\{v}

0 ; if Nc \ {v} = 

c  Cs

c  Cs

max

max

v  Sv


Node selection strategy 3
Node Selection Strategy 3 Maximum-Likelihood Decoding of LDPC Codes

  • Decoder input on node vi

  • Memoryless AWGN channel:

Observation 3: A variable node v is more likely to be incorrect if its decoder

input is less reliable, i.e., if |O(v)| is lower

Strategy 3: Among nodes with maximal degree dGs select node with minimal input reliability|O(v)|


Message passing notation
Message Passing - Notation Maximum-Likelihood Decoding of LDPC Codes

  • Set of log-likelihood ratios messages on v nodes:

    M = (C,O)

  • Decoder input:

    O = [O (v0 ), …, O (vN-1)]

  • Channel detector (BCJR) input

    B = [B (v0 ), …, B (vN-1)]

. . .

. . .

C

. . .

V

. . .

. . .

O

. . .

. . .

T

T

T


Symbol correction procedures

j = 1 Maximum-Likelihood Decoding of LDPC Codes

j = 2

j = 3

7

3

8

1

9

4

10

start

11

5

12

2

13

6

14

Symbol Correction Procedures

  • Replace decoder and detector input LLRs corresponding to selected vp

    • O (vp) = +Sand B (vp) = +S

    • O (vp) = –Sand B (vp) = –S

  • Perform correction in stages

    • Test 2j combinations at stage j

    • For each test perform additional Kjiterations

    • Max number of attempts (stages) jmax


Symbol correction procedures1
Symbol Correction Procedures Maximum-Likelihood Decoding of LDPC Codes

  • “codeword listing” approach

    • Test all 2jmax possibilities

    • W – collection of valid codeword candidates

    • Pick the most likely candidate

      • e.g. for AWGN channel set

        x = argmin d(r,w)

j = 1

j = 2

j = 3

7

3

8

1

9

^

4

wW

start

10

  • “first codeword” approach

    • Stop at a first valid codeword

    • Faster convergence, slightly worse performance for large jmax

11

5

12

2

13

6

14


Parallel and serial implementation j max 3

j = 1 Maximum-Likelihood Decoding of LDPC Codes

j = 2

j = 3

j = 1

j = 2

j = 3

7

3

3

2

8

4

1

1

9

6

4

5

10

7

start

11

10

5

9

12

11

2

8

13

13

12

6

14

14

Parallel and Serial Implementation ( jmax= 3 )

start


Complexity parallel implementation

j = 1 Maximum-Likelihood Decoding of LDPC Codes

j = 2

j = 3

7

3

8

1

9

4

10

start

11

5

12

2

13

6

14

Complexity - Parallel Implementation

  • Decoding continued

    • M need to be stored

    • storage  (2jmax)

    • lower Kj required

    • “first codeword” procedure - fastest convergence

  • Decoding restarted

    • M need not be stored

    • higher Kjrequired


Can we achieve ml

ML decoder Maximum-Likelihood Decoding of LDPC Codes

0

10

“codeword listing” procedure

original BP (max 100 iter)

-1

10

-2

10

WER

-3

10

-4

10

-5

10

0

0.5

1

1.5

2

2.5

3

3.5

4

E

/ N

[dB]

b

0

Can we achieve ML?

Fact 1: As jmax N, “codeword listing” algorithm with Kj = 0, for j < jmax, and Kjmax = 1 becomes ML decoder

  • For low values of jmax (jmax << N) performs very close to ML decoder

  • Tanner (N = 155, k = 64) code

  • jmax = 11, Kj = 10

  • Decoding continued

    • faster decoding

    • M need to be stored

  • ML almost achieved


Pseudo codewords elimination
Pseudo-codewords Elimination Maximum-Likelihood Decoding of LDPC Codes

  • Pseudo-codewords compete with codewords in locally-operating BP decoding (Koetter & Vontobel 2003)

  • c - a codeword in an m-cover of G

  • i - fraction of time vi V assumes incorrect value in c

  •  = (0,1, …,N-1) - pseudo-codeword

  • pseudo-distance (for AWGN)

  • Eliminate a large number of pseudo-codewords by forcing symbol ‘0’ or symbol ‘1’ on nodes vp

    • Pseudo-distance spectra improved

    • Can increase min pseudo-distance if jmax is large enough


Types of bp decoding errors
Types of BP decoding errors Maximum-Likelihood Decoding of LDPC Codes

  • Very high SNRs (error floor region)

    Stable errors on saturated subgraphs:

    • decoder reaches a steady state and fails

    • messages passed in SUC graph saturated

Definition 3: Decoder D has reached a steady state in the interval [L1,L2] if Cs(L) = Cs(L1) for all L [L1,L2]

  • 2. Medium SNRs (waterfall region)

  • Unstable Errors:

    • decoder does not reach a steady state


Suc properties in error floor region
SUC Properties in Error Floor Region Maximum-Likelihood Decoding of LDPC Codes

Theorem 1: In the error floor region

Corollary: For regular LDPC codes with

  • Information correction for high SNRs (error floor region)

    • Pros:

      • Small size SUC

      • Faster convergence

    • Cons:

      • dGsplays no role in node selection


Simulation results

ML decoder Maximum-Likelihood Decoding of LDPC Codes

0

“codeword listing” procedure

10

“first codeword” procedure

original BP (max 100 iter)

-1

10

-2

10

WER

-3

10

-4

10

-5

10

0

0.5

1

1.5

2

2.5

3

3.5

4

E

/ N

[dB]

b

0

Simulation Results

  • Tanner (155,64) code

    • Regular (3,5) code

    • Channel: AWGN

    • Strategy 3

    • jmax = 11, Kj = 10

    • More than 1dB gain

    • ML almost achieved


Simulation results1
Simulation Results Maximum-Likelihood Decoding of LDPC Codes

  • Tanner (155,64) code

    • Regular (3,5) code

    • Channel: AWGN

    • Strategy 3

    • “First codeword” procedure

    • jmax = 4,6,8 and 11

    • Kj = 10


Simulation results error floors
Simulation Results – Error Floors Maximum-Likelihood Decoding of LDPC Codes

  • Margulis (2640,1320) code

    • Regular (3,6) code

    • Channel: AWGN

    • Strategy 3

    • “First codeword” procedure

    • jmax = 5, Kj = 20

    • More than 2 orders of magnitudes WER improvement


Simulation results isi channels
Simulation Results – ISI Channels Maximum-Likelihood Decoding of LDPC Codes

  • Tanner (155,64) code

  • Channels:

    • Dicode (1-D)

    • EPR4 (1-D)(1+D)2

  • Strategy 2

  • jmax = 11, Kj = 20

  • 1dB gain

  • 20 % of detected errors are ML


Conclusion
Conclusion Maximum-Likelihood Decoding of LDPC Codes

  • Information correction in BP decoding of LDPC codes

    • More flexibility in performance vs complexity

    • Can nearly achieve ML performance with much lower computational burden

    • Eliminates a large number of pseudo-codewords

      • Reduces or eliminates LDPC error floors

  • Applications

    • Can use for any “off-the-shelf” LDPC encoder

    • Can apply to any communication/data storage channel


ad