Uniquely decodable and parseable codes
This presentation is the property of its rightful owner.
Sponsored Links
1 / 15

Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a procedure for designing an optimum code PowerPoint PPT Presentation


  • 152 Views
  • Uploaded on
  • Presentation posted in: General

Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a procedure for designing an optimum code Improving bit rate beyond the entropy limit. Average code length.

Download Presentation

Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a procedure for designing an optimum code

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Uniquely decodable and parseable codes instanteneous and prefix codes kraft mc millan inequality huffman coding a proce

  • Uniquely decodable and parseable codes

  • Instanteneous and prefix codes

  • Kraft-Mc-Millan inequality

  • Huffman coding: a procedure for designing an optimum code

  • Improving bit rate beyond the entropy limit

Dr.E.Regentova


Average code length

Average code length

Let N is a no raining and R is a raining event in Las Vegas. Let p(N) calculated over a long period is 0.8 and p(R)=0.2. To decode such a source, we need

= -0.8 log2(0.8)-0.2log20.2=0.7458 bits

Dr.E.Regentova


Can we improve rates by joint coding

Can we improve rates by joint coding?

Average length is 1.2 b/input sequence , that is, 0.4 b/pixels, and the codewords are unique.

Problem : Suppose 0101 is received. It can be decoded as NNNNNRNNNNR or NRNNRN, or else.

Dr.E.Regentova


Parseable codes

Parseable codes

  • Before a codeword can be decoded, it must be parsed.

  • Parsing describes the activity of breaking the message string into its component codewords.

  • After parsing, each codeword can be decoded into its symbol sequence.

Dr.E.Regentova


Example of parseable code

Example of Parseable Code

1.6 bits per sequence, or 0.53 bits/symbol. This is 47% improvement if compare to 1.2 b/s.

Suppose, 101100011010 is received. The only decoded sequence output is NNNNNRRRNNRN.

Dr.E.Regentova


Compare three codes

Compare three codes

Dr.E.Regentova


Code 1 binary tree

Code 1 Binary tree

Code is not parseable

If 00 is received, there is no way to recover either a1a1 or a3 only is sent.

a3 is a descendent of a1, and a4 is a descendent of a2.

a1 is a prefix for a3 and a2 is a prefix of a4

Dr.E.Regentova


Code 2 binary tree

Code 2 Binary tree

Code is parseable

It terminates at 0, or at 111

None of codewords serve as prefix for any other.

Dr.E.Regentova


Instantaneous codes

Instantaneous codes

An instantaneously parseable code is one that can be parsed as soon as the last bit of a codeword is received.

An instantaneous code must satisfy the prefix condition. That is, no codeword may be a prefix of any other codeword.

Dr.E.Regentova


Binary tree code 3

Binary tree: Code 3

Code is parseable, but not instantaneous

Accumulate at 0. The bit before 0 is the last bit of a previous word. Thus, we have to wait until the next symbol is received

Dr.E.Regentova


Prefix code external nodes only are codewords

Prefix code: external nodes only are codewords

Internal nodes:

give rise to other nodes

External nodes:

Do not give rise to other nodes

Dr.E.Regentova


Uniquely decodable and parseable codes instanteneous and prefix codes kraft mc millan inequality huffman coding a proce

Code 2: Prefix code, i.e. none of nodes is a prefix to other nodes

Dr.E.Regentova


Kraft mc millan inequality part a

Kraft-Mc-Millan inequality: part a)

Let C be a code with N codewords with lengths l1,l2,…lN,

and l1≤l2≤…≤lN.

If C is uniquely decodable, then :

*

Dr.E.Regentova


Kraft mc millan inequality part b

Kraft-Mc-Millan inequality- part b)

If we have a sequence of positive integers

l1,l2,…lN,

and l1 ≤ l2 ≤…≤ lN that satisfy inequality (*), then there exists a uniquely decodable code whose codewords lengths are given by the sequence

l1,l2,…lN,

Dr.E.Regentova


Efficiency and optimality

Efficiency and optimality

A measure of the efficiency of the code is its redundancy- the difference between the entropy and the average length.

The optimum code is one with a minimum redundancy

Desired property: minimum variance

Dr.E.Regentova


  • Login