Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a proce

1 / 15

# Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a proce - PowerPoint PPT Presentation

Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a procedure for designing an optimum code Improving bit rate beyond the entropy limit. Average code length.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a proce' - cera

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Uniquely decodable and parseable codes
• Instanteneous and prefix codes
• Kraft-Mc-Millan inequality
• Huffman coding: a procedure for designing an optimum code
• Improving bit rate beyond the entropy limit

Dr.E.Regentova

Average code length

Let N is a no raining and R is a raining event in Las Vegas. Let p(N) calculated over a long period is 0.8 and p(R)=0.2. To decode such a source, we need

= -0.8 log2(0.8)-0.2log20.2=0.7458 bits

Dr.E.Regentova

Can we improve rates by joint coding?

Average length is 1.2 b/input sequence , that is, 0.4 b/pixels, and the codewords are unique.

Problem : Suppose 0101 is received. It can be decoded as NNNNNRNNNNR or NRNNRN, or else.

Dr.E.Regentova

Parseable codes
• Before a codeword can be decoded, it must be parsed.
• Parsing describes the activity of breaking the message string into its component codewords.
• After parsing, each codeword can be decoded into its symbol sequence.

Dr.E.Regentova

Example of Parseable Code

1.6 bits per sequence, or 0.53 bits/symbol. This is 47% improvement if compare to 1.2 b/s.

Suppose, 101100011010 is received. The only decoded sequence output is NNNNNRRRNNRN.

Dr.E.Regentova

Compare three codes

Dr.E.Regentova

Code 1 Binary tree

Code is not parseable

If 00 is received, there is no way to recover either a1a1 or a3 only is sent.

a3 is a descendent of a1, and a4 is a descendent of a2.

a1 is a prefix for a3 and a2 is a prefix of a4

Dr.E.Regentova

Code 2 Binary tree

Code is parseable

It terminates at 0, or at 111

None of codewords serve as prefix for any other.

Dr.E.Regentova

Instantaneous codes

An instantaneously parseable code is one that can be parsed as soon as the last bit of a codeword is received.

An instantaneous code must satisfy the prefix condition. That is, no codeword may be a prefix of any other codeword.

Dr.E.Regentova

Binary tree: Code 3

Code is parseable, but not instantaneous

Accumulate at 0. The bit before 0 is the last bit of a previous word. Thus, we have to wait until the next symbol is received

Dr.E.Regentova

Prefix code: external nodes only are codewords

Internal nodes:

give rise to other nodes

External nodes:

Do not give rise to other nodes

Dr.E.Regentova

Dr.E.Regentova

Kraft-Mc-Millan inequality: part a)

Let C be a code with N codewords with lengths l1,l2,…lN,

and l1≤l2≤…≤lN.

If C is uniquely decodable, then :

*

Dr.E.Regentova

Kraft-Mc-Millan inequality- part b)

If we have a sequence of positive integers

l1,l2,…lN,

and l1 ≤ l2 ≤…≤ lN that satisfy inequality (*), then there exists a uniquely decodable code whose codewords lengths are given by the sequence

l1,l2,…lN,

Dr.E.Regentova

Efficiency and optimality

A measure of the efficiency of the code is its redundancy- the difference between the entropy and the average length.

The optimum code is one with a minimum redundancy

Desired property: minimum variance

Dr.E.Regentova