Uniquely decodable and parseable codes Instanteneous and prefix codes Kraft-Mc-Millan inequality Huffman coding: a procedure for designing an optimum code Improving bit rate beyond the entropy limit. Average code length.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Let N is a no raining and R is a raining event in Las Vegas. Let p(N) calculated over a long period is 0.8 and p(R)=0.2. To decode such a source, we need
= -0.8 log2(0.8)-0.2log20.2=0.7458 bits
Average length is 1.2 b/input sequence , that is, 0.4 b/pixels, and the codewords are unique.
Problem : Suppose 0101 is received. It can be decoded as NNNNNRNNNNR or NRNNRN, or else.
1.6 bits per sequence, or 0.53 bits/symbol. This is 47% improvement if compare to 1.2 b/s.
Suppose, 101100011010 is received. The only decoded sequence output is NNNNNRRRNNRN.
Code is not parseable
If 00 is received, there is no way to recover either a1a1 or a3 only is sent.
a3 is a descendent of a1, and a4 is a descendent of a2.
a1 is a prefix for a3 and a2 is a prefix of a4
Code is parseable
It terminates at 0, or at 111
None of codewords serve as prefix for any other.
An instantaneously parseable code is one that can be parsed as soon as the last bit of a codeword is received.
An instantaneous code must satisfy the prefix condition. That is, no codeword may be a prefix of any other codeword.
Code is parseable, but not instantaneous
Accumulate at 0. The bit before 0 is the last bit of a previous word. Thus, we have to wait until the next symbol is received
give rise to other nodes
Do not give rise to other nodes
Code 2: Prefix code, i.e. none of nodes is a prefix to other nodes
Let C be a code with N codewords with lengths l1,l2,…lN,
If C is uniquely decodable, then :
If we have a sequence of positive integers
and l1 ≤ l2 ≤…≤ lN that satisfy inequality (*), then there exists a uniquely decodable code whose codewords lengths are given by the sequence
A measure of the efficiency of the code is its redundancy- the difference between the entropy and the average length.
The optimum code is one with a minimum redundancy
Desired property: minimum variance