Basic Concepts of Encoding. Codes, their efficiency and redundancy. Cost function. How we can measure the efficiency of encoding?
PowerPoint Slideshow about 'Basic Concepts of Encoding' - cleantha
An Image/Link below is provided (as is) to download presentation
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
The ability to correct the errors is a great “plus”, but we have realized that a code, which is able to correct the errors, is redundant. Where there is an equilibrium point between redundancy and efficiency?
Is it possible to compress the input information using encoding?
To ensure more efficient encoding, we should introduce its cost function. In this terms, the efficient encoding will mean minimization of this function.
Suppose (without loss of generality) that all symbols in all messages have identical cost. Thus, the average cost per message becomes proportional to the average number of symbols per message, that is, the average cost per message = the average length of messages:
It was shown that the lower bound for the average word length is the ratio of the entropy of the original message ensemble to log D, where D is the number of symbols in the encoding alphabet: H(X)/log D
log D is the maximum possible information per symbol.
Example 2. Let us encode a set of the same four messages using a binary alphabet: . Let and , k=1,2,3,4, be the numbers of 0’s and 1’s in the encoding binary vector (word), nk be the length of the encoding word.
The last example shows that it is possible to design a non-uniform code with the 100% efficiency.
Moreover, this code makes it possible to compress the input information: the average length of the encoding vector in the uniform code was 2, while in the non-uniform code, which we have just considered, it was 1.75
The code has the following important property: any message composed from these encoding vectors independently of their amount and order always can be unambiguously decoded. This property of code is referred to as unique decipherability.
A sufficient condition for unique decipherability of a non-uniform code is that no encoding vectors (words) can be obtained from each other by the addition of more letters to the end. This property is referred to as the prefix property or irreducibility. This means that there is no encoding vector, which is a prefix of the different encoding vector.