DCSP-8: Minimal length coding II, Hamming distance, Encryption

1. DCSP-8: Minimal length coding II, Hamming distance, Encryption Jianfeng Feng Jianfeng.feng@warwick.ac.uk http://www.dcs.warwick.ac.uk/~feng/dsp.html

2. Huffman coding The code in Table 1, however, is an instantaneously parsable code. It satisfies the prefix condition.

3. 0.729*1+0.081*3*3+0.009*5*3+0.001*5=1.5980

4. Decoding 1 1 1 0 1 1 0 1 0 1 1 0 0 0 0 0 0 0 0 0 1 Why do we require a code with the shortest average length?

5. The derivation of the Huffman code tree is shown in Fig. and the tree itself is shown in Fig. . In both these figures, the letter A to H have be used in replace of the sequence in Table 2 to make them easier to read.

9. Frequency for alphabetics

10. Turbo coding • Using Bayesian theorem to code and decode • Bayesian theorem basically said we should employ priori knowledge as much as possible • Read yourself

11. Channel coding; Hamming distance The task of source coding is to represent the course information with the minimum of symbols. When a code is transmitted over a channel in the presence of noise, errors will occur. The task of channel coding is to represent the source information in a manner that minimises the error probability in decoding.

12. It is apparent that channel coding requires the use of redundancy. If all possible outputs of the channel correspond uniquely to a source input, this is no possibility of detecting errors in the transmission. To detect, and possibly correct errors, the channel code sequence must be longer than the source sequence.

13. A good channel code is designed so that, if a few errors occur in transmission, the output can still be decoded with the correct input. This is possible because although incorrect, the output is sufficiently similar to the input to be recognisable.

14. The idea of similarity is made more firm by the definition of a Hamming distance. Let x and y be two binary sequence of the same length. The hamming distance between these two codes is the number of symbols that disagree.

15. The hamming distance between these two codes is the number of symbols that disagree. Two example distances: 0100->1001 has distance 3 (red path); 0110->1110 has distance 1 (blue path)

16. The Hamming distance between 1011101 and 1001001 is 2. The Hamming distance between 2143896 and 2233796 is 3. The Hamming distance between "toned" and "roses" is 3.

17. Suppose the code x is transmitted over the channel. Due to error, y is received. The decoder will assign to y the code x that minimises the Hamming distance between x and y.

18. It can be shown that to detect n bit errors, a coding scheme requires the use of codewords with a Hamming distance of at least n+1. it can be also shown that to correct n bit errors requires a coding scheme with a least a Hamming distance of 2n+1 between the codewords. By designing a good code, we try to ensure that the Hamming distance between possible codewords x is larger than the Hamming distance arising from errors.

19. 1111111111 0000000000 0000011111 0100100100

20. Channel Capacity One of the most famous of all results of information theory is Shannon's channel capacity theorem. For a given channel there exists a code that will permit the error-free transmission across the channel at a rate R, provided R<C, the channel capacity.

21. C = B log2 ( 1 + (S/N) ) b/s

22. As we have already noted, the astonishing part of the theory is the existence of a channel capacity. Shannon's theorem is both tantalizing and frustrating.

23. It is offers error-free transmission, but it makes no statements as to what code is required. In fact, all we may deduce from the proof of the theorem is that is must be a long one. No none has yet found a code that permits the use of a channel at its capacity. However, Shannon has thrown down the gauntlet, in as much as he has proved that the code exists.

24. We shall not give a description of how the capacity is calculated. However, an example is instructive. The binary channel is a channel with a binary input and output. Associated with each output is a probability p that the output is correct, and a probability 1-p it is not.

25. For such a channel, the channel capacity turns output to be: C =1+ p log2 p+ (1-p) log2(1-p) Here, p is the bit error probability. • If p=0, then C=1. • If p=0.5, then C=0. Thus if there is equal of receiving a 1 or 0, irrespective of the signal sent, the channel is completely unreliable and no message can be sent across it.

26. So defined., the channel capacity is a non-dimensional number. We normally quote the capacity as a rate, in bits/second. To do this we relate each output to a change in the signal. For the binary channel we have C = B [1+p log 2 p+(1-p) log2(1-p)] We note that C<B, i.e. the capacity is always less than the it rate.

27. Error detection coding

28. A very common code is the single parity check code.

29. A very common code is the single parity check code. This code appends to each K data bits an additional bit whose value is taken to make the K+1 word even or odd.

30. A very common code is the single parity check code. This code appends to each K data bits an additional bit whose value is taken to make the K+1 word even or odd. Such a choice is said to have even (odd) parity.

31. A very common code is the single parity check code. This code appends to each K data bits an additional bit whose value is taken to make the K+1 word even or odd. Such a choice is said to have even (odd) parity. With even off parity, a single bit error will make the received word odd (even).

32. To see how the additional of a parity bit can improve error performance, consider the following example.

33. To see how the additional of a parity bit can improve error performance, consider the following example. A common choice of code block is eight. Suppose that bit error rate is p=10-4. Then

34. So, the probability of a transmission with an error is as above. With the additional of a parity error bit we can detect any single bit error.

35. As can be seen the addition of a parity bit has reduced the uncorrected error rate by three orders or magnitude.

36. Single parity bits are common in asynchronous transmission. Where synchronous transmission is used, additional parity symbols are added that check not only the parity of each 8 bit row, but also the parity of each 8 bit column. The column is formed by listing each successive 8 bit word one beneath the other. This type of parity checking is called lock sum checking, and it can correct any single 2 bit error in the transmitted block of rows and columns. However, there are some combinations of errors that will go undetected in such a scheme.

38. Parity checking in this way provides good protection against single and multiple errors when the probability of the errors are independent. However, in many circumstances, errors occur in groups, or bursts. Parity checking the kind just described than provides little protection. In these circumstances, a polynomial code is used.

39. . Encryption

40. Encryption In all our discussion of coding, we have not mentioned what is popularly supposed to be the purpose of coding: security.

41. Encryption In all our discussion of coding, we have not mentioned what is popularly supposed to be the purpose of coding: security. We have only considered coding as a mechanism for improving the integrity of the communication system in the presence of noise.

42. Encryption In all our discussion of coding, we have not mentioned what is popularly supposed to be the purpose of coding: security. We have only considered coding as a mechanism for improving the integrity of the communication system in the presence of noise. The use of coding for security has a different name: encryption.

43. Encryption In all our discussion of coding, we have not mentioned what is popularly supposed to be the purpose of coding: security. We have only considered coding as a mechanism for improving the integrity of the communication system in the presence of noise. The use of coding for security has a different name: encryption. encryption is the process of obscuring information to make it unreadable without special knowledge The use of digital computers has made highly secure communication a normal occurrence.

45. Enigma machine

46. . Encryption Early examples: Caesar cipher: Plain text: Yet it may be roundly asserted that human ingenuity cannot concoct a cipher which human ingenuity cannot resolve Cipher: Ekz oz sgehkxuatjregyyxzkyzngznasgtotmktaozeigttuziutiuiz g iovakzcnoinnasgtotmktaozeigttuzxkyurbk

47. Hill cipher • With each alphabetic, we associate a number with it • Using matrix rather than use single number, to code them pairwise, for example

48. The basis for key based encryption is that is very much easier to encrypt with knowledge of the key than it is to decipher without knowledge of the key. Secret key cryptography: uses a single secret key for both encryption and decryption.

49. Public key cryptography, also known as matched key cryptography, is a form of cryptography in which a user has a pair of cryptographic keys - a public key and a private key.

50. Public key cryptography, also known as matched key cryptography, is a form of cryptography in which a user has a pair of cryptographic keys - a public key and a private key. The private key is kept secret, while the public key may be widely distributed. The keys are related mathematically, but the private key cannot be practically derived from the public key.