Communicating Efficiently

1. Communicating Efficiently Coding Theory

2. The Problem

3. The Problem • Sending data from one place to another (through a channel) may yield errors. • How do you detect such errors? • How do you correct the errors?

4. Mind Test “It tkaes a geart dael of cuoarge to satnd up to yuor enimees, but a geart dael mroe to satnd up to yuor freidns.” -Dumbledore

5. Your Brain is a Code-cracking Machine. For emaxlpe, it deson’t mttaer in waht oredr the ltteers in a wrod aepapr, the olny iprmoatnt tihng is taht the frist and lsat ltteer are in the rghit pcale. The rset can be a toatl mses and you can sitll raed it wouthit pobelrm.

6. Your Brain is a Code-cracking Machine. • S1M1L4RLY, Y0UR M1ND 15 R34D1NG 7H15 4U70M471C4LLY W17H0U7 3V3N 7H1NK1NG 4B0U7 17. • http://www.livescience.com/18392-reading-jumbled-words.html

7. Sending and Receiving RECEIVER SENDER CHANNEL

8. The Basic Problem of Coding Theory Messages are transmitted over a communication channel which is subject to noise. But noise can distort messages resulting in errors.

9. 10010 10000 11101 10110 11101 10100 HI, BOB! HI, COB! 1001 1000 1110 1011 1110 1010 The Basic Problem of Coding Theory • GOALS of Coding Theory: • Error Detection • Error Correction HOW DO WE ACHIEVE THESE GOALS EFFICIENTLY?

10. YES ??? 00 10 Example Messages: YES → 00 NO → 11 COMMUNICATION FAILURE

11. What is coding theory?

12. What is coding theory? CODING THEORYdeals with the design of error-correcting codes for the reliable transmission of information across noisy channels.

13. Imperfect Transmission of Data Data is usually transferred in the form of a series of 0s and 1s. However, transmission of data is not perfect.

14. Some Causes An electric surge, cross-contamination from another data stream, or human error can easily change some of the 0s to 1s and vice versa.

15. Noise in Communication Information media, such as communication systems and storage devices of data, are not absolutely reliable in practice because of noise or other forms of introduced interference.

16. Applications

17. Applications of Coding Theory • Transmission of pictures from distant space • Qualityof sound in CDs • Establishment of computer networks • Communicationthrough telephone lines • Messaging through wireless communication

18. Applications of Coding Theory When photographs are transmitted to Earth from deep space, error-control codes are used to guard against the noise caused by atmospheric interruptions.

19. Applications of Coding Theory Compact discs (CDs) use error- control codes so that a CD player can read data from a CD even if it has been corrupted causing imperfections on the CD.

20. Tasks of Coding Theory

21. Tasks of Coding Theory • The goal of coding theory is to improve the reliability of digital communication by devising methods that enable the receiver to decide whether there have been errors during the transmission (error detection), and if there are, to possibly recover the original message (error correction).

22. Terms to Understand

23. Source Coding Source codinginvolves changing the message source, such as a data terminal or the human voice, to a suitable code for transmission through the channel. The source encodertransforms the source output into a sequence of symbols which we call a “message”. An example of source coding isthe ASCII code.

24. Terminologies Codeword a string of 0's and 1's representing an actual message Stop → 000 Go → 101 Wait → 110

25. Terminologies Stop → 000 Go → 101 Wait → 110 000, 101, 110 are codewords 100, 111, 001, 010, 011 are NOT codewords The LENGTHOF A CODEWORD is the number of binary digits it has.

26. Terminologies Code - the collection or set of all codewords EXAMPLE: Stop → 000 Go → 101 Wait → 110 The code is C = {000, 101, 110}.

27. Terminologies Encode “message“ becomes a “codeword“ STOP 000 STOP SOURCE ENCODER 000

28. Terminologies Decode “received word“ is reverted back to a “message“ 000 STOP 000 SOURCE DECODER STOP

29. 7-bit ASCII CODE Encode the message “SAT“ using the 7-bit ASCII code table shown above. Decode 1010011 1000101 1010100

30. 7-bit ASCII CODE Decode 1000010 1000101 1010011 1010100

31. Simple Codes

32. Parity Check Codes The simplest form of error detection is parity, where a single bit is appended to a bit string. A bit string has odd parity if the number of 1s in the string is odd. A bit string has even parity if the number of 1s in the string is even. Even parity is more common, but both are used.

33. 0000 0100 ??? YES Parity Check Digit Include a parity check digit to make an even parity code. Messages: YES → 000? NO → 111?

34. 0000 0100 ??? YES Parity Check Digit Include a parity check digit to make an even parity code. Messages: YES → 0000NO → 1111

35. EVEN Parity Codewords Append ONE BIT to each word to form a binary word of EVEN PARITY. 000 000 ? 100 100 ? 010 110 ?

36. EVEN Parity Codewords Append ONE BIT to each word to form a binary word of EVEB PARITY. 000 000 0 100 100 0 010 110 1

37. Odd Parity Codewords Append ONE BIT to each word to form a binary word of ODD PARITY. 000 000 ? 100 100 ? 010 110 ?

38. Odd Parity Codewords Append ONE BIT to each word to form a binary word of ODD PARITY. 000 000 1 100 100 1 010 110 0

39. Even Parity Code (An Example) Three Messages Stop → 000 000 0 Go → 111 111 0 Wait → 000 111 1 This is an even parity code. C = {000 000 0, 111 111 0, 000 111 1}

40. Even Parity Code (Detecting an Error) Suppose the binary word received is r = 000 111 0. Will this make any sense? What do you think is the correct message sent? Three Messages Stop → 000 000 0 Go → 111 111 0 Wait → 000 111 1

41. Odd Parity Code (An Example) • Write all codewords in an odd parity code of length 4. • Identify all binary words of length 3. • How many are they? • Add a parity check digit to each binary word to create all codewords of length 4.

42. Odd Parity Code of Length 4

43. Repetition Codes The simplest possible error-correcting code is the repetition code. For instance, if we wanted to send the message 1010, we could repeat each letter a certain number of times and send, say r = 3 times and we obtain, 111 000 111 000.

44. Example Encode the bit string 011001 by repeating each bit twice. Solution: 00 11 11 00 00 11 2. Encode the bit string 011 by repeating each bit thrice. Solution: 000 111 111

45. Majority decoding Consider a code where each bit is repeated 3 times. Suppose we want to transmit the following bit string: 0110100. If no error is made during transmission, the receiver gets 000 111 111 000 111 000 000.

46. Majority decoding Now assume that the receiver gets 000 111 110 000 111 000 010. Has there been any error in this transmitted word? How do you correct it?

47. Majority decoding The received word 000 111 110 000 111 000 010. will be decoded as 000 111 111 000 111 000 000.

48. Repetition Code (An Example) • Write all codewords in a repetition code of length 9 with r = 3. • Identify all binary words of length 3. • How many are they? • Repeat every bit in each binary word 3 times to form the corresponding codeword.

49. Repetition Code of Length 9 (r = 3)

50. Error Detection and Error Correction