1 / 43

Lots of Bits

Lots of Bits. Representations, Exponential Growth, Moore’s Law. http://www.2wheelbikes.com/sitebuilder/images/cable-lock-comb-bike-accessories-469x345.jpg. How many bit strings of length n ?. Adding one bit doubles the number. Text. 8 bits per character “A” = 01000001 “(” = 00101000

yaphet
Download Presentation

Lots of Bits

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Lots of Bits Representations, Exponential Growth, Moore’s Law

  2. http://www.2wheelbikes.com/sitebuilder/images/cable-lock-comb-bike-accessories-469x345.jpghttp://www.2wheelbikes.com/sitebuilder/images/cable-lock-comb-bike-accessories-469x345.jpg

  3. How many bit strings of length n? Adding one bit doubles the number

  4. Text • 8 bits per character • “A” = 01000001 • “(” = 00101000 • How many combinations of 8 bits? 2· 2· 2· 2· 2· 2· 2· 2 = 28 = 256

  5. Hexadecimal Digits

  6. ASCIIAmerican Standard Code for Information InterchangeCharacter represented by Hex xy, e.g. 4B is “K”

  7. ASCII Underneath • Emails • Web pages

  8. http://farm4.static.flickr.com/3021/2494096946_2bf86f8571.jpg?v=0http://farm4.static.flickr.com/3021/2494096946_2bf86f8571.jpg?v=0

  9. What if you need more than 256 characters? • Unicode • 32 bits per character

  10. = 32 2 4, 294, 967, 296 How many Unicode characters?32 bits each, so there are in all †

  11. Binary counting1+1=10, or 1+1=0 and carry 1

  12. Positive and Negative Numbers • Signed and unsigned numbers • Unsigned: 28=256 bit patterns represent 0 … 255 • Signed: 28 bit patterns represent -128 … +127 • Leftmost bit = sign bit: 0 => 0 or pos, 1 => neg • Largest 8-bit positive number = 01111111 = 127 • 0 = 00000000 • Most negative negative number = 10000000 = -128

  13. Negative numbers -1 = 11111111so addition works the same for positive and negative numbers

  14. Biggest Numbers • Biggest positive number = 01111111 (like 999999 on a car odometer) • Most negative negative number = 10000000

  15. Biggest Positive Number + 1 “=”Most Negative Negative Number OVERFLOW!

  16. The Comair Christmas “Glitch” • 16 bits for monthly count of crew changes • Biggest positive 16-bit number =32,767 • December was a bad month, lots of snowstorms, lots of flights rescheduled • As Christmas approached the count went from 32,767 to -32,768 by adding 1

  17. The Y2010 “Glitch” • Binary representation of decimal 10 = 00001010 • Binary Coded Decimal = write decimal 10 as sequence of 4-bit binary codes for digits • Decimal 10 = BCD 0001 0000 • What if you write decimal 10 in BCD but some other program reads it as decimal? • Binary 0001 0000 = decimal 16

  18. Bytes • 1 byte = 8 bits = 2 hex digits = 1 character • 210 =1024 bytes = 1 kilobyte = 1KB • 220 =1,048,576 bytes = 1 megabyte = 1MB • 230 bytes = 1 gigabyte = 1GB = “a billion” • 240 bytes = 1 terabyte = 1TB = “a trillion” • 250 bytes = 1 petabyte = 1PB = “a quadrillion” • 260 bytes = 1 exabyte = 1EB = a quintillion bytes • 270 = zetta • 280 = yotta

  19. K • All this terminology based on the accident that • Which is 1K? • There are new standard names: • 1 kibibyte = 1000 bytes • vs. 1 kilobyte = 1024 bytes • But almost no one uses “kibi-”, “mebi-”, etc. = = 10 3 1024 2 10 1000 ~ †

  20. Moore’s Law (1965) • The number of transistors on a silicon chip doubles every 18 [or 12, or 24] months • 1965: 64 = 26 • 2008 = 2 billion ~ 231 • 25 doublings in 43 years = one doubling every 20+ months

  21. Example of linear increase

  22. Example of exponential increase • Now for the y axis use instead lg(y) = the exponent e such that 2e=y

  23. Same plot, using lg(y) instead of y

  24. One of the Greatest Engineering Achievements • An increase by a factor of 225 is about 30 millionfold • If human speed had increased that much over the past 43 years, we would now be traveling faster than the speed of light

  25. Probabilities • Fair coin: P(heads) = 1/2 • Fair die: P(rolling 3) = 1/6 • Fair card deck: P(hearts) = 1/4 • P(ace) = 1/13

  26. Probabilities of Independent Events Multiply • P(heads and then heads) = 1/2 · 1/2 = 1/4 • P(3 and then 4) = 1/6 · 1/6 = 1/36 • P(ace and ace) = 1/13·1/13 = 1/169 ≈ .0059 but only if the first card drawn is replaced and the deck is completely reshuffled, otherwise the events are not independent • P(ace and ace without reshuffling) = 1/13 · 3/51 ≈ .0045

  27. Unlikely Events • How likely are 100 heads in a row? • (1/2)100 ≈ 10-32 = .00000000000000000000000000000001

  28. How Small is 2-100 ≈ 10-30? • Age of universe ≈ 1018 sec = 1027 nanoseconds (1 nanosecond = 1 ns = 1 billionth of a second = 10-9 sec) • If you do all 100 coin flips in a billionth of a second, you will get the 100-heads event about once every thousand lifetimes of the universe 1030 = 103 ·1027 • This is “never” for all practical purposes

  29. Morse’s telegraph1844 1848

  30. Morse Code (1838)

  31. Morse Code (1838)

  32. How Long are Morse Codes on Average? • Not the average of the lengths of the letters: (2+4+4+3+…)/26 = 82/26 ≈ 3.2 • We want the average a to be such that in a typicalreal sequence of say 1,000,000 letters, the number of dots and dashes should be about a·1,000,000 • The weightedaverage: (freq of A)·(length of code for A) + (freq of B)·(length of code for B) + … = .08·2 + .01·4 + .03·4 + .04·3+… ≈ 2.4

  33. Data vs. Information • Message sequence: “yea,” “nay,” “yea,” “yea,” “nay,” “nay” … • In ASCII, 3·8 = 24 bits of data per message • But if the only possible answers are “yea” and “nay,” there is only 1 bit of information per message • Entropy is a measure of the information content of a message, as opposed to its size • Entropy of this message sequence = 1 bit/msg

  34. Squeezing out the “Air” • Suppose you want to ship pillows in boxes and are charged by the size of the box • Lossless data compression • Entropy = lower limit of compressibility

  35. Claude Shannon (1916-2001)A Mathematical Theory of Communication (1948)

  36. Communication over a Channel Source Coded Bits Received Bits Decoded Message S X Y T Channel symbols bits bits symbols Encode bits before putting them in the channel Decode bits when they come out of the channel E.g. the transformation from S into X changes “yea” --> 1 “nay” --> 0 Changing Y into T does the reverse For now, assume no noise in the channel, i.e. X=Y

More Related