1 / 17

Huffman Coding Theory: Examples & Algorithm for Optimization

Learn about Huffman coding, an efficient compression method. Explore prefix codes, binary trees, and the Huffman algorithm for optimal code generation. Dive into code examples and correctness proofs.

mgarcia
Download Presentation

Huffman Coding Theory: Examples & Algorithm for Optimization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSE 326Huffman coding Richard Anderson

  2. Coding theory Code examples 000,001,010,011,100,101 1,01,001,0001,00001,000001 00,010,011,100,11,101 • Conversion, Encryption, Compression • Binary coding • Variable length coding

  3. Decode the following 11010010010101011 100100101010 Prefix code Ambiguous

  4. Prefix code • No prefix of a codeword is a codeword • Uniquely decodable

  5. Prefix codes and binary trees • Tree representation of prefix codes

  6. Construct the tree for the following code

  7. Minimum length code • Average cost • Average leaf depth • Huffman tree – tree with minimum weighted path length • C(T) – weighted path length

  8. Compute average leaf depth

  9. Huffman code algorithm • Derivation • Two rarest items will have the longest codewords • Codewords for rarest items differ only in the last bit • Idea: suppose the weights are with and the smallest weights • Start with an optimal code for and • Extend the codeword for to get codewords for and

  10. Huffman code H = new Heap() for each wi T = new Tree(wi) H.Insert(T) while H.Size() > 1 T1 = H.DeleteMin() T2 = H.DeleteMin() T3 = Merge(T1, T2) H.Insert(T3)

  11. Example:Weights 4, 5, 6, 7, 11, 14, 21 21 14 11 6 7 4 5

  12. Draw a Huffman tree for the following data values and show internal weights:3, 5, 9, 14, 16, 35

  13. Correctness proof • The most amazing induction proof • Induction on the number of code words • The Huffman algorithm finds an optimal code for n = 1 • Suppose that the Huffman algorithm finds an optimal code for codes size n, now consider a code of size n + 1 . . .

  14. Key lemma • Given a tree T, we can find a tree T’, with the two minimum cost leaves as siblings, and C(T’) <= C(T)

  15. Modify the following tree to reduce the WPL 29 10 19 6 4 13 6 10 3 5 5

  16. Finish the induction proof • T – Tree constructed by Huffman • X – Any code tree • Show C(T) <= C(X) • T’ and X’ – Trees from the lemma • C(T’) = C(T) • C(X’) <= C(X) • T’’ and X’’ – Trees with minimum cost leaves x and y removed

  17. X : Any tree, X’: – modified, X’’ : Two smallest leaves removed • C(X’’) = C(X’) – x – y • C(T’’) = C(T’) – x – y • C(T’’) <= C(X’’) • C(T) = C(T’) = C(T’’) + x + y <= C(X’’) + x + y = C(X’) <= C(X)

More Related