JPEG

1. JPEG Vaidyanathan A, ECE, Final yr.

2. Agenda • Recap • What is JPEG • Typical Usage • JPEG Characteristics • How it is done??? • Advantages

3. Bit stream Coder .... Lossless image coding Lossy image coding Image coding Objective: To find a way to represent the original image without (?) distortion with the minimum number of bits possible

4. Visually indistinguishable Coder Original Visually distinguishable Lossless and lossy image coding • Lossless image coding : The decoded image is pixel by pixel identical to the original • Lossy image coding : • The decoded image is NOT pixel by pixel identical to the original

5. Agenda • Recap • What is JPEG • Typical Usage • JPEG Characteristics • How it is done??? • Advantages

6. What is JPEG • The name "JPEG" stands for Joint Photographic Experts Group, the name of the committee that created the standard. • The JPEG standard specifies both the codec, which defines how an image is compressed into a stream of bytes and decompressed back into an image, and the file format used to contain that stream.

7. Agenda • Recap • What is JPEG • Typical Usage • JPEG Characteristics • How it is done??? • Advantages

8. Typical Usage • The JPEG compression algorithm is at its best on photographs and paintings of realistic scenes with smooth variations of tone and color. • For web usage, where the bandwidth used by an image is important, JPEG is very popular. JPEG/Exif is also the most common format saved by digital cameras. • On the other hand, JPEG is not as well suited for line drawings and other textual or iconic graphics, where the sharp contrasts between adjacent pixels cause noticeable artifacts. Such images are better saved in a lossless graphics format such as TIFF, GIF, PNG, or a raw image format.

9. Typical Usage • JPEG is also not well suited to files that will undergo multiple edits, as some image quality will usually be lost each time the image is decompressed and recompressed, particularly if the image is cropped or shifted, or if encoding parameters are changed. To avoid this, an image that is being modified or may be modified in the future can be saved in a lossless format such as PNG, and a copy exported as JPEG for distribution. • As JPEG is a lossy compression method, which removes information from the image, it must not be used in astronomical or medical imaging or other purposes where the exact reproduction of the data is required. Lossless formats such as PNG must be used instead.

10. Agenda • Recap • What is JPEG • Typical Usage • JPEG Characteristics • How it is done??? • Advantages

11. JPEG Characteristics • Always Lossy Compression • True 24-bit color (16 million colors) • Compression ration of 2-100 : 1 • Good performance for pictures that are smooth with a lot of colors. • Bad performance for pictures with sharp edges.

12. Uncompressed TIFF(400 x 300 x 24bpp - 360KB)

13. JPEG (19 KB – 5.28% of original image)

14. JPEG (11 KB – 3% of original image)

15. Agenda • Recap • What is JPEG • Typical Usage • JPEG Characteristics • How it is done??? • Advantages

16. Original image Reversible Non-reversible Bit stream Do something Quantizer Q Entropic coder • To prepare the image • To remove redundancy • This an optional block. Although almost always exists • DCT, wavelets, hybrid • Quantizer • Scalar or vectorial • This an optional block. Although almost always exists • Entropic coder • This block always exists Lossy scheme Lossless scheme General scheme of image coding (compression)

17. Dataflow of JPEG Compression Algorithm JPEG File Compression Sampling Quantization Discrete Cosine Transform

18. Sampling JPEG File Compression Sampling Quantization Discrete Cosine Transform

19. Sampling: RGB Color System • Three component representation of the color of a pixel • Represents the intensities of the red, green, and blue components • 24 bit “True Color” • Each component represented with 8 bits of precision • The components each contain roughly the same amount of information

20. Human Visual System • The human eye has a tendency to notice variations of brightness intensity much more than variations of the color in an image • The human eye is not as sensitive to high-frequency chrominance (color) components as it is to luminance (intensity) components • We can take advantage of this by transforming the color space of RGB to another representation

21. Human Visual System • Here is an image represented with 8-bits per pixel

22. Human Visual System • Here is the same image at 7-bits per pixel

23. Human Visual System • And at 6-bits per pixel

24. Human Visual System • And at 5-bits per pixel

25. Human Visual System • And at 4-bits per pixel

26. 2D FFT transform

27. Man-made Scene

28. Can change spectrum, then reconstruct

29. Most information in at low frequencies!

30. Campbell-Robson contrast sensitivity curve We don’t resolve high frequencies too well… … let’s use this to compress images… JPEG!

31. Frequency sensitivity of Human Visual System

32. YUV (YCrCb) Color Space • An ideal format for JPEG compression • The brightness and color information in an image are separated • Concentrates the most important info into one component, allowing for greater compression • Y component represents the color intensity of the image (equivalent to a black and white television signal) • U and V represent the relative redness and blueness of the image

33. YUV Transformation • A linear transformation from RGB to YUV and from YUV to RGB

34. Y u v Formats Y u v Format Y U V RGB 24 bits/pixel YUV 4:2:0 (12 bits/pixel)

35. Discrete Cosine Transform JPEG File Compression Sampling Quantization Discrete Cosine Transform

36. Jean Baptiste Joseph Fourier (1768-1830) had crazy idea (1807): Any periodic function can be rewritten as a weighted sum of sines and cosines of different frequencies. Don’t believe it? • Neither did Lagrange, Laplace, Poisson and other big wigs • Not translated into English until 1878! But it’s true! • called Fourier Series

37. Image compression using DCT • DCT enables image compression by concentrating most image information in the low frequencies • Loose unimportant image info (high frequencies) by cutting B(u,v) at bottom right

38. DCT on 8x8 blocks 8 pixels 8 pixels • We will break the image into non-overlapping 8x8 blocks. • For each block u(m,n), we will take an 8x8 DCT 64 pixels 64 pixels

39. Why 8x8 blocks? V(k) can be coded using R bits/coefficient, with “Distortion” D p = correlation Increasing N beyond some point does little difference R(D) • Typical R approaches asymptote for block sizes of 8x8. • Note: As p goes to 1, the asymptote (1/2N)log(1-p2)2 gets more negative. • Asymptote gets larger • You can do bigger blocks and get better performance. 8x8 Block Size NxN

40. Unitary 2D-DCT Not surprisingly, it turns out that you can get better compression using the DCT if you take into account the horizontal and vertical correlation between pixels simultaneously. • Forward DCT Backward DCT

41. Basis functions for the 8×8 DCT (courtesy Wikipedia)

42. Block-based Discrete Cosine Transform (DCT)

43. A nice set of basis Teases away fast vs. slow changes in the image.

44. Other Tranforms used in Image Processing • KL Transform • Very important theoretically, but not used because no fast algorithm exists and depends on statistics of the image. • KL Transform is optimal in producing transform coefficients that are uncorrelated • Has best average energy compaction for an ensemble of images • Singular Value Decomposition (SVD) • Best energy compaction for a given image.

45. Quantization JPEG File Compression Sampling Quantization Discrete Cosine Transform

46. Quantization • Quantized Value = Round (coefficient / Quantum Value) Choosing a quantum value as small as 20 would convert over half of the coefficients to zeros. The JPEG standard does not specify the quantization values to be used. This is left up to the application. However it does provide some quantization tables that have been tested empirically and found to generate good results

47. 6 4 2 2 4 6 Quantization Code Do something Entropic coder Q

48. Entropic coder Quantization (1) Output levels Input digital levels Output levels 7 5 3 1 0 1 2 3 4 5 6 7 8 9 Input levels

49. Reconstruction levels Output values 192 2 level quantizer 64 1 bit/pixel image(binary) 0 128 255 Input levels Decision levels 8 bits/pixel image Quantization (2)

50. Reconstruction levels Output values 224 4 level quantizer 160 96 32 2 bit/pixel image 0 64 128 192 255 Input levels Decision levels 8 bits/pixel image Quantization (3)