1 / 30

Practical Terrain Compression Techniques for Efficient Data Management

Explore advanced compression methods for terrain data at high resolutions to optimize storage and memory usage while considering predictors, interpolation, lossless techniques, floating-point compression, and traversal strategies. Dive deep into compression philosophy and universal compression concepts.

drewb
Download Presentation

Practical Terrain Compression Techniques for Efficient Data Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Compression Information Management Lawrence Ibarria, CS7491

  2. Practical Example: Terrain • Objective: Compress Terrain • 16k x 16k resolution • 16 Bits precision • 188 Mb data

  3. Considerations • Lossless/Lossy compression • Memory requirements • Compress to reduce disk usage • Compress to reduce memory usage • Complexity of compression

  4. 2D Predictions • Extrapolating • Lorenzo Predictor • Bi-Lorenzian • Interpolating • Linear, Cubic • Radial • Spectral

  5. Lorenzo Predictor • N-Dimensional predictor • Fits polynomials of degree N-1 • 2D: = + -

  6. Test Compression • Dataset N x N • Code: write(data[0][0]); for (i = 1; i<N; i++) write(data[0][i]-data[0][i-1]); for (j = 1; j<N; j++) { write(data[j][0]-data[j-1][0]); for (i = 1; i<N; i++) { p = data[j][i-1]+data[j-1][i]-data[j-1][i-1]; write(data[j][i] - p); } }

  7. Prediction = Transform • Data has been transformed • Histogram and probabilities change • That allows for far better entropy • Result is fed to a Huffman/Arithmetic

  8. Pros/Cons • No complex traversal • 1 Line memory buffer • Fast to compute • It only predicts linear in 2D • Every point depends on its predecessor

  9. Bi-Lorenzian • Increase the number of points • Higher degree prediction

  10. Bi-Lorenzian • Solution:

  11. Interpolation • Interpolation is twice as accurate as extrapolation • Several traversal order, more complex • Progressive transmission easy to add

  12. 2D BiLinear Interpolation • 4 points to predict the middle one

  13. 2D BiCubic Interpolation • 16 point mask

  14. 2D Radial Predictor • Small Footprint • High prediction power: bi-quadratic

  15. Spectral Prediction • Have a prediction for any configuration • That prediction should be optimal • On 2D, a 3x3 neighborhood • 3 possible places we predict • 8 points we may know or not • TOTAL: 768 predictors • Imagine that in 3D, or with 4x4…

  16. Hierarchy • All Interpolating predictors are hierarchical • Decompression by levels allow for progressive decoding

  17. Traversal Orders • In 2D, there are FACE and EDGE pts • Scanline order, EDGE-FACE, FACE-EDGE

  18. Recount of Interpolation • Linear: Simple but limited • Cubic: Very large footprint • Radial: Good predictor, needs many points • Spectral: Huge compendium, several traversals

  19. Lossless compression • A = B - C; B = A + C; • The previous statement is no always true. • Prediction is a reversible transform. It is not reversible under Floatin point.

  20. Lossless Floating Point Compression • Solution: NEW FP Compressor • Developed by Peter Lindstrom, Martin Isenburg, and later collaborations by Jarek Rossignac, Lawrence Ibarria

  21. FP Compress • Here is how it works: Encode(float real, float pred) { unsigned integer r, p, d, k; r = *(unsigned integer *)&real; p = *(unsigned integer *)&pred; if (r > p) { d = r-p; k = bsr(d); encode(k+32+1); verbatim_encode(d-(1<<k)); } else { d = p-r; k = bsr(d); encode(k); verbatim_encode(d-(1<<k)); } else { encode(32); } }

  22. Synthesis • 2D Predictors • Extrapolating: Lorenzo, Bi-Lorenzian • Interpolating: Linear, Cubic, Radial • Traversals • How to lossless encode Floating points

  23. Previously covered in class • Symbolic Lossless encoding: • Huffman encoding • Arithmetic encoding

  24. Universal Compressor • No such thing! • Context is relevant for compression • A good graphics compressor, bad for text • For all files with n bits, 2n compressions • Compression is a 1 to 1

  25. ZIP compression • Works on bytes • Dictionary method, Buffer method • Uses Huffman encoding before output

  26. Dictionary method • Idea: What has appeared will appear again • Keep an array of previous byte combinations • Encode array indices. One index can be a very long array combination

  27. Buffer method • More locality than dictionary method • Have a buffer of previously seen bytes • Encode position, offset of buffer the guy sleeps the girl dreams Buffer

  28. Compression Philosophy • Compression transforms the data • Removes redundancy • Discards irrelevant information • Finds patterns in data • Much more than file size reduction!

  29. Compression is Everywhere • Music: MP3 compression • Images: JPEG compression • Video: DVD use MPEG compression • Communications: Satellite

  30. JPEG compression • JPEG uses Fourier Transforms • Converts an 8x8 pixel square into an 8x8 Fourier square • Fourier coefficients are ordered by relevancy • JPEG discards most of Fourier coefficients

More Related