Practical Terrain Compression Techniques for Efficient Data Management
Explore advanced compression methods for terrain data at high resolutions to optimize storage and memory usage while considering predictors, interpolation, lossless techniques, floating-point compression, and traversal strategies. Dive deep into compression philosophy and universal compression concepts.
Practical Terrain Compression Techniques for Efficient Data Management
E N D
Presentation Transcript
Compression Information Management Lawrence Ibarria, CS7491
Practical Example: Terrain • Objective: Compress Terrain • 16k x 16k resolution • 16 Bits precision • 188 Mb data
Considerations • Lossless/Lossy compression • Memory requirements • Compress to reduce disk usage • Compress to reduce memory usage • Complexity of compression
2D Predictions • Extrapolating • Lorenzo Predictor • Bi-Lorenzian • Interpolating • Linear, Cubic • Radial • Spectral
Lorenzo Predictor • N-Dimensional predictor • Fits polynomials of degree N-1 • 2D: = + -
Test Compression • Dataset N x N • Code: write(data[0][0]); for (i = 1; i<N; i++) write(data[0][i]-data[0][i-1]); for (j = 1; j<N; j++) { write(data[j][0]-data[j-1][0]); for (i = 1; i<N; i++) { p = data[j][i-1]+data[j-1][i]-data[j-1][i-1]; write(data[j][i] - p); } }
Prediction = Transform • Data has been transformed • Histogram and probabilities change • That allows for far better entropy • Result is fed to a Huffman/Arithmetic
Pros/Cons • No complex traversal • 1 Line memory buffer • Fast to compute • It only predicts linear in 2D • Every point depends on its predecessor
Bi-Lorenzian • Increase the number of points • Higher degree prediction
Bi-Lorenzian • Solution:
Interpolation • Interpolation is twice as accurate as extrapolation • Several traversal order, more complex • Progressive transmission easy to add
2D BiLinear Interpolation • 4 points to predict the middle one
2D BiCubic Interpolation • 16 point mask
2D Radial Predictor • Small Footprint • High prediction power: bi-quadratic
Spectral Prediction • Have a prediction for any configuration • That prediction should be optimal • On 2D, a 3x3 neighborhood • 3 possible places we predict • 8 points we may know or not • TOTAL: 768 predictors • Imagine that in 3D, or with 4x4…
Hierarchy • All Interpolating predictors are hierarchical • Decompression by levels allow for progressive decoding
Traversal Orders • In 2D, there are FACE and EDGE pts • Scanline order, EDGE-FACE, FACE-EDGE
Recount of Interpolation • Linear: Simple but limited • Cubic: Very large footprint • Radial: Good predictor, needs many points • Spectral: Huge compendium, several traversals
Lossless compression • A = B - C; B = A + C; • The previous statement is no always true. • Prediction is a reversible transform. It is not reversible under Floatin point.
Lossless Floating Point Compression • Solution: NEW FP Compressor • Developed by Peter Lindstrom, Martin Isenburg, and later collaborations by Jarek Rossignac, Lawrence Ibarria
FP Compress • Here is how it works: Encode(float real, float pred) { unsigned integer r, p, d, k; r = *(unsigned integer *)ℜ p = *(unsigned integer *)&pred; if (r > p) { d = r-p; k = bsr(d); encode(k+32+1); verbatim_encode(d-(1<<k)); } else { d = p-r; k = bsr(d); encode(k); verbatim_encode(d-(1<<k)); } else { encode(32); } }
Synthesis • 2D Predictors • Extrapolating: Lorenzo, Bi-Lorenzian • Interpolating: Linear, Cubic, Radial • Traversals • How to lossless encode Floating points
Previously covered in class • Symbolic Lossless encoding: • Huffman encoding • Arithmetic encoding
Universal Compressor • No such thing! • Context is relevant for compression • A good graphics compressor, bad for text • For all files with n bits, 2n compressions • Compression is a 1 to 1
ZIP compression • Works on bytes • Dictionary method, Buffer method • Uses Huffman encoding before output
Dictionary method • Idea: What has appeared will appear again • Keep an array of previous byte combinations • Encode array indices. One index can be a very long array combination
Buffer method • More locality than dictionary method • Have a buffer of previously seen bytes • Encode position, offset of buffer the guy sleeps the girl dreams Buffer
Compression Philosophy • Compression transforms the data • Removes redundancy • Discards irrelevant information • Finds patterns in data • Much more than file size reduction!
Compression is Everywhere • Music: MP3 compression • Images: JPEG compression • Video: DVD use MPEG compression • Communications: Satellite
JPEG compression • JPEG uses Fourier Transforms • Converts an 8x8 pixel square into an 8x8 Fourier square • Fourier coefficients are ordered by relevancy • JPEG discards most of Fourier coefficients
