Binary compression rates for ascii formats
Download
1 / 47

Binary Compression Rates for ASCII Formats - PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on

Binary Compression Rates for ASCII Formats. Martin Isenburg Jack Snoeyink University of North Carolina at Chapel Hill. Take this home:. “We can store compressed geometry as ASCII and achieve benchmark compression rates.”. “Geometry compression does not require binary.”.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Binary Compression Rates for ASCII Formats' - sal


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Binary compression rates for ascii formats

Binary Compression RatesforASCII Formats

Martin Isenburg Jack Snoeyink

University of North Carolina at Chapel Hill


Take this home
Take this home:

“We can store compressed geometry as ASCIIand achieve benchmarkcompression rates.”

“Geometry compressiondoes not require binary.”

“This enables full conformancebetween ASCII and binary X3D.”


Overview
Overview

  • Motivation

  • Benchmark Compressor

  • Storing Bits as ASCII

  • Results

  • More Motivation

  • Demo

  • Summary



Why ascii scenes
Why ASCII Scenes?

  • authors like text-based 3D

    • wide acceptance

    • read, understand, modify, and save scene with any text editor

    • Web3D Developer survey: “very important”

  • no binary standard

  • rather readable ASCII than binary


Why compression
Why Compression?

  • authors want compressed 3D

    • smaller files

    • faster download

    • Web3D Developer survey: “very important”

  • data-heavy nodes

    • audio / images / video

    • geometry


Approaches to compression
Approaches to Compression

  • binary formats:

    • proprietary compression schemes

    • automatically “protects” the content

    • Shockwave3D, Viewpoint, Virtue3D, Cult3D ...

  • gzipped text-based formats:

    • human readable ASCII content

    • WRL, X3D, OBJ, PLY, SMF, OFF…


Compression standards
Compression Standards

  • software to read, edit, and write content is available and widely used

  • binary compression standards:

    • audio data: MP3

    • image data: JPEG, GIF

    • movie data: MPEG

    • geometry data: ?


Why no standard yet 1
Why no Standard yet? (1)

 structure of 3D data is complex

  • audio: sequence of numbers

  • image: block of numbers

  • video: sequence of blocks of numbers

  • geometry: vertex positions

+ triangles (or polygons ?)+ texcoords (1, 2, 4, 8 layers ?)+ normals (smoothing groups ?)+ colors (per-face ? per-vertex ?)+ bones (1, 2, 3 attachments ?)


Why no standard yet 2

my mom, dad, sister,aunt, uncle, grandma,friends, neighbors, …

gamers, researchers

Why no Standard yet? (2)

 3D data is much less used

  • audio: everybody

  • image: everybody

  • video: everybody

  • geometry: anybody ?

     3D data is not a consumable product on its own

     profitability issues


Geometry compression x3d
Geometry compression & X3D

  • long wanted feature (`96)

    • compression requires binary VRML

    • Compressed Binary Format workgroup

    • binary VRML + 5 new compressed nodes

  • but … CBF proposal refused (`98)

    • problem-child: the binary specification

  • second call for BF proposals (`00)

    • unanswered


Compression without binary
Compression without Binary

 Web3D 2002 paper

  • eliminate binary requirement

  • only requires specification changes where it matters

  • does not interfere with how authors publish content

  • does not affect (overall) readability

Coding Polygon Meshes as Compressable ASCII [Isenburg & Snoeyink, 02]


Design objective
Design Objective

  • work with Shout3D API

  • pure Java viewer (plugin-less)

  • fast, light-weight decoding

    • low computational complexity

    • small size for decoder class

  • be worthwhile

less than 400lines of code

5,381 bytes

1 : 6


Ascii coder binary coder
ASCII coder binary coder

  • How good is the compression?

    • compare to benchmark coder

    • oops …

  • New Design Objective:>>> Maximum Compression <<<

    • produce ASCII output

    • compress with “gzip -9”

    • be as good as benchmark


BenchmarkCompressor


Benchmark compressor
Benchmark Compressor

  • Connectivity Coder

    • codes connectivity as sequence of vertex degrees

Triangle Mesh Compression [Touma & Gotsman, 98]

  • Geometry Coder

    • codes geometry as sequence of corrective vectors

 only handles fully triangulated meshes

 no support for texture coordinates


Polygon mesh compressor

Compressing Polygon Mesh Connectivity withDegree Duality Prediction [Isenburg, 02]

Compressing Polygon Mesh Geometry withParallelogram Prediction [Isenburg & Alliez, 02]

handling textured meshes:

Compressing the Property Mapping of Polygon Meshes [Isenburg & Snoeyink, 01]

Compressing Texture Coordinates with Selective Linear Predictions [Isenburg & Snoeyink, 03]

Polygon Mesh Compressor

handling non-triangular meshes:


It s available
It’s available!

  • a working Webimplementation

  • Java applet

  • based on node set of Shout3D

  • WRL,OBJ,PLY,OFF,SMF,…

  • compress anymodel online


Binary vs ascii

[results from Web3D’02 paper]

66.2

29.4

34.9

40.9

39.9

34.6

Binary vs. ASCII

binary

difference

ASCII

scene

lion

wolf

raptor

horse

cat

dog

average

42.6

18.4

21.5

20.3

22.5

21.5

55 %

60 %

62 %

101 %

77 %

61 %

74 %


Mesh compression process
Mesh Compression Process

grow region + store symbols


Why is binary so much better
Why is binary so much better?

Because it …

  • is binary?

  • stores symbols in binary?

  • uses better predictiveencoding scheme?

  • compresses symbols withentropy coding?

  • allows to store a bit-stream?

No!

Not quite!

Sort of.

Yes, but …

Yes!


Entropy coders

0.2 bits

1.3 bits

2.0 bits

Entropy Coders

for a symbol sequence of t types

t

1

Entropy = pi• log2( ) bits

pi

i =1

# of type t

pi=

# total


Contents of binary file
Contents of Binary File

  • an array of bits

    [0 0 1 1 0 1 0 0 1 1 1 1 0 … 1 0 0 0 1 0 1 0]

  • six floats

    xmin= -1.2323 ymin= -2.1324 zmin= -1.7123

    xmax= 2.6731 ymax= 0.8372 zmax= 3.4971

  • an integer

    precision = 12


Triceratops wrl
triceratops.wrl

Shape {appearance Appearance {material Material {diffuseColor1 .5 0 }}geometry IndexedFaceSet { creaseAngle 0.9 coord Coordinate { point [-0.0715 1.7609 ... -0.4479 -1.5153 1.5304] } coordIndex [7 6 209 204 -1 4 … 4577 4223 4222 -1]}

}


Triceratops compressed wrl
triceratops_compressed.wrl

Shape {appearance Appearance {material Material {diffuseColor1 .5 0 }}geometry CompressedIndexedFaceSet { creaseAngle 0.9 code [ 0 0 1 1 1 0 1 1 0 1 1 1 0 1 1 0 1 1 1 0 1 1 1 0 1 1 0 1 1 1 1 0 1 1 1 0 … 1 0 0 0 1 1 1 0 1 1 0 1 1 0 1 1 0] bits 12 box [ -1.2323-2.1324 -1.7123 2.67310.83723.4971]}

}


Triceratops compressed wrl1
triceratops_compressed.wrl

Shape {appearance Appearance {material Material {diffuseColor1 .5 0 }}geometry CompressedIndexedFaceSet { creaseAngle 0.9 code “AQg+N9gOd+Rhh+lXdfzKEyz0CCZUJHLs3+ fWpsTQFxortsk8XW8j7jEZwL … vXS7GVbHHUuX” bits 12 box [ -1.2323-2.1324 -1.7123 2.67310.83723.4971]}

}


Storing Bitsas ASCII


Design goals
Design Goals

  • resulting ASCII should be “safe”

obvious choices:

  • uuencode

  • base64

  • Hexbin

  • represent binary files with a set of 64 ASCII characters

  • other possibilities: 2 …. 96 chars


Design goals1

LT[6]

LT[19]

Design Goals

  • resulting ASCII should be “safe”

  • conversion should be simple

efficient implementation byusing 2k ASCII characters

  • value of k bits

  • look-up table

LT = {‘A’,‘B’,‘C’,‘D’,‘E’,‘F’,‘G’,‘H’, ……,‘+’,‘=’}

[0 0 1 1 0 1 0 0 1 1 1 1 0 1 … ]


Design goals2

number of ASCII chars

expansion factor: 8 / log2(n)

Design Goals

  • resulting ASCII should be “safe”

  • conversion should be simple

  • ASCII representation should be compact

  • 32 chars = 60.0 %

  • 64 chars = 33.0 %

  • 96 chars = 21.5 %


Design goals3
Design Goals

  • resulting ASCII should be “safe”

  • conversion should be simple

  • ASCII representation should be compact

  • size of ASCII representation of bits after “gzipping” should be close to original



Fish wrl
fish.wrl

Shape {appearance Appearance { material Material { modulateTextureWithDiffuse truediffuseColor1 1 1 } texture ImageTexture { url fish.jpg }}geometry IndexedFaceSet { creaseAngle 0.9 coord Coordinate { point [-0.0715 4.7609 6.3930 ... -0.4479 -4.5153 4.5304] } coordIndex [ 7 6 209 204 -1 4 217 … 4577 -1 4577 4223 4222 -1 ] texCoord TextureCoordinate { point [ 0.3735 0.9441 0.3289 … 0.2666 0.4990 0.1082 ] } texCoordIndex [ 0 1 2 3 -1 4 5 6 7 … 4311 -1 4311 4293 4683 -1 ]}

}


Fish compressed wrl
fish_compressed.wrl

Shape {appearance Appearance { material Material { modulateTextureWithDiffuse truediffuseColor1 1 1 } texture ImageTexture { url fish.jpg }}geometry CompressedIndexedFaceSet { creaseAngle 0.9 code “AQg+N9gOd+Rhh+lXdfzK20Eyz0CCZUJHMK8EKLs3QrP+ fWpsTQFxo4dPVVZ0pGK720hVQw3a9xXrtsk8XW4d8j7jEDfP7pC hVQw3a9xXrtPTsk8XW8j7jEDfP7pCgEofTPoTnfUYfTWzzAbc9eh Fji6UL0ZwL … vXS7GVbHHUuX“ bits 12 box [ -1.9725 -7.6861 -10 1.9725 7.6861 10 ] bits_tex 10 box_tex [ 0.0146 0.0749 0.9896 0.9834 ]}

}


Resulting file sizes
resulting file sizes

[without gzipping the text file]

IFS

ratio

comp IFS

scene

lion

wolf

raptor

horse

cat

dog

average

1,360

569

586

749

791

586

57

25

29

27

30

29

1 : 24

1 : 23

1 : 20

1 : 28

1 : 26

1 : 20

1 : 25


Resulting file sizes1
resulting file sizes

[with gzipping the text file]

IFS

ratio

comp IFS

scene

lion

wolf

raptor

horse

cat

dog

average

442

183

200

266

267

186

43

19

22

21

23

22

1 : 10

1 : 10

1 : 9

1 : 13

1 : 11

1 : 8

1 : 11


Binary vs ascii1
Binary vs. ASCII

binary

difference

ASCII

scene

lion42.6 43.2 1.3 %

wolf 18.4 18.7 1.8 %

raptor 21.5 21.9 1.7 %

horse 20.3 20.6 1.7 %

cat 22.5 22.8 1.6 %

dog 21.5 21.9 1.7 %

average 1.7 %


MoreMotivation


Ascii binary compression

Scene.x3d

.gz

.gz

“living room”

“living room”

IFS

IFS

?

“frame”

“frame”

compress

T

IFS

T

IFS

“mona lisa”

“mona lisa”

IMG

12 bits

IMG

“statue”

“statue”

T

IFS

T

CIFS

asc2bin

asc2bin

bin2asc

bin2asc

Scene.bx3d

Scene.bx3d

“living room”

“living room”

IFS

IFS

“frame”

“frame”

compress

T

IFS

T

IFS

“mona lisa”

“mona lisa”

IMG

12 bits

IMG

“statue”

“statue”

T

IFS

T

CIFS

ASCII binary Compression

Scene.x3d

ASCII

binary


Complete conformance
Complete Conformance

  • compressed nodes exist in ASCII and binary

  • going back and forth between the two is a very simple mapping

  • same decompression algorithm used – no matter if compressed node stored in ASCII or binary


Demo

> localweb <



Summary 1

two alternativerepresentations

author’s decisionat publishing time

Summary (1)

  • completely independent things:

    • ASCII format

    • binary format

    • use of compressed geometry nodes

  • store compressed nodes in ASCII

  • as compact as if stored in binary (after gzipping)


Summary 2
Summary (2)

  • provide “bit-field” in ASCII formats

  • same decompression algorithm for ASCII and binary version

  • full conformance between ASCII and binary version of X3D --- --- including compressed nodes

 design/decide compression now without waiting for a binary format !!!



Example xml node
Example XML node

CompressedIndexedFaceSet : IndexedFaceSet{ MFbyte [out] code NULL IC SFint32 [out] bit [1..24] IC SFint32 [out] bit_tex [1..16] ICSFint32 [out] bit_col [1..16] IC SFint32 [out] bit_nor [1..16] IC

SFvec6f [out] box [1..24] IC SFvec4f [out] box_tex [0 0 1 1] ICSFvec6f [out] box_col [0 0 0 1 1 1] IC

SFbool [] valid [false] IC

}


ad