slide1
Download
Skip this Video
Download Presentation
PACKMAN – Texture Compression for Mobile Phones

Loading in 2 Seconds...

play fullscreen
1 / 81

PACKMAN Texture Compression for Mobile Phones - PowerPoint PPT Presentation


  • 193 Views
  • Uploaded on

PACKMAN – Texture Compression for Mobile Phones. Jacob Ström, Tomas Akenine-Möller Ericsson Research. Outline. Motivation and Previous work Design Goals Basic Idea Decompression of a Block Compression Results. Why 3D Graphics on a Mobile Phone?. Man-Machine Interfaces. Screen Savers.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PACKMAN Texture Compression for Mobile Phones' - lark


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

PACKMAN – Texture Compression for Mobile Phones

Jacob Ström, Tomas Akenine-Möller

Ericsson Research

slide2

Outline

  • Motivation and Previous work
  • Design Goals
  • Basic Idea
  • Decompression of a Block
  • Compression
  • Results
slide3

Why 3D Graphics on a Mobile Phone?

  • Man-Machine Interfaces
  • Screen Savers
  • Games
  • Maps, Messaging, Browsing and more...
slide4

Why is 3D Graphics Hard on a Mobile Phone?

Limited resources:

  • Small amount of memory
  • Little memory bandwidth
  • Little chip area for special purpose
  • Powered by batteries
slide5

Texture Compression Helps

  • Small amount of memory
    • More texture data can fit in the limited amount of memory
  • Little memory bandwidth
    • More texturing possible for same amount of bandwidth
  • Little chip area for special purpose
    • A texture cache using compressed data can be made smaller
  • Powered by batteries
    • Reduced bandwidth means lower energy consumption
slide10

Design Goals

Major Design Goals

  • Minimal decompression complexity
  • Acceptable quality

Minor Design Goals

  • Low compression complexity
  • Small block size
  • Reasonable compression (around 4 bpp)
slide11

Early Design Decisions

Block size of 32 bits was chosen

  • Few bits -> small bit widths after texture cache
  • Fits 32-bit wide buses on systems without texture cache

No indirect addressing of colors

  • LUT of colors increases latency and complicates hardware

Implications:

  • 2x4 was the only reasonable block size, 4 bpp
slide12

Basic Idea

  • The Human Visual System (HSV) is more sensitive to luminance than to chrominance
  • In video and still image coding, chrominance information is most often subsampled in the x- and y- direction (MPEG, JPEG, H263, H264 etc).
  • Our scheme has basically only one color per 2x4 block. The rest is luminance information
slide13

Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block

12-bit “general color”

slide14

Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block
  • Modify the luminance for each pixel in the block

+

12-bit “general color”

per-pixel luminance

slide15

Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block
  • Modify the luminance for each pixel in the block

+

=

12-bit “general color”

per-pixel luminance

resulting image

slide16

Luminance modification

Luminance is modified additively:

Example: “general block color” is R=17, G=34, B = 51

Luminance modifier for the pixel is 220,

The new pixel value is

R = 17+220 = 237G = 34+220 = 254B = 51+220 = 255 (after clamping)

slide17

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.
  • Problem: Small values [-8, -2, 2, 8] 
slide18

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.
  • Problem: Small values [-8, -2, 2, 8] 
    • smooth transitions OK
slide19

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.
  • Problem: Small values [-8, -2, 2, 8] 
    • smooth transitions OK
    • sharp edges bad
slide20

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.
  • Problem: Small values [-8, -2, 2, 8] 
    • smooth transitions OK
    • sharp edges bad
  • Big values [-255, -127, 127, 255] 
slide21

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.
  • Problem: Small values [-8, -2, 2, 8] 
    • smooth transitions OK
    • sharp edges bad
  • Big values [-255, -127, 127, 255] 
    • smooth transitions bad
slide22

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.
  • Problem: Small values [-8, -2, 2, 8] 
    • smooth transitions OK
    • sharp edges bad
  • Big values [-255, -127, 127, 255] 
    • smooth transitions bad
    • sharp edges OK
slide23

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.
  • Problem: Small values [-8, -2, 2, 8] 
    • smooth transitions OK
    • sharp edges bad
  • Big values [-255, -127, 127, 255] 
    • smooth transitions bad
    • sharp edges OK
  • Solution: Codebook of tables, one/block.
slide24

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images
slide25

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images
slide26

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images
    • simulated annealing
    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)
  • Symmetry was added to reduce on-chip codebook size
slide27

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images
    • simulated annealing
    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)
  • Symmetry was added to reduce on-chip codebook size
slide28

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images
    • simulated annealing
    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)
  • Symmetry was added to reduce on-chip codebook size
slide29

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images
    • simulated annealing
    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)
  • Symmetry was added to reduce on-chip codebook size
slide30

Decompressing a Block

  • First 12 bits is RGB444 which gives the general color for the entire block. It is extended to 24 bits.

153 153 85

extend to 24 bits

12 bit RGB444

slide31

Decompressing a Block

  • Next 4 bits select a table from the code book

5

12 bit RGB444

slide32

Decompressing a Block

  • Next 4 bits select a table from the code book

5

12 bit RGB444

slide33

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table…

5

10

12 bit RGB444

slide34

general color

153

153

85

+

-19

-19

-19

=

134

134

66

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table…

5

10

12 bit RGB444

slide35

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table… and so on…

5

10

11

12 bit RGB444

slide36

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table… and so on…

5

10

11

01

12 bit RGB444

slide37

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table… and so on…

5

10

11

01

01

12 bit RGB444

slide38

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table… and so on…

5

10

11

01

01

10

12 bit RGB444

slide39

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table… and so on…

5

10

11

01

01

10

01

12 bit RGB444

slide40

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table… and so on…

5

10

11

01

01

10

01

11

12 bit RGB444

slide41

Decompressing a Block

  • The next 2 bits modify the first pixel according to the table… and so on…

5

10

11

01

01

10

01

11

11

12 bit RGB444

slide42

Simple Decompression

  • Only three adders and some mux:es.
  • Only twelve adders for four parallel units needed for bilinear interpolation
  • S3TC: ~ 60 adders
  • PVRTC: ~ 60 adders
slide43

Simple Decompression

  • The correct texel is selected
slide44

Simple Decompression

  • The correct texel is selected
  • The modifier value is looked up
slide45

Simple Decompression

  • The correct texel is selected
  • The modifier value is looked up
  • The general color is extended to 24 bits
slide46

Simple Decompression

  • The correct texel is selected
  • The modifier value is looked up
  • The general color is extended to 24 bits
  • The modifier value is added
slide47

Simple Decompression

  • The correct texel is selected
  • The modifier value is looked up
  • The general color is extended to 24 bits
  • The modifier value is added
  • The result is clamped
slide48

Compression

  • To compress a block, we need to find
    • “general color”

general color

slide49

Compression

  • To compress a block, we need to find
    • “general color”
    • table

general color

table

slide50

Compression

  • To compress a block, we need to find
    • “general color”
    • table
    • pixel indices.

pixel indices

general color

table

slide51

average

Quantize

to 12 bits

Average Compression

  • In average compression, the average color of the 2x4 block is used as general color.
  • Exhaustive search is used for the table and the pixel indices.
  • 30 milliseconds for a 64 x 64 texture.

pixel indices

general color

table

slide52

PSNR

1.5 dB

Average

Exhaustive

Exhaustive Compression

  • In exhaustive compression, exhaustive search is used for the general color as well (optimal compression).
  • One minute for a 64x64 texture.
  • On average, about 1.5 dB better than average compression.
slide53

Exhaustive Compression

  • Why is Exhaustive Compression better than Average?
  • Often they represent the same 24 bit color, but it has been quantized differently.
slide54

Quantization of General Color

  • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

G

R

slide55

Quantization of General Color

  • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

G

R

slide56

Quantization of General Color

  • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.
  • Ordinary Quantizing just chooses the closest one

G

R

slide57

Quantization of General Color

  • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.
  • Ordinary Quantizing just chooses the closest one

G

R

slide58

Quantization of General Color

  • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

G

R

slide59

Quantization of General Color

  • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)
  • We can reach (almost) all points on the dotted lines

G

R

slide60

Quantization of General Color

  • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)
  • We can reach (almost) all points on the dotted lines

G

  • The best quantization point is therefore different.

R

slide61

Quantization of General Color

  • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)
  • We can reach (almost) all points on the dotted lines

G

  • The best quantization point is therefore different.

R

slide62

Quantization of General Color

  • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)
  • We can reach (almost) all points on the dotted lines

G

  • The best quantization point is therefore different.

R

slide63

average

Combined Quantization

to 12 bits

Combined Quantization Compression

  • We call this type of quantization “combined quantization”, since the R, G and B values are now quantized together to the closest line.

pixel indices

general color

table

slide64

Combined Quantization Compression (cont.)

  • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression
slide65

Combined Quantization Compression (cont.)

  • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression
  • Only 0.5 dB worse than Exhaustive search

PSNR

0.5 dB

Average

Exhaustive

Combined

slide66

Combined Quantization Compression (cont.)

  • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression
  • Only 0.5 dB worse than Exhaustive search
  • Still only 30 milliseconds to compress a 64x64 texture.

PSNR

0.5 dB

Average

Exhaustive

Combined

slide67

original

average

exhaustive

combined

Examples

slide68

Error Metric

  • When selecting one way to compress a block over another, an error metric is used to tell which is better.
  • An obvious error metric to use is to sum the squared error over the block:

e2 = (R-R’)2 + (G-G’)2 + (B-B’)2

  • However, since the eye is more sensitive to errors in green than in red and blue, it makes sense to add more weight to the green component:

e2 = wR(R-R’) 2 + wG(G-G’) 2 + wB(B-B’) 2

slide69

Original

Normal error metric

Weighted error metric

Error Metric (cont.)

slide70

original

S3TC

proposed system

Image Quality

  • Peak Signal to Noise Ratio (PSNR) was measured for a number of images.
    • Proposed scheme on average 1 dB worse than S3TC
    • However, luminance component on average 0.5 dB better
slide71

Game Example

Scene with original textures

slide72

Game Example

Scene with PACKMAN compressed textures

slide73

Game Example

Scene with original light maps

slide74

Game Example

Scene with PACKMAN compressed light maps

slide75

Game Example

Scene with original textures and light maps combined

slide76

Game Example

Scene with PACKMAN compressed textures and light maps combined

slide78

original

PACKMAN

Weaknesses

  • Block cannot contain two different hues
    • E.g., A block with red and blue of same luminance.
slide79

Conclusion

  • Emphasis on low complexity
    • 20% of the adders of rivaling schemes
  • Quality is about 1 dB worse than S3TC
  • A fast and near optimal compression scheme has been developed
  • Perceptual quality can be enhanced using weighted error metrics
  • PACKMAN is implemented in Bitboys’ latest graphics processors for mobile phones (announced yesterday).
thanks
Thanks
  • www.gametutorials.com for their Quake3 scene and BSP-culler.
  • Timo Aila for feedback on the sketch.
ad