Slide1 l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 81

PACKMAN – Texture Compression for Mobile Phones PowerPoint PPT Presentation


  • 157 Views
  • Uploaded on
  • Presentation posted in: General

PACKMAN – Texture Compression for Mobile Phones. Jacob Ström, Tomas Akenine-Möller Ericsson Research. Outline. Motivation and Previous work Design Goals Basic Idea Decompression of a Block Compression Results. Why 3D Graphics on a Mobile Phone?. Man-Machine Interfaces. Screen Savers.

Download Presentation

PACKMAN – Texture Compression for Mobile Phones

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

PACKMAN – Texture Compression for Mobile Phones

Jacob Ström, Tomas Akenine-Möller

Ericsson Research


Slide2 l.jpg

Outline

  • Motivation and Previous work

  • Design Goals

  • Basic Idea

  • Decompression of a Block

  • Compression

  • Results


Slide3 l.jpg

Why 3D Graphics on a Mobile Phone?

  • Man-Machine Interfaces

  • Screen Savers

  • Games

  • Maps, Messaging, Browsing and more...


Slide4 l.jpg

Why is 3D Graphics Hard on a Mobile Phone?

Limited resources:

  • Small amount of memory

  • Little memory bandwidth

  • Little chip area for special purpose

  • Powered by batteries


Slide5 l.jpg

Texture Compression Helps

  • Small amount of memory

    • More texture data can fit in the limited amount of memory

  • Little memory bandwidth

    • More texturing possible for same amount of bandwidth

  • Little chip area for special purpose

    • A texture cache using compressed data can be made smaller

  • Powered by batteries

    • Reduced bandwidth means lower energy consumption


Slide6 l.jpg

Previous Work


Slide7 l.jpg

Previous Work


Slide8 l.jpg

Previous Work


Slide9 l.jpg

Previous Work


Slide10 l.jpg

Design Goals

Major Design Goals

  • Minimal decompression complexity

  • Acceptable quality

    Minor Design Goals

  • Low compression complexity

  • Small block size

  • Reasonable compression (around 4 bpp)


Slide11 l.jpg

Early Design Decisions

Block size of 32 bits was chosen

  • Few bits -> small bit widths after texture cache

  • Fits 32-bit wide buses on systems without texture cache

    No indirect addressing of colors

  • LUT of colors increases latency and complicates hardware

    Implications:

  • 2x4 was the only reasonable block size, 4 bpp


Slide12 l.jpg

Basic Idea

  • The Human Visual System (HSV) is more sensitive to luminance than to chrominance

  • In video and still image coding, chrominance information is most often subsampled in the x- and y- direction (MPEG, JPEG, H263, H264 etc).

  • Our scheme has basically only one color per 2x4 block. The rest is luminance information


Slide13 l.jpg

Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block

12-bit “general color”


Slide14 l.jpg

Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block

  • Modify the luminance for each pixel in the block

+

12-bit “general color”

per-pixel luminance


Slide15 l.jpg

Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block

  • Modify the luminance for each pixel in the block

+

=

12-bit “general color”

per-pixel luminance

resulting image


Slide16 l.jpg

Luminance modification

Luminance is modified additively:

Example: “general block color” is R=17, G=34, B = 51

Luminance modifier for the pixel is 220,

The new pixel value is

R = 17+220 = 237G = 34+220 = 254B = 51+220 = 255 (after clamping)


Slide17 l.jpg

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 


Slide18 l.jpg

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK


Slide19 l.jpg

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad


Slide20 l.jpg

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 


Slide21 l.jpg

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 

    • smooth transitions bad


Slide22 l.jpg

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 

    • smooth transitions bad

    • sharp edges OK


Slide23 l.jpg

How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 

    • smooth transitions bad

    • sharp edges OK

  • Solution: Codebook of tables, one/block.


Slide24 l.jpg

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images


Slide25 l.jpg

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images


Slide26 l.jpg

Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images

  • simulated annealing

  • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Slide27 l.jpg

    Modifier Codebook

    • We started with random values and optimized by minimizing the error for a set of images

    • simulated annealing

    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Slide28 l.jpg

    Modifier Codebook

    • We started with random values and optimized by minimizing the error for a set of images

    • simulated annealing

    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Slide29 l.jpg

    Modifier Codebook

    • We started with random values and optimized by minimizing the error for a set of images

    • simulated annealing

    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Slide30 l.jpg

    Decompressing a Block

    • First 12 bits is RGB444 which gives the general color for the entire block. It is extended to 24 bits.

    153 153 85

    extend to 24 bits

    12 bit RGB444


    Slide31 l.jpg

    Decompressing a Block

    • Next 4 bits select a table from the code book

    5

    12 bit RGB444


    Slide32 l.jpg

    Decompressing a Block

    • Next 4 bits select a table from the code book

    5

    12 bit RGB444


    Slide33 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table…

    5

    10

    12 bit RGB444


    Slide34 l.jpg

    general color

    153

    153

    85

    +

    -19

    -19

    -19

    =

    134

    134

    66

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table…

    5

    10

    12 bit RGB444


    Slide35 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    12 bit RGB444


    Slide36 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    12 bit RGB444


    Slide37 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    12 bit RGB444


    Slide38 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    12 bit RGB444


    Slide39 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    01

    12 bit RGB444


    Slide40 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    01

    11

    12 bit RGB444


    Slide41 l.jpg

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    01

    11

    11

    12 bit RGB444


    Slide42 l.jpg

    Simple Decompression

    • Only three adders and some mux:es.

    • Only twelve adders for four parallel units needed for bilinear interpolation

    • S3TC: ~ 60 adders

    • PVRTC: ~ 60 adders


    Slide43 l.jpg

    Simple Decompression

    • The correct texel is selected


    Slide44 l.jpg

    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up


    Slide45 l.jpg

    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up

    • The general color is extended to 24 bits


    Slide46 l.jpg

    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up

    • The general color is extended to 24 bits

    • The modifier value is added


    Slide47 l.jpg

    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up

    • The general color is extended to 24 bits

    • The modifier value is added

    • The result is clamped


    Slide48 l.jpg

    Compression

    • To compress a block, we need to find

      • “general color”

    general color


    Slide49 l.jpg

    Compression

    • To compress a block, we need to find

      • “general color”

      • table

    general color

    table


    Slide50 l.jpg

    Compression

    • To compress a block, we need to find

      • “general color”

      • table

      • pixel indices.

    pixel indices

    general color

    table


    Slide51 l.jpg

    average

    Quantize

    to 12 bits

    Average Compression

    • In average compression, the average color of the 2x4 block is used as general color.

    • Exhaustive search is used for the table and the pixel indices.

    • 30 milliseconds for a 64 x 64 texture.

    pixel indices

    general color

    table


    Slide52 l.jpg

    PSNR

    1.5 dB

    Average

    Exhaustive

    Exhaustive Compression

    • In exhaustive compression, exhaustive search is used for the general color as well (optimal compression).

    • One minute for a 64x64 texture.

    • On average, about 1.5 dB better than average compression.


    Slide53 l.jpg

    Exhaustive Compression

    • Why is Exhaustive Compression better than Average?

    • Often they represent the same 24 bit color, but it has been quantized differently.


    Slide54 l.jpg

    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    G

    R


    Slide55 l.jpg

    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    G

    R


    Slide56 l.jpg

    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    • Ordinary Quantizing just chooses the closest one

    G

    R


    Slide57 l.jpg

    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    • Ordinary Quantizing just chooses the closest one

    G

    R


    Slide58 l.jpg

    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    G

    R


    Slide59 l.jpg

    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    R


    Slide60 l.jpg

    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    • The best quantization point is therefore different.

    R


    Slide61 l.jpg

    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    • The best quantization point is therefore different.

    R


    Slide62 l.jpg

    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    • The best quantization point is therefore different.

    R


    Slide63 l.jpg

    average

    Combined Quantization

    to 12 bits

    Combined Quantization Compression

    • We call this type of quantization “combined quantization”, since the R, G and B values are now quantized together to the closest line.

    pixel indices

    general color

    table


    Slide64 l.jpg

    Combined Quantization Compression (cont.)

    • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression


    Slide65 l.jpg

    Combined Quantization Compression (cont.)

    • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression

    • Only 0.5 dB worse than Exhaustive search

    PSNR

    0.5 dB

    Average

    Exhaustive

    Combined


    Slide66 l.jpg

    Combined Quantization Compression (cont.)

    • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression

    • Only 0.5 dB worse than Exhaustive search

    • Still only 30 milliseconds to compress a 64x64 texture.

    PSNR

    0.5 dB

    Average

    Exhaustive

    Combined


    Slide67 l.jpg

    original

    average

    exhaustive

    combined

    Examples


    Slide68 l.jpg

    Error Metric

    • When selecting one way to compress a block over another, an error metric is used to tell which is better.

    • An obvious error metric to use is to sum the squared error over the block:

      e2 = (R-R’)2 + (G-G’)2 + (B-B’)2

    • However, since the eye is more sensitive to errors in green than in red and blue, it makes sense to add more weight to the green component:

      e2 = wR(R-R’) 2 + wG(G-G’) 2 + wB(B-B’) 2


    Slide69 l.jpg

    Original

    Normal error metric

    Weighted error metric

    Error Metric (cont.)


    Slide70 l.jpg

    original

    S3TC

    proposed system

    Image Quality

    • Peak Signal to Noise Ratio (PSNR) was measured for a number of images.

      • Proposed scheme on average 1 dB worse than S3TC

      • However, luminance component on average 0.5 dB better


    Slide71 l.jpg

    Game Example

    Scene with original textures


    Slide72 l.jpg

    Game Example

    Scene with PACKMAN compressed textures


    Slide73 l.jpg

    Game Example

    Scene with original light maps


    Slide74 l.jpg

    Game Example

    Scene with PACKMAN compressed light maps


    Slide75 l.jpg

    Game Example

    Scene with original textures and light maps combined


    Slide76 l.jpg

    Game Example

    Scene with PACKMAN compressed textures and light maps combined


    In the end the rendered images should be compared l.jpg

    In the end, the rendered images should be compared.

    original textures

    PACKMAN textures


    Slide78 l.jpg

    original

    PACKMAN

    Weaknesses

    • Block cannot contain two different hues

      • E.g., A block with red and blue of same luminance.


    Slide79 l.jpg

    Conclusion

    • Emphasis on low complexity

      • 20% of the adders of rivaling schemes

    • Quality is about 1 dB worse than S3TC

    • A fast and near optimal compression scheme has been developed

    • Perceptual quality can be enhanced using weighted error metrics

    • PACKMAN is implemented in Bitboys’ latest graphics processors for mobile phones (announced yesterday).


    Thanks l.jpg

    Thanks

    • www.gametutorials.com for their Quake3 scene and BSP-culler.

    • Timo Aila for feedback on the sketch.


  • Login