Slide1 l.jpg
Sponsored Links
This presentation is the property of its rightful owner.
1 / 81

PACKMAN – Texture Compression for Mobile Phones PowerPoint PPT Presentation


  • 167 Views
  • Uploaded on
  • Presentation posted in: General

PACKMAN – Texture Compression for Mobile Phones. Jacob Ström, Tomas Akenine-Möller Ericsson Research. Outline. Motivation and Previous work Design Goals Basic Idea Decompression of a Block Compression Results. Why 3D Graphics on a Mobile Phone?. Man-Machine Interfaces. Screen Savers.

Download Presentation

PACKMAN – Texture Compression for Mobile Phones

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


PACKMAN – Texture Compression for Mobile Phones

Jacob Ström, Tomas Akenine-Möller

Ericsson Research


Outline

  • Motivation and Previous work

  • Design Goals

  • Basic Idea

  • Decompression of a Block

  • Compression

  • Results


Why 3D Graphics on a Mobile Phone?

  • Man-Machine Interfaces

  • Screen Savers

  • Games

  • Maps, Messaging, Browsing and more...


Why is 3D Graphics Hard on a Mobile Phone?

Limited resources:

  • Small amount of memory

  • Little memory bandwidth

  • Little chip area for special purpose

  • Powered by batteries


Texture Compression Helps

  • Small amount of memory

    • More texture data can fit in the limited amount of memory

  • Little memory bandwidth

    • More texturing possible for same amount of bandwidth

  • Little chip area for special purpose

    • A texture cache using compressed data can be made smaller

  • Powered by batteries

    • Reduced bandwidth means lower energy consumption


Previous Work


Previous Work


Previous Work


Previous Work


Design Goals

Major Design Goals

  • Minimal decompression complexity

  • Acceptable quality

    Minor Design Goals

  • Low compression complexity

  • Small block size

  • Reasonable compression (around 4 bpp)


Early Design Decisions

Block size of 32 bits was chosen

  • Few bits -> small bit widths after texture cache

  • Fits 32-bit wide buses on systems without texture cache

    No indirect addressing of colors

  • LUT of colors increases latency and complicates hardware

    Implications:

  • 2x4 was the only reasonable block size, 4 bpp


Basic Idea

  • The Human Visual System (HSV) is more sensitive to luminance than to chrominance

  • In video and still image coding, chrominance information is most often subsampled in the x- and y- direction (MPEG, JPEG, H263, H264 etc).

  • Our scheme has basically only one color per 2x4 block. The rest is luminance information


Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block

12-bit “general color”


Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block

  • Modify the luminance for each pixel in the block

+

12-bit “general color”

per-pixel luminance


Basic Idea

  • Use only 12 bits to specify a “general color” for a 2x4 block

  • Modify the luminance for each pixel in the block

+

=

12-bit “general color”

per-pixel luminance

resulting image


Luminance modification

Luminance is modified additively:

Example: “general block color” is R=17, G=34, B = 51

Luminance modifier for the pixel is 220,

The new pixel value is

R = 17+220 = 237G = 34+220 = 254B = 51+220 = 255 (after clamping)


How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 


How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK


How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad


How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 


How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 

    • smooth transitions bad


How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 

    • smooth transitions bad

    • sharp edges OK


How to specify luminance

  • Two bits per pixel are used to specify the luminance – modifier is one out of four values.

  • Problem: Small values [-8, -2, 2, 8] 

    • smooth transitions OK

    • sharp edges bad

  • Big values [-255, -127, 127, 255] 

    • smooth transitions bad

    • sharp edges OK

  • Solution: Codebook of tables, one/block.


Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images


Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images


Modifier Codebook

  • We started with random values and optimized by minimizing the error for a set of images

  • simulated annealing

  • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Modifier Codebook

    • We started with random values and optimized by minimizing the error for a set of images

    • simulated annealing

    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Modifier Codebook

    • We started with random values and optimized by minimizing the error for a set of images

    • simulated annealing

    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Modifier Codebook

    • We started with random values and optimized by minimizing the error for a set of images

    • simulated annealing

    • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80)

  • Symmetry was added to reduce on-chip codebook size


  • Decompressing a Block

    • First 12 bits is RGB444 which gives the general color for the entire block. It is extended to 24 bits.

    153 153 85

    extend to 24 bits

    12 bit RGB444


    Decompressing a Block

    • Next 4 bits select a table from the code book

    5

    12 bit RGB444


    Decompressing a Block

    • Next 4 bits select a table from the code book

    5

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table…

    5

    10

    12 bit RGB444


    general color

    153

    153

    85

    +

    -19

    -19

    -19

    =

    134

    134

    66

    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table…

    5

    10

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    01

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    01

    11

    12 bit RGB444


    Decompressing a Block

    • The next 2 bits modify the first pixel according to the table… and so on…

    5

    10

    11

    01

    01

    10

    01

    11

    11

    12 bit RGB444


    Simple Decompression

    • Only three adders and some mux:es.

    • Only twelve adders for four parallel units needed for bilinear interpolation

    • S3TC: ~ 60 adders

    • PVRTC: ~ 60 adders


    Simple Decompression

    • The correct texel is selected


    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up


    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up

    • The general color is extended to 24 bits


    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up

    • The general color is extended to 24 bits

    • The modifier value is added


    Simple Decompression

    • The correct texel is selected

    • The modifier value is looked up

    • The general color is extended to 24 bits

    • The modifier value is added

    • The result is clamped


    Compression

    • To compress a block, we need to find

      • “general color”

    general color


    Compression

    • To compress a block, we need to find

      • “general color”

      • table

    general color

    table


    Compression

    • To compress a block, we need to find

      • “general color”

      • table

      • pixel indices.

    pixel indices

    general color

    table


    average

    Quantize

    to 12 bits

    Average Compression

    • In average compression, the average color of the 2x4 block is used as general color.

    • Exhaustive search is used for the table and the pixel indices.

    • 30 milliseconds for a 64 x 64 texture.

    pixel indices

    general color

    table


    PSNR

    1.5 dB

    Average

    Exhaustive

    Exhaustive Compression

    • In exhaustive compression, exhaustive search is used for the general color as well (optimal compression).

    • One minute for a 64x64 texture.

    • On average, about 1.5 dB better than average compression.


    Exhaustive Compression

    • Why is Exhaustive Compression better than Average?

    • Often they represent the same 24 bit color, but it has been quantized differently.


    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    G

    R


    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    G

    R


    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    • Ordinary Quantizing just chooses the closest one

    G

    R


    Quantization of General Color

    • When Quantizing to 12 bits, we go from many positions in RGB space to relatively few.

    • Ordinary Quantizing just chooses the closest one

    G

    R


    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    G

    R


    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    R


    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    • The best quantization point is therefore different.

    R


    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    • The best quantization point is therefore different.

    R


    Quantization of General Color

    • However, the per-pixel luminance modification can compensate for errors in the direction (1,1,1)

    • We can reach (almost) all points on the dotted lines

    G

    • The best quantization point is therefore different.

    R


    average

    Combined Quantization

    to 12 bits

    Combined Quantization Compression

    • We call this type of quantization “combined quantization”, since the R, G and B values are now quantized together to the closest line.

    pixel indices

    general color

    table


    Combined Quantization Compression (cont.)

    • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression


    Combined Quantization Compression (cont.)

    • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression

    • Only 0.5 dB worse than Exhaustive search

    PSNR

    0.5 dB

    Average

    Exhaustive

    Combined


    Combined Quantization Compression (cont.)

    • On average, Combined Quantization Compression yields 1.0 dB better PSNR than average Compression

    • Only 0.5 dB worse than Exhaustive search

    • Still only 30 milliseconds to compress a 64x64 texture.

    PSNR

    0.5 dB

    Average

    Exhaustive

    Combined


    original

    average

    exhaustive

    combined

    Examples


    Error Metric

    • When selecting one way to compress a block over another, an error metric is used to tell which is better.

    • An obvious error metric to use is to sum the squared error over the block:

      e2 = (R-R’)2 + (G-G’)2 + (B-B’)2

    • However, since the eye is more sensitive to errors in green than in red and blue, it makes sense to add more weight to the green component:

      e2 = wR(R-R’) 2 + wG(G-G’) 2 + wB(B-B’) 2


    Original

    Normal error metric

    Weighted error metric

    Error Metric (cont.)


    original

    S3TC

    proposed system

    Image Quality

    • Peak Signal to Noise Ratio (PSNR) was measured for a number of images.

      • Proposed scheme on average 1 dB worse than S3TC

      • However, luminance component on average 0.5 dB better


    Game Example

    Scene with original textures


    Game Example

    Scene with PACKMAN compressed textures


    Game Example

    Scene with original light maps


    Game Example

    Scene with PACKMAN compressed light maps


    Game Example

    Scene with original textures and light maps combined


    Game Example

    Scene with PACKMAN compressed textures and light maps combined


    In the end, the rendered images should be compared.

    original textures

    PACKMAN textures


    original

    PACKMAN

    Weaknesses

    • Block cannot contain two different hues

      • E.g., A block with red and blue of same luminance.


    Conclusion

    • Emphasis on low complexity

      • 20% of the adders of rivaling schemes

    • Quality is about 1 dB worse than S3TC

    • A fast and near optimal compression scheme has been developed

    • Perceptual quality can be enhanced using weighted error metrics

    • PACKMAN is implemented in Bitboys’ latest graphics processors for mobile phones (announced yesterday).


    Thanks

    • www.gametutorials.com for their Quake3 scene and BSP-culler.

    • Timo Aila for feedback on the sketch.


  • Login