810 likes | 1.14k Views
PACKMAN – Texture Compression for Mobile Phones. Jacob Ström, Tomas Akenine-Möller Ericsson Research. Outline. Motivation and Previous work Design Goals Basic Idea Decompression of a Block Compression Results. Why 3D Graphics on a Mobile Phone?. Man-Machine Interfaces. Screen Savers.
E N D
PACKMAN – Texture Compression for Mobile Phones Jacob Ström, Tomas Akenine-Möller Ericsson Research
Outline • Motivation and Previous work • Design Goals • Basic Idea • Decompression of a Block • Compression • Results
Why 3D Graphics on a Mobile Phone? • Man-Machine Interfaces • Screen Savers • Games • Maps, Messaging, Browsing and more...
Why is 3D Graphics Hard on a Mobile Phone? Limited resources: • Small amount of memory • Little memory bandwidth • Little chip area for special purpose • Powered by batteries
Texture Compression Helps • Small amount of memory • More texture data can fit in the limited amount of memory • Little memory bandwidth • More texturing possible for same amount of bandwidth • Little chip area for special purpose • A texture cache using compressed data can be made smaller • Powered by batteries • Reduced bandwidth means lower energy consumption
Design Goals Major Design Goals • Minimal decompression complexity • Acceptable quality Minor Design Goals • Low compression complexity • Small block size • Reasonable compression (around 4 bpp)
Early Design Decisions Block size of 32 bits was chosen • Few bits -> small bit widths after texture cache • Fits 32-bit wide buses on systems without texture cache No indirect addressing of colors • LUT of colors increases latency and complicates hardware Implications: • 2x4 was the only reasonable block size, 4 bpp
Basic Idea • The Human Visual System (HSV) is more sensitive to luminance than to chrominance • In video and still image coding, chrominance information is most often subsampled in the x- and y- direction (MPEG, JPEG, H263, H264 etc). • Our scheme has basically only one color per 2x4 block. The rest is luminance information
Basic Idea • Use only 12 bits to specify a “general color” for a 2x4 block 12-bit “general color”
Basic Idea • Use only 12 bits to specify a “general color” for a 2x4 block • Modify the luminance for each pixel in the block + 12-bit “general color” per-pixel luminance
Basic Idea • Use only 12 bits to specify a “general color” for a 2x4 block • Modify the luminance for each pixel in the block + = 12-bit “general color” per-pixel luminance resulting image
Luminance modification Luminance is modified additively: Example: “general block color” is R=17, G=34, B = 51 Luminance modifier for the pixel is 220, The new pixel value is R = 17+220 = 237G = 34+220 = 254B = 51+220 = 255 (after clamping)
How to specify luminance • Two bits per pixel are used to specify the luminance – modifier is one out of four values. • Problem: Small values [-8, -2, 2, 8]
How to specify luminance • Two bits per pixel are used to specify the luminance – modifier is one out of four values. • Problem: Small values [-8, -2, 2, 8] • smooth transitions OK
How to specify luminance • Two bits per pixel are used to specify the luminance – modifier is one out of four values. • Problem: Small values [-8, -2, 2, 8] • smooth transitions OK • sharp edges bad
How to specify luminance • Two bits per pixel are used to specify the luminance – modifier is one out of four values. • Problem: Small values [-8, -2, 2, 8] • smooth transitions OK • sharp edges bad • Big values [-255, -127, 127, 255]
How to specify luminance • Two bits per pixel are used to specify the luminance – modifier is one out of four values. • Problem: Small values [-8, -2, 2, 8] • smooth transitions OK • sharp edges bad • Big values [-255, -127, 127, 255] • smooth transitions bad
How to specify luminance • Two bits per pixel are used to specify the luminance – modifier is one out of four values. • Problem: Small values [-8, -2, 2, 8] • smooth transitions OK • sharp edges bad • Big values [-255, -127, 127, 255] • smooth transitions bad • sharp edges OK
How to specify luminance • Two bits per pixel are used to specify the luminance – modifier is one out of four values. • Problem: Small values [-8, -2, 2, 8] • smooth transitions OK • sharp edges bad • Big values [-255, -127, 127, 255] • smooth transitions bad • sharp edges OK • Solution: Codebook of tables, one/block.
Modifier Codebook • We started with random values and optimized by minimizing the error for a set of images
Modifier Codebook • We started with random values and optimized by minimizing the error for a set of images
Modifier Codebook • We started with random values and optimized by minimizing the error for a set of images • simulated annealing • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80) • Symmetry was added to reduce on-chip codebook size
Modifier Codebook • We started with random values and optimized by minimizing the error for a set of images • simulated annealing • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80) • Symmetry was added to reduce on-chip codebook size
Modifier Codebook • We started with random values and optimized by minimizing the error for a set of images • simulated annealing • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80) • Symmetry was added to reduce on-chip codebook size
Modifier Codebook • We started with random values and optimized by minimizing the error for a set of images • simulated annealing • modified version of the Generalized Lloyd algorithm (Linde Buzo Gray ’80) • Symmetry was added to reduce on-chip codebook size
Decompressing a Block • First 12 bits is RGB444 which gives the general color for the entire block. It is extended to 24 bits. 153 153 85 extend to 24 bits 12 bit RGB444
Decompressing a Block • Next 4 bits select a table from the code book 5 12 bit RGB444
Decompressing a Block • Next 4 bits select a table from the code book 5 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… 5 10 12 bit RGB444
general color 153 153 85 + -19 -19 -19 = 134 134 66 Decompressing a Block • The next 2 bits modify the first pixel according to the table… 5 10 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… and so on… 5 10 11 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… and so on… 5 10 11 01 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… and so on… 5 10 11 01 01 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… and so on… 5 10 11 01 01 10 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… and so on… 5 10 11 01 01 10 01 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… and so on… 5 10 11 01 01 10 01 11 12 bit RGB444
Decompressing a Block • The next 2 bits modify the first pixel according to the table… and so on… 5 10 11 01 01 10 01 11 11 12 bit RGB444
Simple Decompression • Only three adders and some mux:es. • Only twelve adders for four parallel units needed for bilinear interpolation • S3TC: ~ 60 adders • PVRTC: ~ 60 adders
Simple Decompression • The correct texel is selected
Simple Decompression • The correct texel is selected • The modifier value is looked up
Simple Decompression • The correct texel is selected • The modifier value is looked up • The general color is extended to 24 bits
Simple Decompression • The correct texel is selected • The modifier value is looked up • The general color is extended to 24 bits • The modifier value is added
Simple Decompression • The correct texel is selected • The modifier value is looked up • The general color is extended to 24 bits • The modifier value is added • The result is clamped
Compression • To compress a block, we need to find • “general color” general color
Compression • To compress a block, we need to find • “general color” • table general color table
Compression • To compress a block, we need to find • “general color” • table • pixel indices. pixel indices general color table