1 / 48

Programming Languages for Compressing Graphics

Programming Languages for Compressing Graphics Morgan McGuire Shriram Krishnamurthi John F. Hughes Brown University { morgan | sk | jfh}@cs.brown.edu Encoding Images as Programs Describing Images (1) setPixel(0,0,BLUE); setPixel(1,0,BLUE); setPixel(2,0,BLUE); … setPixel(50,0,WHITE); …

johana
Download Presentation

Programming Languages for Compressing Graphics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Programming Languages for Compressing Graphics Morgan McGuire Shriram Krishnamurthi John F. Hughes Brown University { morgan | sk | jfh}@cs.brown.edu

  2. Encoding Images as Programs

  3. Describing Images (1) setPixel(0,0,BLUE); setPixel(1,0,BLUE); setPixel(2,0,BLUE); … setPixel(50,0,WHITE); … setPixel(100,0,RED); setPixel(101,0,RED); …

  4. The Cost of Bandwidth • Major cost of doing business on the web • Yahoo!: 65,000,000 pages/day • Valve: 1M €/software patch • Image compression is a one-shot activity • Low vs. high bandwidth users • Multiresolution

  5. Describing Images (2) repeat 50 times setNextPixel(BLUE); repeat 50 times setNextPixel(WHITE); repeat 50 times setNextPixel(RED); nextRow(); repeat 50 times setNextPixel(BLUE); …

  6. American Flag Doesn’t compress as well as the French flag in the “repeat n times” language.

  7. Adding More Primitives drawRectangle(BLUE, …); drawRectangle(RED, …); drawRectangle(WHITE, …); drawStar(WHITE, …); … Is this enough?

  8. Describing Complex Images

  9. Describing Complex Images • JPEG: language consists of frequency domain instructions • GIF: language consists of setPixel and dictionary lookup • Preferred format depends on the image

  10. Observations • Images are programs • Even within one language, many possible descriptions produce similar images • Lossy compression • Description length depends on language and image complexity • Best compression when the language matches the image

  11. The Obvious Compression Scheme • Compress the image in several formats • TGA, GIF, JPG, SVG, SWF • Choose the best • Add a byte to the front of the file specifying the compression language

  12. Problems with the Obvious Scheme • None of the formats may be particularly good for our image • Even JPEG tops out around 50:1, has serious artifacts • Lacking ideal features like: • Multiresolution • Time/Space tradeoff • Introduction of new formats requires new browser plug-ins

  13. Describing Complex Images How can we do better than the obvious approach for images like this?

  14. Using an Expressive Language (compose (vertical-gradient BLUE WHITE) (polygon DARK-BLUE …) (polygon BLACK …) (* (polygon …) (blur tree-texture) YELLOW))

  15. Using an Expressive Language • What if we design a really expressive language for representing images? • Because the “data” is a program the decompressor is part of the “data” • Each image gets its own custom format • High compression • We have control over multiresolution, perceptual artifacts • Package for the web as a plug-in • We only need to upgrade the plug-in when the language itself changes What’s wrong with this plan?

  16. Example setPixel Language Expressive Language ? setPixel(0,0,WHITE); setPixel(1,0,WHITE); setPixel(2,0,WHITE); … (blur (+ (rotate (banana)) (distort (* RED (circle))) …) 2 Mb 2 kb

  17. The Encoding Problem ? Pierre can’t code!

  18. The Encoding Problem Photoshop menu EMACS setPixel(0,0,WHITE); setPixel(1,0,WHITE); setPixel(2,0,WHITE); … (blur (+ (rotate (banana)) (distort (* RED (circle))) …) 2 Mb 2 kb

  19. Where is the “Save as Code” Menu? • It is easy to convert from an image to programs in the GIF/JPG language • More expressive language = harder conversion • How much harder?

  20. Much Harder! • Converting the image into a program that produces it is a search problem • The search space is the space of all possible programs • This is an infinitely large space

  21. Tempering Expressiveness “Good” compression languages are ones where: • Expressive power is large • Searching is easy How do we make searching easy?

  22. Steerable Search Techniques • Genetic Algorithms • Inject domain information through fitness function • Metropolis Search • Inject domain information through transition probabilities • Simulated Annealing • Inject domain information through gradient estimation

  23. Perceptual Fitness Function • Tweaking this is the domain-expert’s job • Perfect fitness function not necessary (or possible!) e (i ) = edge filter, b (i ) = convolve with Gaussian, |ix| = color magnitude

  24. Designing the Language • Desirable language properties for compression • Expresses many images compactly • There are many programs for which another, shorter program exists that produces visually similar output • Desirable search properties • Mutations safety • All programs terminate

  25. The Evolver Language Scalar := real between 0 and 1 Vector := Scalar x Scalar x Scalar Matrix := Vector* Value := Matrix | Scalar | Vector Operator := Add | Collage | Blur | Noise | … Call := Operator x Expr* Expr := Call | Scalar | Vector

  26. The Evolver Language • Automatic coercion between Matrix, Vector, Scalar • Every operator has the same domain and range • Primitives include stock images and textures • No looping constructs • No functions!

  27. l: Not the Ultimate Compressor!

  28. l: Not the Ultimate Compressor! • Copying code is sometimes good • Multiple instances of a pattern in an image often differ slightly • Hard to evolve both definition and multiple applications

  29. Added Benefits of Search • Artist can save the image immediately • Webmaster uses Evolver toolkit to search for an equivalent program • It is easy to find a large program quickly • Webmaster lets Evolver continue to run • Upload new, smaller encodings as they are found • More time = less space = less cost • Multiple constraints: size, time, artifacts, decompression time

  30. Results

  31. Proof of Concept TGA 1:1 (128x128)

  32. Proof of Concept JPG 24:1 (64x64)

  33. Proof of Concept Evolver 50:1, infinite detail resolution

  34. Proof of Concept TGA Evolver JPG

  35. Lake Matheson Original

  36. Lake Matheson Original Compressed 50:1

  37. Gradient Original Compressed

  38. Aspens Original

  39. Aspens Original Compressed 54:1

  40. Maples Original

  41. Maples Code Collage(HueShift(HueShift(Min(RockImage(), Collage(Rotate(0.22693111, {0.4339271, -0.060890462, -0.14983156}), {0.9689341, -0.31166452, 1.0}, EnvironmentMap(Interpolate(Derivative({0.5260445, -0.9943271, -0.83629435}), FishImage(), 0.22693111), Collage(VSplit(VGradient(), Interpolate(Min(RockImage(), Collage(0.22693111, {0.90638816, -0.3161332, 1.0}, EnvironmentMap({0.3538252, -0.11179591, 0.76402247}, {0.3538252, -0.11179591, 0.76402247}))), -0.75621074, Derivative(HueShift(LowColorNoise(), {-0.60136193, -0.9961748, 0.956824})))), {-0.6025814, -0.5151359, -0.2444776}, MiniBlur(Blur({0.48381335, 0.37744927, 0.18049468})))))), LeafImage()), LeafImage()), Rotate(Rotate(Interpolate(Max(Rotate(LeafImage(), Cosine({0.13357106, -0.48899084, 0.46273336})), LeafImage()), {0.26036343, -0.2474052, 0.3318561}, Add(Rotate(Interpolate(Max(LowNoise(), LeafImage()), {0.26036343, -0.2474052, 0.3318561}, Add(Rotate(Interpolate({0.26036343, -0.2474052, 0.3318561}, {0.26036343, -0.2474052, 0.3318561}, Max(Rotate(Blur(Noise()), FrequencyStars()), BitAnd(VSplit(LeafImage(), {-0.3782095, 0.06973941, 0.7708523}), Collage(Blur({0.19214037, 0.7060751, 0.9632803}), {-0.94875985, 0.9535051, 0.9628181}, {-0.94875985, 0.9535051, 0.9628181})))), FrequencyStars()), Zoom({0.84155905, 0.44450688, -0.6368634}, Interpolate(0.97325927, {0.43938103, 0.8003519, -0.8865588}, FrequencyStars())))), FrequencyStars()), Zoom({1.0, 0.35874906, -0.42753658}, {1.0, 0.35874906, -0.42753658}))), FrequencyStars()), {0.30287892, -0.7879979, 0.756324}), Distort(Distort(HueShift(Distort(Distort(Distort(HueShift(Distort(Rotate(Distort(Distort(HueShift(Distort(Rotate(Distort(Distort(Blur(Distort(Distort(RockImage(), ArcTangent(ExpandRange(ColorNoise()))), ArcTangent(ExpandRange(ColorNoise())))), ArcTangent(ExpandRange(ColorNoise()))), ArcTangent(ColorNoise())), RockImage()), ArcTangent(ColorNoise())), RockImage()), ColorNoise()), Blur(Add({0.214469, -0.05106278, -0.8334819}, Blur(Blur({-0.44914088, 0.86714524, -0.038012877}))))), Distort(Distort(Blur(Distort(Distort(-0.32682085, SunriseImage()), ArcTangent(0.84263784))), LeafImage()), ArcTangent(ColorNoise()))), ArcTangent(ColorNoise())), ExpandRange(RockImage())), 0.315652), ColorNoise()), Blur(ExpandRange(Add({0.44753784, 0.15750253, -0.9017423}, {0.214469, -0.05106278, -0.8334819})))), ExpandRange(RockImage())), ColorNoise()), Blur(Rotate({0.10588625, 0.2359776, -0.20337643}, {0.2809281, -0.97692156, -0.49766022}))))

  42. Maples Original Compressed 56:1

  43. Multi-resolution JPEG 14:1 Evolver 56:1

  44. Multi-resolution JPEG 14:1 Evolver 56:1

  45. Related work • Searching for programs • Massalin’s Superoptimizer • Frigo & Johnson’s FFTW • Palsberg, Lucier & Mamillapalli • Karl Sims • Programmatic image compression • Fractal compression • MPEG-7 • Steerable search techniques in graphics • MLT • Radiosity

  46. Conclusions • It is tractable to search the space of all programs! • The “visually similar” criterion makes computer graphics an interesting domain • Keep using JPEG for now…

  47. Future directions • Improve image quality/compression • How can the design of searchable languages be formalized? • How do expressive constructs affect the search problem? • Other interesting domains: animation, audio compression, image search, robot controllers

  48. Questions http://www.cs.brown.edu/people/morgan/evolver

More Related