programming languages for compressing graphics l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Programming Languages for Compressing Graphics PowerPoint Presentation
Download Presentation
Programming Languages for Compressing Graphics

Loading in 2 Seconds...

play fullscreen
1 / 48

Programming Languages for Compressing Graphics - PowerPoint PPT Presentation


  • 285 Views
  • Uploaded on

Programming Languages for Compressing Graphics Morgan McGuire Shriram Krishnamurthi John F. Hughes Brown University { morgan | sk | jfh}@cs.brown.edu Encoding Images as Programs Describing Images (1) setPixel(0,0,BLUE); setPixel(1,0,BLUE); setPixel(2,0,BLUE); … setPixel(50,0,WHITE); …

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Programming Languages for Compressing Graphics' - johana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
programming languages for compressing graphics

Programming Languages for Compressing Graphics

Morgan McGuire

Shriram Krishnamurthi

John F. Hughes

Brown University

{ morgan | sk | jfh}@cs.brown.edu

describing images 1
Describing Images (1)

setPixel(0,0,BLUE);

setPixel(1,0,BLUE);

setPixel(2,0,BLUE);

setPixel(50,0,WHITE);

setPixel(100,0,RED);

setPixel(101,0,RED);

the cost of bandwidth
The Cost of Bandwidth
  • Major cost of doing business on the web
    • Yahoo!: 65,000,000 pages/day
    • Valve: 1M €/software patch
  • Image compression is a one-shot activity
  • Low vs. high bandwidth users
  • Multiresolution
describing images 2
Describing Images (2)

repeat 50 times setNextPixel(BLUE);

repeat 50 times setNextPixel(WHITE);

repeat 50 times setNextPixel(RED);

nextRow();

repeat 50 times setNextPixel(BLUE);

american flag
American Flag

Doesn’t compress as well as the French flag in the “repeat n times” language.

adding more primitives
Adding More Primitives

drawRectangle(BLUE, …);

drawRectangle(RED, …);

drawRectangle(WHITE, …);

drawStar(WHITE, …);

Is this enough?

describing complex images9
Describing Complex Images
  • JPEG: language consists of frequency domain instructions
  • GIF: language consists of setPixel and dictionary lookup
  • Preferred format depends on the image
observations
Observations
  • Images are programs
  • Even within one language, many possible descriptions produce similar images
    • Lossy compression
  • Description length depends on language and image complexity
  • Best compression when the language matches the image
the obvious compression scheme
The Obvious Compression Scheme
  • Compress the image in several formats
    • TGA, GIF, JPG, SVG, SWF
  • Choose the best
  • Add a byte to the front of the file specifying the compression language
problems with the obvious scheme
Problems with the Obvious Scheme
  • None of the formats may be particularly good for our image
  • Even JPEG tops out around 50:1, has serious artifacts
  • Lacking ideal features like:
    • Multiresolution
    • Time/Space tradeoff
  • Introduction of new formats requires new browser plug-ins
describing complex images13
Describing Complex Images

How can we do better than the obvious approach for images like this?

using an expressive language
Using an Expressive Language

(compose

(vertical-gradient BLUE WHITE)

(polygon DARK-BLUE …)

(polygon BLACK …)

(* (polygon …)

(blur tree-texture)

YELLOW))

using an expressive language15
Using an Expressive Language
  • What if we design a really expressive language for representing images?
  • Because the “data” is a program the decompressor is part of the “data”
    • Each image gets its own custom format
    • High compression
  • We have control over multiresolution, perceptual artifacts
  • Package for the web as a plug-in
    • We only need to upgrade the plug-in when the language itself changes

What’s wrong with this plan?

example
Example

setPixel Language

Expressive Language

?

setPixel(0,0,WHITE);

setPixel(1,0,WHITE);

setPixel(2,0,WHITE);

(blur (+

(rotate (banana))

(distort (* RED

(circle))) …)

2 Mb

2 kb

the encoding problem
The Encoding Problem

?

Pierre can’t code!

the encoding problem18
The Encoding Problem

Photoshop menu

EMACS

setPixel(0,0,WHITE);

setPixel(1,0,WHITE);

setPixel(2,0,WHITE);

(blur (+

(rotate (banana))

(distort (* RED

(circle))) …)

2 Mb

2 kb

where is the save as code menu
Where is the “Save as Code” Menu?
  • It is easy to convert from an image to programs in the GIF/JPG language
  • More expressive language = harder conversion
  • How much harder?
much harder
Much Harder!
  • Converting the image into a program that produces it is a search problem
  • The search space is the space of all possible programs
  • This is an infinitely large space
tempering expressiveness
Tempering Expressiveness

“Good” compression languages are ones where:

  • Expressive power is large
  • Searching is easy

How do we make searching easy?

steerable search techniques
Steerable Search Techniques
  • Genetic Algorithms
    • Inject domain information through fitness function
  • Metropolis Search
    • Inject domain information through transition probabilities
  • Simulated Annealing
    • Inject domain information through gradient estimation
perceptual fitness function
Perceptual Fitness Function
  • Tweaking this is the domain-expert’s job
  • Perfect fitness function not necessary (or possible!)

e (i ) = edge filter, b (i ) = convolve with Gaussian, |ix| = color magnitude

designing the language
Designing the Language
  • Desirable language properties for compression
    • Expresses many images compactly
    • There are many programs for which another, shorter program exists that produces visually similar output
  • Desirable search properties
    • Mutations safety
    • All programs terminate
the evolver language
The Evolver Language

Scalar := real between 0 and 1

Vector := Scalar x Scalar x Scalar

Matrix := Vector*

Value := Matrix | Scalar | Vector

Operator := Add | Collage | Blur | Noise | …

Call := Operator x Expr*

Expr := Call | Scalar | Vector

the evolver language26
The Evolver Language
  • Automatic coercion between Matrix, Vector, Scalar
  • Every operator has the same domain and range
  • Primitives include stock images and textures
  • No looping constructs
  • No functions!
l not the ultimate compressor28
l: Not the Ultimate Compressor!
  • Copying code is sometimes good
    • Multiple instances of a pattern in an image often differ slightly
  • Hard to evolve both definition and multiple applications
added benefits of search
Added Benefits of Search
  • Artist can save the image immediately
  • Webmaster uses Evolver toolkit to search for an equivalent program
    • It is easy to find a large program quickly
  • Webmaster lets Evolver continue to run
  • Upload new, smaller encodings as they are found
  • More time = less space = less cost
  • Multiple constraints: size, time, artifacts, decompression time
proof of concept
Proof of Concept

TGA 1:1 (128x128)

proof of concept32
Proof of Concept

JPG 24:1 (64x64)

proof of concept33
Proof of Concept

Evolver 50:1, infinite detail resolution

proof of concept34
Proof of Concept

TGA

Evolver

JPG

lake matheson36
Lake Matheson

Original

Compressed 50:1

gradient
Gradient

Original

Compressed

aspens
Aspens

Original

aspens39
Aspens

Original

Compressed 54:1

maples
Maples

Original

maples code
Maples Code

Collage(HueShift(HueShift(Min(RockImage(), Collage(Rotate(0.22693111, {0.4339271, -0.060890462, -0.14983156}), {0.9689341, -0.31166452, 1.0}, EnvironmentMap(Interpolate(Derivative({0.5260445, -0.9943271, -0.83629435}), FishImage(), 0.22693111), Collage(VSplit(VGradient(), Interpolate(Min(RockImage(), Collage(0.22693111, {0.90638816, -0.3161332, 1.0}, EnvironmentMap({0.3538252, -0.11179591, 0.76402247}, {0.3538252, -0.11179591, 0.76402247}))), -0.75621074, Derivative(HueShift(LowColorNoise(), {-0.60136193, -0.9961748, 0.956824})))), {-0.6025814, -0.5151359, -0.2444776}, MiniBlur(Blur({0.48381335, 0.37744927, 0.18049468})))))), LeafImage()), LeafImage()), Rotate(Rotate(Interpolate(Max(Rotate(LeafImage(), Cosine({0.13357106, -0.48899084, 0.46273336})), LeafImage()), {0.26036343, -0.2474052, 0.3318561}, Add(Rotate(Interpolate(Max(LowNoise(), LeafImage()), {0.26036343, -0.2474052, 0.3318561}, Add(Rotate(Interpolate({0.26036343, -0.2474052, 0.3318561}, {0.26036343, -0.2474052, 0.3318561}, Max(Rotate(Blur(Noise()), FrequencyStars()), BitAnd(VSplit(LeafImage(), {-0.3782095, 0.06973941, 0.7708523}), Collage(Blur({0.19214037, 0.7060751, 0.9632803}), {-0.94875985, 0.9535051, 0.9628181}, {-0.94875985, 0.9535051, 0.9628181})))), FrequencyStars()), Zoom({0.84155905, 0.44450688, -0.6368634}, Interpolate(0.97325927, {0.43938103, 0.8003519, -0.8865588}, FrequencyStars())))), FrequencyStars()), Zoom({1.0, 0.35874906, -0.42753658}, {1.0, 0.35874906, -0.42753658}))), FrequencyStars()), {0.30287892, -0.7879979, 0.756324}), Distort(Distort(HueShift(Distort(Distort(Distort(HueShift(Distort(Rotate(Distort(Distort(HueShift(Distort(Rotate(Distort(Distort(Blur(Distort(Distort(RockImage(), ArcTangent(ExpandRange(ColorNoise()))), ArcTangent(ExpandRange(ColorNoise())))), ArcTangent(ExpandRange(ColorNoise()))), ArcTangent(ColorNoise())), RockImage()), ArcTangent(ColorNoise())), RockImage()), ColorNoise()), Blur(Add({0.214469, -0.05106278, -0.8334819}, Blur(Blur({-0.44914088, 0.86714524, -0.038012877}))))), Distort(Distort(Blur(Distort(Distort(-0.32682085, SunriseImage()), ArcTangent(0.84263784))), LeafImage()), ArcTangent(ColorNoise()))), ArcTangent(ColorNoise())), ExpandRange(RockImage())), 0.315652), ColorNoise()), Blur(ExpandRange(Add({0.44753784, 0.15750253, -0.9017423}, {0.214469, -0.05106278, -0.8334819})))), ExpandRange(RockImage())), ColorNoise()), Blur(Rotate({0.10588625, 0.2359776, -0.20337643}, {0.2809281, -0.97692156, -0.49766022}))))

maples42
Maples

Original

Compressed 56:1

multi resolution
Multi-resolution

JPEG 14:1

Evolver 56:1

multi resolution44
Multi-resolution

JPEG 14:1

Evolver 56:1

related work
Related work
  • Searching for programs
    • Massalin’s Superoptimizer
    • Frigo & Johnson’s FFTW
    • Palsberg, Lucier & Mamillapalli
    • Karl Sims
  • Programmatic image compression
    • Fractal compression
    • MPEG-7
  • Steerable search techniques in graphics
    • MLT
    • Radiosity
conclusions
Conclusions
  • It is tractable to search the space of all programs!
  • The “visually similar” criterion makes computer graphics an interesting domain
  • Keep using JPEG for now…
future directions
Future directions
  • Improve image quality/compression
  • How can the design of searchable languages be formalized?
  • How do expressive constructs affect the search problem?
  • Other interesting domains: animation, audio compression, image search, robot controllers
questions

Questions

http://www.cs.brown.edu/people/morgan/evolver