low level plumbing for media integration l.
Skip this Video
Loading SlideShow in 5 Seconds..
Low-Level Plumbing for Media Integration PowerPoint Presentation
Download Presentation
Low-Level Plumbing for Media Integration

Loading in 2 Seconds...

play fullscreen
1 / 21

Low-Level Plumbing for Media Integration - PowerPoint PPT Presentation

  • Uploaded on

Low-Level Plumbing for Media Integration Turner Whitted Microsoft Research Outline Part I: Implementation chronology Then, Now, Then again … aka Wheel Of Reincarnation Part II: Architectural musing Perceptual/content requirements Data paths, data types Not about programming models

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Low-Level Plumbing for Media Integration

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
low level plumbing for media integration

Low-Level PlumbingforMedia Integration

Turner Whitted

Microsoft Research

  • Part I: Implementation chronology
    • Then, Now, Then again …
    • aka Wheel Of Reincarnation
  • Part II: Architectural musing
    • Perceptual/content requirements
    • Data paths, data types
    • Not about programming models
starting points e s frame buffer
Starting points – E&S Frame Buffer
  • NO fixed function units
  • mCode for basic logic
  • mController programmable by designer only
  • Treated as peripheral

Ref: Kajiya, J.T., Sutherland, I.E., and Cheadle, E.C., "A Random-Access Video Frame Buffer," Proceedings of the Conference on Computer Graphics, Pattern Recognition, and Data Structure, UCLA Extension, Los Angeles, California, May 14-16, 1975

starting points ikonas rds3000
Starting points – Ikonas RDS3000
  • Single, 32-bit wide datapath based on 2901 bit-sliced DSP
  • Programmed in C (Gary’s Ikonas Assemler)
  • NO fixed function units
  • Bound [later memory mapped] to single application

Ref: N. England, A graphics system architecture for interactive application-specific display functions, IEEE CGA, pp. 60-70, Jan 1986.

starting points pixar chap
Starting points – Pixar CHAP
  • SIMD processors
  • Loops, conditional execution
  • Focus on parallel programming issues

Ref: Adam Levinthal and Thomas Porter, “Chap – A SIMD Graphics Processor,” Proceedings of SIGGRAPH 84, (18) 3, July 1984, pp. 77 – 82.

the pipeline

Host Bus

Example: AT&T Pixel Machine

One or, optionally two, 9-element

16, 20, 32, 40, or 64

geometry pipelines

Pixel Nodes

“The” pipeline
  • [Mostly] Fixed structure, programmable nodes
  • Vector graphics legacy
structural evolution
Structural evolution
  • Texture engine with remnants of line drawing DNA
  • Ikonas RDS3000 (1980)
    • 20 MB/s processor to memory
    • 20 MIPS equiv.
  • Pixar CHAP (1984)
    • 240 MB/s processor to memory (P-bus)
    • 64 MIPS peak
  • ATI 9700 (2002)
    • 20800 MB/s chip to memory
    • 8400 MIPS
api abstractions opengl dx
API abstractions (OpenGL, DX)
  • Fixed structure pipeline accessed through API
    • API tracks hardware through several generations while maintaining consistency
  • Integrated with mainstream computing
  • Easy to program
    • Single largest key to commercial success of graphics systems and applications
complete the circle
Complete the circle
  • FPGA+memory
  • NO fixed function
  • Programmable in C (Verilog actually)
  • Good for simple prototyping
    • No API
    • No device drivers
    • No SDK
part ii going forward
Part II: Going forward …
  • Motivation
    • We’ve tacked every imaginable feature onto what was initially a line drawing pipeline
    • It’s time to start over
    • The window of opportunity is wide open
going forward with graphics processors

Feed eyeballs with

Integrated media

Real-time photorealism

Parallel execution for scientific applications

Parallel execution for interactive applications

Going forward …with graphics processors

3D Raster


integrated media partial illustration
Integrated media(partial illustration)

Concentric mosaics

Image centric

Geometry centric

Sprites with depth



View-dependent geometry

Light field


Fixed geometry

Polygon rendering + texture mapping



geometry image
Geometry Image

completely regular sampling

3D geometry

geometry image257 x 257; 12 bits/channel

Ref: X. Gu, S. Gortler, H. Hoppe, “Geometry images,” ACM Transactions on Graphics 21(3): 355-361 (2002)

from first principles function
From first principles: function

U-VWL* Content

Line drawing

Display Processor

Display Device


Still photos

Animated 3D shapes


*Ultra-Vast Wasteland

3d text experiment
3D text experiment
  • Extend IBR/volume rendering to text
  • Superior image reconstruction
  • Higher image quality than mip-mapped texture

Olynyk, Mitchell, Snyder, MSR

generic physical blocks
General purpose front end

Fixed function back end

Generic physical blocks











Ref: T. Whitted, “Overview of IBR:Hardware and Software Issues,” ICIP 2000.

from first principles implementation
Don’t count on quantum GPUs soon – stick to CMOS digital logic

Count on CAD more than feature size

Heat is the enemy

The economy of commodity DRAM is hard to beat

But there is huge performance pressure on DRAM

Designers are restricted only by a lack of experience

CMOS digital circuitry

Commodity DRAM

From first principles: implementation


design challenges
Design challenges
  • Essence of the problem
    • We don’t have a function to implement
    • We must design specifically for unknown methods
    • Brute force is prohibited
  • Feeds and speeds …What do we know?
    • Regulated by content
      • But we rarely turn content conventions into quantitative measures
    • Limited by perception
      • Which we don’t fully understand
    • Function of representation
      • If we don’t know the representation, we don’t know the flows
      • In my group’s research we limit flow to HW “sanity” and then work backwards
  • Graphics hardware has nearly completed the circuit back to its starting point
    • Flexible, powerful, programmable
  • Media processing requirements extend beyond the classic 3D pipeline
  • Unusual window of opportunity
    • to match architecture with a broader range of applications and content