Low level plumbing for media integration l.jpg
Advertisement
This presentation is the property of its rightful owner.
1 / 21

Low-Level Plumbing for Media Integration PowerPoint PPT Presentation

Low-Level Plumbing for Media Integration Turner Whitted Microsoft Research Outline Part I: Implementation chronology Then, Now, Then again … aka Wheel Of Reincarnation Part II: Architectural musing Perceptual/content requirements Data paths, data types Not about programming models

Download Presentation

Low-Level Plumbing for Media Integration

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Low level plumbing for media integration l.jpg

Low-Level PlumbingforMedia Integration

Turner Whitted

Microsoft Research


Outline l.jpg

Outline

  • Part I: Implementation chronology

    • Then, Now, Then again …

    • aka Wheel Of Reincarnation

  • Part II: Architectural musing

    • Perceptual/content requirements

    • Data paths, data types

    • Not about programming models


Starting points e s frame buffer l.jpg

Starting points – E&S Frame Buffer

  • NO fixed function units

  • mCode for basic logic

  • mController programmable by designer only

  • Treated as peripheral

Ref: Kajiya, J.T., Sutherland, I.E., and Cheadle, E.C., "A Random-Access Video Frame Buffer," Proceedings of the Conference on Computer Graphics, Pattern Recognition, and Data Structure, UCLA Extension, Los Angeles, California, May 14-16, 1975


Starting points ikonas rds3000 l.jpg

Starting points – Ikonas RDS3000

  • Single, 32-bit wide datapath based on 2901 bit-sliced DSP

  • Programmed in C (Gary’s Ikonas Assemler)

  • NO fixed function units

  • Bound [later memory mapped] to single application

Ref: N. England, A graphics system architecture for interactive application-specific display functions, IEEE CGA, pp. 60-70, Jan 1986.


Starting points pixar chap l.jpg

Starting points – Pixar CHAP

  • SIMD processors

  • Loops, conditional execution

  • Focus on parallel programming issues

Ref: Adam Levinthal and Thomas Porter, “Chap – A SIMD Graphics Processor,” Proceedings of SIGGRAPH 84, (18) 3, July 1984, pp. 77 – 82.


The pipeline l.jpg

Host Bus

Example: AT&T Pixel Machine

One or, optionally two, 9-element

16, 20, 32, 40, or 64

geometry pipelines

Pixel Nodes

“The” pipeline

  • [Mostly] Fixed structure, programmable nodes

  • Vector graphics legacy


Structural evolution l.jpg

Structural evolution

  • Texture engine with remnants of line drawing DNA


Performance l.jpg

Performance

  • Ikonas RDS3000 (1980)

    • 20 MB/s processor to memory

    • 20 MIPS equiv.

  • Pixar CHAP (1984)

    • 240 MB/s processor to memory (P-bus)

    • 64 MIPS peak

  • ATI 9700 (2002)

    • 20800 MB/s chip to memory

    • 8400 MIPS


Api abstractions opengl dx l.jpg

API abstractions (OpenGL, DX)

  • Fixed structure pipeline accessed through API

    • API tracks hardware through several generations while maintaining consistency

  • Integrated with mainstream computing

  • Easy to program

    • Single largest key to commercial success of graphics systems and applications


Complete the circle l.jpg

Complete the circle

  • FPGA+memory

  • NO fixed function

  • Programmable in C (Verilog actually)

  • Good for simple prototyping

    • No API

    • No device drivers

    • No SDK


Part ii going forward l.jpg

Part II: Going forward …

  • Motivation

    • We’ve tacked every imaginable feature onto what was initially a line drawing pipeline

    • It’s time to start over

    • The window of opportunity is wide open


Going forward with graphics processors l.jpg

Alternatives

Feed eyeballs with

Integrated media

Real-time photorealism

Parallel execution for scientific applications

Parallel execution for interactive applications

Going forward …with graphics processors

3D Raster

Graphics


Integrated media partial illustration l.jpg

Integrated media(partial illustration)

Concentric mosaics

Image centric

Geometry centric

Sprites with depth

View-dependent

texture

View-dependent geometry

Light field

Lumigraph

Fixed geometry

Polygon rendering + texture mapping

Interpolation

Warping


Geometry image l.jpg

Geometry Image

completely regular sampling

3D geometry

geometry image257 x 257; 12 bits/channel

Ref: X. Gu, S. Gortler, H. Hoppe, “Geometry images,” ACM Transactions on Graphics 21(3): 355-361 (2002)


From first principles function l.jpg

From first principles: function

U-VWL* Content

Line drawing

Display Processor

Display Device

Video

Still photos

Animated 3D shapes

Text

*Ultra-Vast Wasteland


3d text experiment l.jpg

3D text experiment

  • Extend IBR/volume rendering to text

  • Superior image reconstruction

  • Higher image quality than mip-mapped texture

Olynyk, Mitchell, Snyder, MSR


Generic physical blocks l.jpg

General purpose front end

Fixed function back end

Generic physical blocks

Read

Mapper

Cache

Common

Memory

Recon-

Write

Struction/

Filtering

Cache

Ref: T. Whitted, “Overview of IBR:Hardware and Software Issues,” ICIP 2000.


From first principles implementation l.jpg

Don’t count on quantum GPUs soon – stick to CMOS digital logic

Count on CAD more than feature size

Heat is the enemy

The economy of commodity DRAM is hard to beat

But there is huge performance pressure on DRAM

Designers are restricted only by a lack of experience

CMOS digital circuitry

Commodity DRAM

From first principles: implementation

Content


Design challenges l.jpg

Design challenges

  • Essence of the problem

    • We don’t have a function to implement

    • We must design specifically for unknown methods

    • Brute force is prohibited

  • Feeds and speeds …What do we know?

    • Regulated by content

      • But we rarely turn content conventions into quantitative measures

    • Limited by perception

      • Which we don’t fully understand

    • Function of representation

      • If we don’t know the representation, we don’t know the flows

      • In my group’s research we limit flow to HW “sanity” and then work backwards


Summary l.jpg

Summary

  • Graphics hardware has nearly completed the circuit back to its starting point

    • Flexible, powerful, programmable

  • Media processing requirements extend beyond the classic 3D pipeline

  • Unusual window of opportunity

    • to match architecture with a broader range of applications and content


  • Login