- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Abstract Shade Trees Morgan McGuire1 George Stathis2 Hanspeter Pfister2,3 Shriram Krisnamurthi1 1Brown University 2Harvard Extension School 3MERL

2. [Proudfoot01] Shaders • Programs that color (“shade”) a pixel • Often execute on a graphics card • Pervasive in film and game rendering

3. [Proudfoot01] Shaders • Programs that color (“shade”) a pixel • Often execute on a graphics card • Pervasive in film and game rendering

4. Let’s author a pixel shader

5. Shading Language (Renderman, RTSL, HLSL, GLSL, Cg) // glass_ball.glsl uniform float etaRatio; uniform float fresnelPower; uniform samplerCube environmentCubeMap; varying vec3 objectSpaceIncomingVector; varying vec3 n, b,t; uniform sampler2D normalMap; varying vec3 tangentSpaceViewVector; varying vec2 textureCoordinates; void main(void) { float bump = ((texture2D(heightFieldTexture, textureCoordinates).a) + bumpBias) * bumpScale; vec2 bumpCoords = bump * vec2(tangentSpaceViewVector.x, tangentSpaceViewVector.y) + textureCoordinates; vec3 tangentSpaceNormalVector = texture2D(normalMap, vec2(bumpCoords.x,bumpCoords.y)).xyz * 2.0 - 1.0; mat3 tangentObjectSpaceMat = mat3(t,b,n); vec3 objectSpaceNormalVector = tangentObjectSpaceMat*tangentSpaceNormalVector; vec3 objectSpaceReflectedVector = normalize(reflect(objectSpaceIncomingVector,objectSpaceNormalVector)); vec4 color1 = textureCube(environmentCubeMap, objectSpaceReflectedVector); float fresnelRatio = max(0.0,min(1.0, fresnelBias + fresnelScale* pow(1.0 + dot(normalize(objectSpaceIncomingVector), normalize(objectSpaceNormalVector)),fresnelPower))); vec3 objectSpaceRefractedVector = normalize(refract(objectSpaceIncomingVector,objectSpaceNormalVector,etaRatio)); vec4 color2 = textureCube(environmentCubeMap, objectSpaceRefractedVector); vec4 mixedColorOutput = mix(color1,color2,fresnelRatio); gl_FragColor = mixedColorOutput; }

6. Parameter Function call Shade Tree for Visual Programming [Cook84; Abram+Whitted90; Borau04; Unreal06; Hargreaves04]

7. Authoring in a production context…

8. Artist Programmer Shader Authoring Problems • Dependent workflow • Artists define and use • Programmers implement • Frequently block on each other • Shaders are rarely modular • Overlapping features [Dijkstra76] • Exacerbated by restrictive execution environments • Programmers frequently rewrite from scratch

9. Artist Programmer Shader Authoring Problems • Dependent workflow • Artists define and use • Programmers implement • Frequently block on each other • Shaders are rarely modular • Overlapping features [Dijkstra76] • Exacerbated by restrictive execution environments • To combine features, programmers frequently rewrite both from scratch void main() {

10. New authoring idea…

11. Parallax Bump Mapping Reflection Refraction Superposition Abstract Shade Tree Abstract… Shade Tree

12. Parallax Bump Mapping Reflection Refraction Superposition Abstract Shade Tree • Node = Code atom • Like functions with funny scopes (See the paper for details) • Arrow = Dependency • Not parameters! • Outline = Feature • Note the overlap at Fresnel • Requires a weaver to convert into executable code

13. New Workflow • Programmer writes atoms • Both can visually combine atoms into features • Artist visually combines features into trees • Weaver synthesizes: • Shader entry and exit points • Program structure • Parameter linkage • Type coercions • “Glue” code • Rendered in IDE for feedback

14. Problems Addressed • Workflow dependency reduced • Programmer writes atoms once • Artist can independently generate many shaders • Abstract Tree is independent of atom signatures • Modularity improved • Visualized feature interaction • Weaver resolves overlapping features • Abstract Tree is independent of atom signatures

15. Related Feature Work • Features as abstractions [Parnas72; Dijkstra76; Turner99] • Feature/Aspect-oriented programming [Batory92; Kiczales97; Batory04] • Software product lines [Clements02] • Sh Language, Combining shaders [McCool et al. 02, McCool et al. 04] • Über-Shader alternative [Barzel97/Gritz98; Pellacini+Vidimce04; McGuire05]

16. The Weaver

17. Weaving Algorithm • Input: Abstract shade tree • Output: GLSL shader • -Rename parameters • Topologically sort nodes • Match inputs to outputs • Generate code • Generate boilerplate

18. Match Inputs to Outputs Normal Map Some node in the middle of an Abstract Shade Tree Lambertian Shading

19. Normal Map Unpack Transform Normalize Lambertian Shading Match Inputs to Outputs The abstracted parameters the Weaver must reconstruct

20. Upstream Output Globals Normal Map Storage Class Change Unpack Basis Change Transform Normalize Lambertian Shading Match Inputs to Outputs The abstracted parameters the Weaver must reconstruct

21. Matching Algorithm • Bind each input parameter to the “closest” matching output parameter of some ancestor node • “Closest” = shortest/fastest type coercion • Typical GLSL types are just storage classes • e.g., float3  vector  color  point  … • float LambertianShading(float3 normal, float3 light) • Not much information in “float3” • Need a richer type system…

22. Semantic Types TLK C/GLSL Java Semantic Types • Encode information like “world-space vector” in types • Two variables with precisely the same semantic type are likely semantically interchangeable 1  Number of types Too general  Useful Too specific

23. Some Semantic Types for Shading Dimension: {2, 3, 4, any } Length: {Unit, any } Basis: {Tangent, Object, World, Screen, any } Interpretation: {Color, Texcoord, Normal, Point, Vector, any} Precision: {float32, int32, Boolean, any } • Type coercion now includes normalization, basis change, unpacking, storage change, etc. • “any” allows polymorphism • Could add “meters,” “non-negative,” etc. Float1 LambertianShading( UnitWorldNormalFloat3 normal, UnitWorldVectorFloat3 light)

24. Parameter Matching is Semantic Coercion Tree Search

25. Weaver Properties • Always terminates • Bounded (and small) memory requirements • Always generates a legal program • …but sometimes it is the wrong program • Most common error: Parameter aliasing • Artist missed a dependency • Programmer assigned an overly generic type • Anecdotally infrequent

26. Results

27. Bumpy Glass

28. Bumpy Glass User experience?

29. IDE

30. Phong Illum. + Parallax Map + Texture Map

31. Anisotropic + Diffuse Lighting

32. Phong Illum. + Projective Light + Shadow Map

33. Summary of Contributions • Abstract shade trees • Semantic type system • Weaver for shader synthesis • Visual editor

34. Future Directions • Consider performance model during parameter matching • Beyond pixel shaders • Other interesting semantic types • How do you debug abstract trees? • Applications to general scripting

35. Thanks • NVIDIA • MERL • NSF • Iron Lore Entertainment