Modern OpenGL ES 3.0 and 3.1 Lighting and Materials

1. Modern OpenGL ES 3.0 and 3.1Lighting and Materials Presenters: Ron Fosner & Rudy Cazabón

2. Overview • What is the Mission of the Modern OpenGL Graphics Pipeline? • Definitions and Terminology • Lights – Physical Motivation for Illumination • Materials - Physical Motivation for Visible Surface Phenomena • “Classic” OpenGL Materials and Lighting • Modern OpenGL Implementations of Materials and Lights with GLSL • Non-Photorealistic Rendering - Alternative Illumination and Shading Models

3. What is the Mission of the Modern OpenGL Graphics Pipeline? • Present a modular framework • Describe sources of illumination in a scene • Bind lighting functions and data for visible surface phenomena to groups of geometries in a scene float NdotL,NdotHV; lightDir = vec3(gl_LightSource[0].position); vec4 color = ambient; n = normalize(normal); NdotL = max(dot(n,lightDir),0.0);

4. Definitions and Terminology • Illumination - the transport of light from a source to a point via direct and indirect paths • Illumination Models: • Empirical - approximations to observed light properties • Physically based - applying physics properties of light and its interactions with matter • Lighting - computing the luminous intensity for a specified 3D point, given a viewpoint • Material – In OpenGL domain, refers to the surface properties that determines how it shall interact with N-lights in a scene • Shading - assigning colors to pixels

5. Modern OpenGL with GLSLWhat We Know Thus Far We already know how to render the world from a viewpoint.

6. Lights and Materials in the Modern OpenGL Context • There are two components that go into the final color of a pixel • 1. Lighting Model or Shading Model How we calculate the intensity at a point on the surface • 2. Surface Rendering Method – How we calculate the intensity at each pixel

7. Simple OpenGL ES shader-based Pipeline GPU Data Flow Application Framebuffer Rasterizer Fragment Processing Vertex Processing Vertex Shader Fragment Shader Shader Code Shader Code

8. Lights – A Physical Motivation for Illumination • Lighting is one of the most complex aspects of the rendering process • No-one has yet come up with a "perfect" simulation of rendering for use in real-time graphics - even non-real-time graphics haven't really solved the problem for every material • Lighting is required as it conveys the sense of surface roughness/smoothness and material visual appearance

9. …In the Beginning - Fiat Lux Maxwell Equations- Integral Form Maxwell Equations - Differential Form Rendering Equation Simplifying Assumptions Illumination

10. A Reasonable OpenGL Illumination Model Illumination Model Attenuation = Distance-based decay factor

11. Materials - Physical Motivation for Visible Surface Phenomena • “Materials” in CG refers to the group of properties that model the phenomena of reflection, refraction, absorption, and emission as it interacts with illumination • Equally hard as light to model!

12. Reflection Terms N R L V 

13. Calculation of per-color Component • The contribution to the object’s color from each light source is given by: • C = (att)(spot)[acm*acli+ dcm*dcli*dot(L,N) + (att)srmscm*scli*dot(R,N)] • respectively, and acm, acli, dcm, dcli, scm, and scli are the ambient material color, ambient light color, diffuse material color, diffuse light color, specular material color, and specular light color, respectively. Finally, att is the light source attenuation factor, spot is the spotlight contribution

14. Ambient + Diffuse + Specular

15. Fixed-function OpenGL Per-vertex Lighting • “Classic” OpenGL had a fixed lighting model • Operated at the per-vertex level • Referred to as the Blinn-Phong lighting model • Example usage glEnable(GL_LIGHTING); glEnable(GL_LIGHT0); glLightfv(GL_LIGHT0, GL_POSITION, position); glLightfv(GL_LIGHT0, GL_DIFFUSE, diffuse); glLightfv(GL_LIGHT0, GL_SPECULAR, specular); • Relied on • Per-vertex normal value sent with glNormal3f • And per-vertex material properties sent with glMaterialfv

16. Fixed-function OpenGL Materials • “Classic” OpenGL material properties assigned to vertices • Lighting calculations were done at vertices only, and the results of those calculations were interpolated to other points of the object • Material properties are set using the glMaterial* family of commands, principallyglMaterialfvand glMateriali. • Example usage: glMaterialfv(GL.GL_FRONT, GL.GL_SPECULAR, new float[] { 0.75f, 0.75f, 0.75f, 1 }, 0); glMaterialfv(GL.GL_FRONT, GL.GL_AMBIENT_AND_DIFFUSE, new float[] { 0.75f, 0.75f, 0, 1 }, 0); glMateriali(GL.GL_FRONT, GL.GL_SHININESS, i*16);

17. Modern OpenGL Implementations of Materials and Lights with GLSL • It is about programmable stages • Vertex Shader • Operates on each vertex of geometric primitives • Passes through per-vertex color • Transforms the vertex to match fixed-function processing • Fragment Shader • Operates on each fragment (think pixel) • Outputs the fragment’s interpolated color to the framebuffer void main(void){ gl_FrontColor = gl_Color; gl_Position = ftransform(); } void main(void){ gl_FragColor = gl_Color; }

18. Light Source Properties • Color - We usually assume the light has one wavelength • Point Light - approximate the light source as a 3D point in space. Light rays emanate in all directions. • Directional - approximating the light source as coming from infinity. Light rays are parallel • Spotlights – approximates a nearby light at a specific point in space, a direction vector, a fall-off function from central direction vector and distance structLightInfo { vec4 Position; // Light position in eye coords. vec3 La; // Ambient light intensity vec3 Ld; // Diffuse light intensity vec3 Ls; // Specular light intensity }; uniform LightInfo Light;

19. Attenuation • Energy from a light source falls off proportional to 1/d2. • Using only 1/d2, makes it difficult to correctly light things. Think if d=1 and d=2. Why? • We are approximating things. Lighting model is too simple AND most lights are not point sources

20. Contributions from lights • Breakdown what a light does at a surface into three different components. This APPROXIMATES what a light does. • Illumination at a pixel from a light • Intensity = Ambient + Diffuse + Specular contributions. Ilight= Iambient+ Idiffuse + Ispecular #version 300 Void main() { out vec3 LightIntensity; uniform vec3 Iambient; uniform vec3 Idiffuse; uniform vec3 Ispecular; <stuff happens here> LightIntensity = Iambient + Idiffuse + Ispecular; }

21. Contributions from materials • How a surface reflects into three different components. • Reflectance at a pixel from a reflected point • Intensity = Ambient + Diffuse + Specular contributions structMaterialInfo { vec3 Ka; // Ambient reflectivity vec3 Kd; // Diffuse reflectivity vec3 Ks; // Specular reflectivity float Shininess; // Specular shininess factor }; uniform MaterialInfo Material;

22. Quick overview of textures • Say you have this triangle, with these texture coordinates. • They go from 0 to 1 in X and Y*, with the origin being at the bottom left by convention. * There can also be 1 or 3 dimensional textures, but the coordinates all go 0 - 1

23. Quick overview of textures • And you have this texture loaded. • Note that it’s texture cords also start in the lower left corner.

24. Quick overview of textures • When you align the triangle’s and the texture’s coordinates, the align like this.

25. Quick overview of textures • When the triangle is rendered, you just the part of the image that is aligned with the triangle’s texture coordinates. • Textures are used to add details to a trianglewithout using a denserset of triangles.

26. Quick overview of textures • If you multiply the vertex texture coordinates by 4, then you can wrap the images across the triangle multiple times.* 4 images 4 images * Assuming texture coordinate wrapping is how the state is set!

27. Contributions from Textures • How a surface reflects into three different components. • Reflectance at a pixel from a reflected point • Intensity = Ambient + Diffuse + Specular contributions uniform sampler2D uSourceTex; structMaterialInfo { <previous stuf> attribute vec2 aPosition; attribute vec2 aTexCoord; varying vec2 vTexCoord; }; uniform MaterialInfo Material;

28. Shading • Shading is how we “color” a triangle. • Constant Shading • Gouraud Shading • Phong Shading

29. Constant Shading • Constant Intensity or Flat Shading • Per-triangle normal • One color for the entire triangle • Fast • Good for some objects • What happens if triangles are small? • Sudden intensity changes at borders

30. Gouraud Shading • Compute the color at each vertex first • Compute the color inside the polygon by interpolating the colors of the vertices composing the polygon

31. Gouraud Shading • Relatively fast, only do three calculations • No sudden intensity changes • Problem with Gouraud shading for larger triangles: • If light is near-middle of triangle it is possible to not have lights/highlights in the middle as intended

32. Phong Shading • Phonglighting model is object-space illumination model • It considers four types of lightings: diffuse, specular, ambient and emissive. • Requires interpolation of the normal as it represents the “curvature” • Performed foreachpixel, as during scan convert • This Calculatesthe lighting per pixel normal.

33. Comparison of Techniques • Flat shading is easy but results are bad • Gouraudshading is usually used for simple scenes • Phongshading is better visually and with modern HW it is computationally feasible

34. Shading Terms Implemented in GLSLAmbient

35. Shading Terms Implemented in GLSLDiffuse

36. Shading Terms Implemented in GLSLTexturing

37. Non-Photorealistic Rendering • Nearly all of the energy and efforts in computer graphics is towards achieving “photorealistic” or “real-world” representations for rendered objects • Non-Photorealistic Rendering (NPR) is an area of computer graphics that engages the mathematics and techniques of rendering to the goal of arriving at rendering methods that expressively stylized to painting, drawing, or drafting • The end-results of these NPR renderings may be artistic or have a “hard usage” such as technical illustrations

38. Examples of NPR Cartoon/Cel Shading • Gooch, Amy and Bruce, et. Al Salesin, D. and Lee Markosian, et. Al Graftals

39. Back up slides