Ying Zhu Georgia State University

1. Computer Graphics Algorithms Advanced Texture Mapping Techniques:displacement mapping, multi-texture, bump mapping, and projective texture mapping Ying Zhu Georgia State University

2. Heightmap and displacement mapping • Heightmap is an image used to store surface elevation data for display in 3D computer grahics. • In displacement mapping, a heightmap can be used to displace the position of vertices. • In bump mapping, a heightmap can be used for generating the normal map.

3. Heightmap

4. Heightmap • Heightmaps are widely used in games, terrain rendering software, GIS, scientific visualization, etc. • Another name for heightmaps is “digital elevation models” • Heightmaps are good for storing digital terrain elevations because they require less memory for a given level of detail than regular polygon mesh models.

5. Heightmap • How to generate a heightmap? • Terragen • Picogen • Many video games have editors for generating heightmaps • See http://en.wikipedia.org/wiki/Heightmap for a list

6. Displacement mapping in OpenGL/GLSL • Create a heightmap • Load the heightmap in an OpenGL program, and then pass it to the vertex shader as a texture image. • In the vertex shader, retrieve an RGB or RGBA vector from the heightmap for the vertex. • This is called vertex texture.

7. Displacement mapping • Convert the RGB or RGBA vector to a grayscale value. This is the amount of displacement for this vertex. • Multiply the grayscale value to the normal vector of this vertex, and then add the result to the current vertex position. • The normal vector guides the direction of the displacement. • Multiply the new vertex position with the model-view-projection matrix.

8. Displacement mapping • After vertex displacement, the original normal is no longer valid. • You may need to recalculate the normal vectors, or create a normal map from the heightmap, or simply don’t do lighting at all. • http://en.wikipedia.org/wiki/Displacement_mapping

9. Displacement mapping

10. Multi-texture mapping • Multi-texture is quite easy with GLSL • Load multiple texture images in the OpenGL program and then pass them to the shader. • In the pixel shader, you can multiple, add, or mix the texture colors from multiple texture images. • http://www.opengl.org/wiki/Multitexture_with_GLSL

11. Multi-texture mapping

12. Bump mapping • Create the illusion of a bumpy, uneven surface without using a lot of small polygons • Bump mapping is actually a per-pixel lighting technique

13. Bump mapping • Basic idea • Create a normal map that stores the normals for each pixel • The “bumpiness” is encoded in the normal map. • Load the normal map in the OpenGL program and then pass the normal map to the shader program as a texture image. • In the fragment shader, retrieve the normal for the pixel from the normal map and then calculate lighting color with this normal vector. • The key idea: Instead of interpolating the normals from the vertices, the normals are retrieved from the normal map.

14. Normal maps

15. Normal maps • How to create a normal map? • Convert a picture to a normal map using image editors (e.g. Photoshop, GIMP) • Create a high resolution version and a low resolution version of the same 3D model, and then generate a normal map from the high resolution version for the low resolution version. • Advanced modeling tools: Maya, Blender, etc. • Nvidia’s Melody (http://developer.nvidia.com/object/melody_home.htm)

16. Normal maps • How to generate normal map in GIMP? • Install GIMP (http://gimp-win.sourceforge.net/) • Download the GIMP Normalmap plugin (http://code.google.com/p/gimp-normalmap/) and follow the instructions in the readme file to install it. • Open an image in GIMP. • From the menu, choose Filters  Map  Normalmap • Change the “Scale” value if you want to increase the “bumpiness”. • Create the normal map and save it to an image file.

17. Simple bump mapping • Bump mapping on a flat surface is quite easy • Simply use the normal retrieved from the normal map for per-pixel lighting calculation. • There is no need to transform the normal. • This is because the coordinate system of the normal map and the coordinate system of the pixel is the same.

18. Simple bump mapping

19. General bump mapping • Bump mapping on a curved surface is more complicated • Retrieve the normal from the normal map • Transform the pixel position to the tangent space (i.e. the normal map’s coordinate space) • Calculate lighting in the tangent space. • This is called tangent space bump mapping

20. General bump mapping • Tutorials on the tangent space bump mapping • http://wiki.gamedev.net/index.php/OpenGL:Tutorials:GLSL_Bump_Mapping • http://www.blacksmith-studios.dk/projects/downloads/bumpmapping_using_cg.php • http://http.developer.nvidia.com/CgTutorial/cg_tutorial_chapter08.html

21. Projective texture mapping • Project a texture from a (sometimes invisible) projector. • Projective texture mapping is the basis for shadow mapping, lightmap, texture decal, etc.

22. Projective texture mapping

23. Projective texture mapping • In regular texture mapping, the texture coordinates from each vertex is static, often calculated in the modeling software such as Maya, 3DS Max, or Blender. • In projective texture mapping, the texture coordinates are calculated dynamically from the position of the vertex. • Ignore the texture coordinates provided by the 3D model file.

24. Projective texture mapping • The basic idea • Suppose you look from the projector’s viewpoint, you will always see the texture image -- front and center. • If you can project a 3D vertex to this 2D image, then the 2D location of the projected vertex is its projective texture coordinate. • So the key is to construct a matrix to project any vertex to a 2D image from the projector’s viewpoint.

25. Projective texture mapping • The steps: • Construct a model-view-projection-bias matrix from the projector's viewpoint, using GLM’s lookat() and perspective() functions. • A scale and bias matrix is used to scale the eventual texture coordinates to [0, 1] range. • Pass this projectorMVPB matrix to the vertex shader.

26. Projective texture mapping • The steps (continued): • In the vertex shader, multiply the vertex position with this projectorMVPB matrix, and then use the first two component's of the transformed position as the texture coordinates for this vertex. • Pass the texture coordinates to the fragment shader. • Carry out texture mapping in the fragment shader as usual.

27. Projective texture mapping • Since the texture may be clamped at the border, it’s better to use a texture with a black border.

28. Projective texture mapping • An (old) tutorial on projective texture mapping from Nvidia • http://developer.nvidia.com/object/Projective_Texture_Mapping.html • This tutorial describes the matrices for transforming the vertex position to projective texture coordinates.