NPR Today

1. NPR Today • “Stylized Rendering Techniques for Scalable Real-Time 3D Animation”, Adam Lake, Carl Marshall, Mark Harris and Marc Blackstein, NPAR 2001 • "Computer-Generated Pen-and-Ink Illustration of Trees", Oliver Deussen and Thomas Strothotte, SIGGRAPH 2000 • "Sketchy Drawings", Marc Nienhaus and Jurgen Dollner, AfriGraph 2004 © 2005 University of Wisconsin

2. Stylized Rendering Techniques for Scalable Real-Time 3D Animation Paper Overview Leo Chao © 2005 University of Wisconsin

3. Paper Information • Published • NPAR 2001 • (Non-Photorealistic Animation and Rendering) • Authors • Adam Lake • Carl Marshall • Mark Harris • Marc Blackstein • Graphics Algorithms and 3D Technologies Group • Intel Architecture Labs • UNC – Chapel Hill © 2005 University of Wisconsin

4. Paper Information • Prior Work • No novel techniques, adapts from prior works to hardware • Motion Lines • Maic Masuch. “Speedlines: Depicting Motion in Motionless Pictures.” ACM SIGGRAPH 99 Technical sketch. • Siu Chi Hsu and Irene H. H. Lee. "Drawing and Animation using Skeletal Strokes". In Proceedings of ACM SIGGRAPH 94, pages 109-118. 1994. • Silhouette Edges • Lee Markosian, Michael Kowalski, Samuel Trychi, Lubomir Bourdev, Daniel Goldstein, and John Hughes. Real-Time Nonphotorealistic Rendering. In Proceedings of ACM SIGGRAPH 97, pages 113-122. 1997. • Cartoon Shading • Philippe Decaudin. Rendu de scenes 3D imitant le style dessin anime. Rapport de Recherche 2919, Institut National de Recherche en Informatique et en Automatique. 1996. © 2005 University of Wisconsin

5. Contributions • Hardware Implementation of NPR Techniques • Stylistic Shading • Cartoon Shading • Pencil Sketch Shading • Stylistic Inking • Silhouette Edge Detection • Stylized Silhouette Rendering • Motion Lines © 2005 University of Wisconsin

6. Stylistic Shading • Fundamentals • Preprocess colour textures • Render as normal • Except: • Modify lighting/shading calculations • May require texture coordinate processing • “Painter” – Modified Shading Step • Cartoon Shading • Pencil Sketch Shading © 2005 University of Wisconsin

7. Stylistic Shading • Cartoon Shading • Reducing detail aids comprehension • No Smooth Shading • Color Reduction • Hard Transitions • Variants Simple © 2005 University of Wisconsin

8. Stylistic Shading © 2005 University of Wisconsin

9. Stylistic Shading • ALGORITHM Cartoon Shade • Preprocess: • Calculate the illuminated diffuse color for each material: • 2. Calculate the shadowed diffuse color: • 3. For each material, create a texture with two colors • - u=1 is coloured Ci • - u=0 is coloured Cs Ci Vertex Color Cs Shadow Diffuse Color al Light Ambient dl Light Diffuse am Material Ambientdm Material Diffuse ag Coefficient of Global Ambient Lighting © 2005 University of Wisconsin

10. Stylistic Shading • ALGORITHM Cartoon Shade • Runtime: • Calculate the one-dimensional texture coordinate at each vertex • 2. Render the model using a standard graphics API • - Lighting disabled • - Enable texture mapping L normalized light vector n unit surface normal © 2005 University of Wisconsin

11. Stylistic Shading • Pencil Sketch Shading • Extends cartoon shading to 2D textures • Extends cartoon shading to multiple textures • Multiple textures to represent light density • Associate texture coordinates • Project texture from the viewport onto the model © 2005 University of Wisconsin

12. Stylistic Shading © 2005 University of Wisconsin

13. Stylistic Inking • Fundamentals • Identify “important” edges • Silhouette • Border • Crease • Modify drawing of these edges • “Inker” – Modified Line Drawing • Visually important edge detection • Stylized edge drawing © 2005 University of Wisconsin

14. Stylized Inking • Visually important edge detection • Silhouettes • Front-Face and Back-Face share an edge • Borders • Edge shared by two materials • Unshared polygon edge • Creases • Large dihedral angle between polygons © 2005 University of Wisconsin

15. Stylized Inking © 2005 University of Wisconsin

16. Stylized Inking • ALGORITHM SED • Preprocess: • Create edge-list structure with Edge Type flags • Create a unique edge list using a hash table. • Edge Hash Function: sum the two vertex indices • Border Flag • Edges with only one neighboring face • Two neighboring faces with different materials • Set crease flags • Non-deformable geometry • Dihedral angle of two faces > threshold © 2005 University of Wisconsin

17. Stylized Inking • ALGORITHM SED • Runtime: • For deformable meshes: • Calculate face normals • Detect crease edges from face normals • Set Crease Edge flags • 2. Detect silhouette edges • a. © 2005 University of Wisconsin

18. Stylized Inking • Stylized “Important” Lines • Emphasis on these lines • Option 1: Straight Line • Fast/Easy • Option 2: Texture Maps • “Artistic” Style • Curvature driven “strokes” • Involves a graph driven search • This is a HACK © 2005 University of Wisconsin

19. Stylized Inking © 2005 University of Wisconsin

20. Motion Lines • Gives the appearance of motion • For fast moving objects • Simple algorithm • Not much overhead • One circular buffer to maintain position data © 2005 University of Wisconsin

21. Performance • Claim: Maintain Real-Time Speeds • 30 fps, in 2001 • Tests: • Integrated with a level of detail scheme • Altered model complexity to maintain FPS © 2005 University of Wisconsin

22. Performance © 2005 University of Wisconsin

23. Future Work • Inker • Better “artistic” lines • Painter • Pencil sketch “doesn’t look right” • Follow surface gradient perhaps • Incorporate other NPR techniques • Those used by artists • Doesn’t say which would amenable © 2005 University of Wisconsin

24. Questions? • ques·tion    (kw s ch n)n. • An expression of inquiry that invites or calls for a reply. • heck·le   (h k l)tr.v. • To try to embarrass and annoy (someone speaking or performing in public) by questions, gibes, or objections © 2005 University of Wisconsin

25. © 2005 University of Wisconsin

26. “Computer-Generated Pen-and-Ink Illustration of Trees”(SIGGRAPH 2000) Oliver Deussen Thomas Strothotte © 2005 University of Wisconsin

27. Motivation • The paper presents a method for creating pen-and-ink-like illustrations from detailed geometric models. • Motivated by the need for illustrative representations of plant life in architectural and landscape renderings. • We want to be able to produce images similar to this: © 2005 University of Wisconsin

28. Related Work • “Cartoon Tree” – Alvay Ray Smith • Use small disks to represent bunches of leaves • Reeves and Blau • Use structured particle system to represent realistic trees. • Kowalski et al. • Two step process • Render shaded primitives that approximate foliage • Overlay graftals on rendered image using Salisbury’s line drawing method © 2005 University of Wisconsin

29. Related Work Examples • Smith’s “cartoon tree”: • Kowalski’s Graftal Trees: • Reeves “particle forest”: © 2005 University of Wisconsin

30. Traditional Illustration of Trees • The Skeleton is drawn up to the second level of branching. • Foliage is broken up into three parts: • The directly lit canopy – represented by outlines and scarce inner detail • The half shadow – greater detail provides an overall darker grey level • The shaded area – achieved by thicker lines or drawing leaves in black • Artists will general use a combination of these to draw the leaves. Rarely do they use all three. © 2005 University of Wisconsin

31. Drawing the Skeleton • The tree skeleton is represented by generalized cylinders. • Analytical Silhouette Algorithms • Markosian et al. • Rakar and Cohen • Depth Differences • The skeleton is shaded with crosshatch marks. Marks could be placed using the algorithm outlined by Salisbury et al. • The authors actually use a variant of Floyd-Steinberg diffusion that places strokes when the accumulated grey value is above a threshold. © 2005 University of Wisconsin

32. Drawing the Foliage • The xfrog modeling system can generate models with 200,000 leaves on a tree. We want to pair this down to 10,000 or so. • Leaves are represented abstractly by particles: a location and an orientation vector. • The goal is to illustrate groups of with a set of strokes. • Idea: Render particles as primitives and use depth buffer information to determine which leaves are spatially close and outline them. © 2005 University of Wisconsin

33. Depth Differences • Initially, each particle is rendered with a view-facing primitive, such as a disk. • Basic Algorithm: • For each pixel compute the maximum positive difference between it and its neighbors • If the distance is above the given threshold, mark as an edge. © 2005 University of Wisconsin

34. Depth Differences Continued… • The threshold and primitive size can be varied to achieve different effects: Primitive size: 0.15 Primitive size: 0.70 Threshold: 1000 Threshold: 2000 • The threshold can be varied using other buffers, achieving greater detail in areas that results in a shadowing effect: © 2005 University of Wisconsin

35. Abstract Drawing Primitives • More realistic leaves can be achieved by using the particles’ orientation vectors to interpolate between a set of polygons that represent leaves from different views © 2005 University of Wisconsin

36. Level-of-Abstraction • Two Approaches • Convert raw z-buffer values from non-linear, projected values to linear values using formula below and vary threshold according to depth: d: depth value between 0 and 1 d0: minimal depth value (e.g. 0) d1: maximal depth value (e.g. 65535) z0: depth value of near clipping plane z1: depth value of far clipping plane • Take advantage of the non-linearity to automatically generate more abstract representations. • Geometry farther away will be pushed into fewer values in the z-buffer. • A tree in the background will represented by fewer strokes than the same tree in the foreground. © 2005 University of Wisconsin

37. Level of Abstraction Continued… • The level-of-abstraction can be furthered by scaling the primitives so that the amount of screen space they occupy is independent of their depth. r: scaling factor d: desired screen space s: actual screen space No ScalingScaled © 2005 University of Wisconsin

38. Software Implementation • Calculate depth differences and mark pixels above threshold. • In the interactive mode this bitmap is directly used and blended with other geometry. • Generate shadows in software. Large number of isolated surfaces precludes hardware implementation. • Vectorize bitmap using least squares fitting or primitive identification method. • Draw lines using spline interpolation • Apply line styles © 2005 University of Wisconsin

39. Results © 2005 University of Wisconsin

40. Future Work • Apply cross-hatch shading to the leaves • Create a continuous level-of-detail algorithm that stays performance while getting rid of artifacts that come with discrete representations. • Apply new styles and color for cartoons. • Questions? © 2005 University of Wisconsin

41. © 2005 University of Wisconsin

42. Sketchy Drawings Marc Niehaus Jürgen Döllner University of Potsdam University of Potsdam

43. Sketchy Drawings • Sketchiness shows unfinished state • Useful for architectural and product design

44. Previous Work • G-Buffers (Saito & Takahashi 1990) • Edge enhancement (Nienhaus and Döllner 2003) • Loose and Sketchy filter (Curtis 1998) • Artistic Strokes (Northrup & Markosian 2000)

45. Algorithm Overview • Obtain edge and shading data • Perturb this data to get sketchy look • Problem: It needs to be done in real time

46. G-Buffers • Need to access in real time • Depth/Color/Alpha/Stencil Buffer on graphics card • Textures • Current hardware allows multiple textures • Manipulate with fragment and pixel shaders

47. Edge Map • Need to find edges for most NPR algorithms • Store three types of edges • Silhouette • Crease • Border • See Edge Enhancement (Nienhaus & Döllner 2003 )for more details

48. Shade Map • Render unlit 3D geometry to capture flat colors • Store results in screen aligned texture instead of color buffer

49. Adding Uncertainty • Apply uncertainty function to edge and shade maps • Store function in texture • Use Perlin function to get frame to frame consistency • Multiply uncertainty values with 2x2 matrix • Use different matrix for edge and shade maps to get a more sketchy feel

50. End Result • Use shaders to combine results into texture • Render texture on screen-aligned quad