Animating a Three Dimensional Whole Body Point Scan

1. Animating a Three DimensionalWhole Body Point Scan Pushkar Deshmukh Computer Science Department Clemson University July 1999

2. Outline • Background and Motivation • Animation Process • Basic Idea • Software Architecture • Extended Free Form Deformation • Designing Lattices • Skeleton and Joint Structure • Generating Animations • Results • Applications and Future Work

3. 3D Whole Body Scanners Cyberware Whole Body 3D scanner 4 cameras 120K+ points 17-sec. Scan x, y, z, RGB values

4. 3D Whole Body Scanners [TC]2 Whole Body 3D scanner 6 cameras 350K+ points 2-sec. Scan x, y, z values

5. 3D Whole Body Scan

6. 3D Whole Body Scan Close-up view

7. Motivation • Ergonomics • Study of relation between workers and their environment • Used for product design • Faster and Cheaper design process • Measurements from 3D scan • 3DM Project • Animating 3D scan

8. 3D Scan Animation One Approach: Segmenting Scan

9. W V U Free Form Deformation • Basic Idea • Embed an object in a deformable region • Deform the region to deform the object • Developed by Thomas Sederberg and Scott Parry (Free-Form Deformation of Solid Geometric Models. SIGGRAPH '86, volume 20, pages 151-160, ACM, August 1986.) • Lattices Parallelepiped Lattice

10. Free Form Deformation • Mathematical Definition • Tensor product trivariate Bernstein polynomial where Lijk is the (i, j, k)th control point of the lattice, (u,v,w) are lattice space coordinates of point P and the lattice has l+1, m+1, n+1 planes in the U, V, W directions respectively.

11. Free Form Deformation • Steps • Calculate lattice space coordinates (u, v, w) (Freeze lattice) • Deform the lattice • Deform object Example of FFD

12. Free Form Deformation • Limitation • Only parallelepiped lattices • Extended Free Form Deformation • developed by Coquillart (Extended Free-Form Deformation: A Sculpturing Tool for 3D Geometric Modelling. Computer Graphics, Volume 24, Number 4, pages 187-196, August 1990, ACM SIGGRAPH) • allows non-parallelepiped lattices

13. Animation Process Basic Idea • Enclose scan data points in EFFD lattices • Freeze lattices • Move lattice control points • Deform scan to generate animations

14. Steps in Animation Process • Read a 3D scan • Build EFFD lattices around the scan • Build skeleton and joint structure • Freeze the lattices • Generate postures by varying joint angles • Interpolate between postures to generate animation

15. User Interface CMainAnim Skeleton & Joint Structure CDataPoint CSkeleton CMatrix CJoint CLattice Extended Free Form Deformation CComplexJoint 3D Human Scan Software Architecture Class diagram Software Modules

16. V P(u, v, w) U W Start: A point P(x, y, z) enclosed in a lattice First Step: Calculate (u, v, w) coordinates of P Y P(x', y', z') X Z Second Step: Find new (x, y, z) coordinates of P using its (u, v, w) coordinates and the lattice control points. Y P(x, y, z) X Z Extended Free Form Deformation

17. Find Cartesian Coordinate Where, - Lijk is the cartesian coordinate of the (i, j, k)th control point of the lattice - (u, v, w) is the lattice space coordinates of point P - the lattice has l + 1, m + 1, and n + 1 planes in the U, V, and W directions respectively and

18. Y V L110 L010 P(xf, yf, zf) L011 L111 V U X L000 P(ui, vi, wi) P(xi, yi, zi) L100 FFD Equation L001 L101 W Z Cartesian Coordinate Space Lattice Space Find Lattice Space Coordinate • Iterative Process • Aim, find (u, v, w) coordinate for point P(xf, yf, zf) • Start with point P(u0 = 0.5, v0 = 0.5, w0 = 0.5) • Find cartesian coordinate P(x0, y0, z0)

19. Find Lattice Space Coordinate • We obtain where is inverse of transpose of local approximate of Jacobian at (u0, v0, w0) • In general • Terminating condition

20. Jacobian Suppose we have a function where X is of dimension n and L is of dimension m, then dX = J(L) dL and, the (i, j)th element of J is given by From above equation

21. Jacobian Using forward differences to obtain an estimate of Jacobian at lattice control points we have:

22. Jacobian • Estimate jacobian at point P(u, v, w) using trilinear interpolation • Extend the jacobian

23. Terminating the iterative process • V = P(xf, yf, zf) - P(xi, yi, zi) < • Number of Iterations (NI) and 0 < NI < 50 = 0.0001 50 < NI < 500 = 0.01 NI = 500 = 0.1 • Average number of iterations ~ 4

24. V P(u, v, w) Y U P(x', y', z') W Start: A point P(x, y, z) enclosed in a lattice First Step: Calculate (u, v, w) coordinates of P X Z Second Step: Find new (x, y, z) coordinates of P Y P(x, y, z) X Z Extended Free Form Deformation • Point-in-lattice membership

25. Five EFFD lattices Each lattice has Two planes in X direction Two planes in Y direction Seven planes in Z direction Plane 6 Z Plane 0 Y X Designing Lattices

26. Plane 6 Plane 3 Plane 6 Plane 0 Plane 3 Plane 0 Designing Arm and Leg Lattices

27. Plane 6 Plane 4 Plane 2 Plane 0 Designing Torso Lattice

28. Designing Lattices

29. Designing Lattices • Moving Control Points • Maintaining Continuity • Avoid Overlap

30. Skeleton and Joint Structure • Need • Deform the lattice control points in a controlled manner • Provides • Ability to move the control points by specifying joint angles

31. Root Position Right Shoulder Left Shoulder Right Hip Center Left Hip Neck Base Right Elbow Left Elbow Right Knee Spine Left Knee Spine Neck Base Right Hip Left Hip Center Right Knee Left Knee Right Shoulder Left Shoulder Right Elbow Left Elbow Skeleton and Joint Structure Skeleton and Joint Structure Joint Dependency Hierarchy

32. Associating Joints with Lattices • Link to lattices • Using lattice plane numbers • Control points affected by the joint • Additional Constraints • Proper placement of lattice planes Plane 6 Plane 3 Plane 0

33. X Z X Y Y Z X Z Y X Y Z X Y Z Z X Y X Z X Y Y Z Z X Y X Y Y Z Z X Joints and Lattice Control Points • Joint local coordinate system • Local angle of rotations • Sequence of rotation • Z, X, and then Y Joint local coordinate system

34. Y Y Z Z X Y P X Y Z X Z P X Joints and Lattice Control Points • Local coordinates of the control points • Changing local angle of rotation Initial state Rotating shoulder about X axis

35. Joints and Lattice Control Points • Find initial joint angles

36. Initial postureA posture with hands raised and upper body turned Generating Animations • Postures

37. Generating Animations • Saving posture files • Animations generated by linear interpolation between postures ROOT_POSITION 0.010000 -0.050000 0.000000 CENTER 0.000000 0.000000 0.000000 SPINE 0.000000 0.000000 0.000000 NECK_BASE 0.000000 0.000000 0.000000 NECK 0.000000 0.000000 0.000000 RIGHT_ELBOW 48.366463 25.538908 0.000000 RIGHT_SHOULDER -50.856022 0.000000 0.000000 RIGHT_HIP -2.726310 -2.723229 0.000000 RIGHT_KNEE -4.720062 5.182696 0.000000 LEFT_ELBOW 36.594429 -25.992308 0.000000 LEFT_SHOULDER -49.398701 0.000000 0.000000 LEFT_HIP -0.000002 2.862404 0.000000 LEFT_KNEE -9.512845 -7.522387 0.000000 Sample posture file

38. Results Upper body movement

39. Results Movement of lattice control points

40. Results Leg Movement

41. Results Handshake

42. Applications • Improving ergonomics in product design • Visualizing simulations involving human characters • Computer games • Designing safer and comfortable apparel • Representing human participants in VR • Film industry

43. Conclusion • Simple animations can be created from a 3D human scan • The 3D scan works as a good human model • Further improvement in term of speed and quality of animation are possible

44. Future Work • Improving speed of the process • Reduce number of data points • Automate lattice designing • Using accelerated search algorithm for point-in-lattice membership • Providing multi-processor support

45. Future Work • Improving quality of generated animations • Implementing effects like squash and stretch of muscles • Increasing number of joints • Restricting the degree of freedom and adding limits on magnitude of rotations • Using better interpolation algorithms • Interfacing with human simulation packages

46. Acknowledgements I would like to thank my committee members Dr. Roy Pargas (Committee Chair), Dr. Andrew Duchowski and Dr. Gene Tagliarini for their guidance and for their valuable and insightful criticism and advice.