Tour Into the Picture: Using a Spidery Mesh to Make Animation from a Single Image

1. Tour Into the Picture:Using a Spidery Mesh to Make Animation from a Single Image Youichi Horry, Ken-ichi Anjyo, and Kiyoshi Arai SIGGRAPH 1997 presented by Dave Edwards 10/31/2000

2. A Unique Problem • Can we create animation from a single image? • Paintings • Single photographs • Extremely difficult for a computer • No information for depth or 3D structure • Not as difficult for a human brain

3. A Unique Solution • A system that allows humans and computers to do what they do best • Human supplies 3D scene information • Computer supplies efficient interface for: • Specifying vanishing points • Differentiating foreground & background • Building the scene with simple polygons

4. Tour Into the Picture (TIP) • System that allows efficient specification of 3D information • Basically, construct a diorama-like model of the scene • Simple, generic representation • 5 (or fewer) polygons for background • Small number of polygons for each foreground object

5. Overview • Scene modeling • Separate foreground & background • Specify a vanishing point • Model the background • Model foreground objects • Scene animation & rendering • Conclusions • Demo

6. Scene Modeling • Assumptions • Scene is basically a box • Sides of the box are orthogonal to each other • Virtual camera may not be tilted • View up vector is perpendicular to “floor”

7. Separate Fore- & Background • Start with a single image:

8. Separate Fore- & Background • Create two images with standard image editor • Foreground mask: • One gray region for each object • Grayscale map of alpha values • Background image: • Original image w/o foreground • Use neighboring pixels to fill gaps

9. Separate Fore- & Background • Example foreground mask:

10. Separate Fore- & Background • Example background image:

11. Specify a Vanishing Point • Use a “spidery mesh” • Inner rectangle • Specifies “rear wall” of scene • Radial perspective lines • Specify the location of the vanishing point • Should be parallel to guide lines in the picture • Interface allows rapid changes in the mesh

12. Specify a Vanishing Point • Example of Interface:

13. Model the Background • Partition the background into 5 pieces: • Rear wall (inner rectangle) • Floor (y = 0) • Ceiling • Left Wall • Right Wall • Calculate 3D coords for the vertices of the box

14. Model the Background • Example of background model:

15. Model the Foreground • Foreground objects defined by mask • Construct polygon model of each object • Must be orthogonal to one side of the box • Calculate 3D coords for corners of polygon

16. Model the Foreground • More complex objects can be made of multiple polygons • Each additional polygon must be connected to the previous one • This allows the system to determine its 3D coordinates

17. Model the Foreground • Example of foreground model:

18. Scene Animation & Rendering • Camera transformations: • Rotation (change view-plane normal) • Translation • Zoom (move viewpoint forward) • Change view angle • Look around (orbit about central object)

19. Scene Animation & Rendering • We know 3D coordinates for points in the scene • Given a pixel to render, find its 3D image in the original scene • Apply smoothing for the background • Apply alpha blending for the foreground objects

20. Conclusions • TIP is very flexible • In general, assumptions do not create severe constraints • Generic foreground/background model works with many types of images • Interface is (fairly) simple & intuitive, even in the demo version • It could use a little internationalization...

21. Future Possibilities • Handle two-point perspective drawings • Allow multiresolution images • Improve the model for foreground objects

22. Demonstration