IMAGE REVIVING

1. IMAGE REVIVING Faculty of computer and information science Ain shams university

2. Supervisors:- - Prof.Dr. Sayed El- Harbete. - Dr. Ahmed Gawish. Assistants:- - Amr Gamgoum. Project Team:- - Sara Ezzat Lamey. - Christina Saweres Yousef. - Marim Tewfik Rabee. - Mina Mazooz Fares. - Mina Esam Kamel.

3. AGENDA • Problem Definition. • History. • Objectives. • Challenges • Aspects In Converting from 2D into 3D Model. • Project Phases. • Supporting Tools. • References.

4. ProblemDefinition How to convert 2D image to 3D model in order to simulate the real world like (visualizing, navigating and simulating buildings and museums), That will be help games developers, campus mapping , help engineers in design and modify buildings structure and helping governments to track roads for reaching to the perfect design.

5. Problem Definition There are many 3D models of buildings available for purchase online, but if you want to have a custom 3D of a particular house or building, you are going to either have to hire a 3D modeler to make it, or make it yourself !!!! Our program will solve that …. 

6. AGENDA • Problem Definition. • History. • Objectives. • Challenges • Aspects In Converting from 2D into 3D Model. • Project Phases. • Supporting Tools. • References.

7. History - There are many web sites and programs can convert the 2D image into 3D model like: 1 - http://3dsee.net/Main.aspx It created by Dr. David McKinnon from Queensland University of Technology, has recently launched this site that turns your sets of 2D images into realistic 3D bump maps (beta version and provide a gray scale bump map) 2- http://make3d.stanford.edu was created by Ashutosh Saxena, Prof. Andrew Y. Ng, and other team members of the Stanford 3D Reconstruction Group.

8. History(cont’) • 3- 3D pop-up model based on the geometrysoftware that construct 3D model out of a single outdoor image. The system labels each region of an outdoor image as ground, vertical, or sky. Is based on Geometric Context from a Single Imageresearches. • it is a software provided by Carnegie Melon University at (10/07/05).

9. AGENDA • Problem Definition. • History. • Objectives. • Challenges • Aspects In Converting from 2D into 3D Model. • Project Phases. • Supporting Tools. • References.

10. Objectives To revive the image and navigate inside it with converting it to 3D model , our goal is to do that on building aspect, and if the image contain humans or cars our program will do 3D models to them also.

11. AGENDA • Problem Definition. • History. • Objectives. • Challenges • Aspects In Converting from 2D into 3D Model. • Project Phases. • Supporting Tools. • References.

12. Challenges Dealing with the inherent ambiguity of the image, One image by itself, simply does not contain enough information to recover 3D spatial layout. knowing CPOV Camera point of view and Putting Objects in Perspective.

13. Challenges (cont’) How to use 3D scene information to recognize objects within the scene. • Computer vision algorithms may be able to detect a few cars and pedestrians in this image but have no understanding of the scene as a whole, as a result, computers cannot answer even simple questions, such as “Am I about to be hit by a bicycle?”.

14. Challenges (cont’) • Recovering occlusions boundaries from the image. • Our strategy is to simultaneously reason about the regions and boundaries in the image and the 3D surfaces of the scene using learned models We learn to identify boundaries based on a wide variety of cues: color, position, and alignment of regions; strength and length of boundaries; 3D surface orientation estimates; and depth estimates

15. Challenges (cont’) • First, if the image contain humans or cars we will remove them from the image so the challenge is:- how to fill the empty places with the similar degree of the image background color. • Recovering surface layout from the 2D single image . • Humans have an amazing ability to instantly grasp the overall 3D structure of a scene—ground orientation, relative positions of major landmarks, etc.—even from a single image How can we represent the 3D space of the scene in away that facilitates tasks such as navigation and recognition?

16. AGENDA • Problem Definition. • History. • Objectives. • Challenges • Aspects In Converting from 2D into 3D Model. • Project Phases. • Supporting Tools. • References.

17. Aspects in Converting 2D into 3D Model Medical Aspect. Converting from 2D logo to 3D model. Building Aspect. Converting from 2D human image to 3D model. General 3D modeling to 2D images of (Animals, Cars, Bicycles,..).

18. AGENDA • Problem Definition. • History. • Objectives. • Challenges • Aspects In Converting from 2D into 3D Model. • Project Phases. • Supporting Tools. • References.

19. Project Phases • Determining the geometric class of an image region • Across ( color, texture, location in the image, shape, and projective geometry cues). • Color : is valuable in identifying the material of a surface (sky is usually blue or white and the ground is often green (grass) or brown (dirt). • We represent color using two color spaces: • RGB : allows the “blueness” or “greenness” of a region to be easily extracted. • HSV : allows perceptual color attributes such as “hue” and “grayness” to be measured. • Texture : provides additional information about the material of a surface (sharpness ,smoothing,….).

20. Project Phases(cont’) Location : in the image also provides strong cues for distinguishing between ground (tends to be low in the image), vertical structures, and sky (tends to be high in the image). Shape : region shape helps distinguish vertical regions (often roughly convex) from ground and sky regions (often non-convex and large). Projective geometry cues : 3D Geometry features help determine the 3D orientation of surfaces. Knowledge of the vanishing line of a plane completely specifies its 3D orientation relative to the viewer.

21. Project Phases(cont’) • Labeling the Image • We gradually build our structural knowledge of the image, from pixels to super pixels to constellations of super pixels, Once we have formed multiple sets of constellations, we estimate the constellation label likelihoods and the likelihood that each constellation is homogeneously labeled from which we infer the most likely geometric labels of the super pixels. • Obtaining Super pixels :small, nearly-uniform regions in the image ,first step is to form super pixels from those raw pixel intensities . • Forming Constellations : we group super pixels that are likely to share a common geometric label into “constellations”. • Geometric Classification : For each constellation, we estimate the (label likelihood) whether all super pixels in the Constellationshave the same label.

22. Project Phases(cont’) Input Image Vertical Labels Support Sky Left Right Center

23. Project Phases(cont’) Input Image Super pixels Multi Segmentations

24. Project Phases(cont’) • Determining Camera Parameters • To obtain true 3D world coordinates, we would need to know the camera parameters. • We can, however, create a reasonable scaled model by estimating the horizon line (giving the angle of the camera with respect to the ground plane).

25. Project Phases(cont’) Original Image Object representation due to the 2D image Objects representation due to 3D Computer Vision

26. Project Phases(cont’) • Creating the 3D Model (Finally) • Cutting and Folding • We construct a simple 3D model by making “cuts” and “folds” in the image based on the geometric labels (texture mapping).

27. AGENDA • Problem Definition. • History. • Objectives. • Challenges • Aspects In Converting from 2D into 3D Model. • Project Phases. • Supporting Tools. • References.

28. Supporting Tools Microsoft Visual Studio 2008. MATLAB. OpenGL.

29. References http://www.cs.cmu.edu/~efros/ImageInterpretation/,Geometrically Coherent Image Interpretation, Graduate Student Researcher: Derek Hoiem . Seeing the World Behind the Image: Spatial Layout for 3D Scene Understanding ,Derek Hoiem PhD Thesis, Robotics Institute, Carnegie Mellon University, August 2007 . Closing the Loop on Scene Interpretation ,Derek Hoiem, Alexei A. Efros, Martial Hebert ,in CVPR 2008 , See 3Dreconstruction compared to Photo Pop-up and Make3D. Recovering Occlusion Boundaries from a Single Image Derek Hoiem, Andrew Stein, Alexei A. Efros, Martial Hebert , in ICCV 2007. Putting Objects in Perspective ,Derek Hoiem, Alexei A. Efros, Martial HebertIn CVPR 2006,Best Paper Award . Geometric Context from a Single Image , Derek Hoiem, Alexei A. Efros, Martial Hebert ,In ICCV 2005. Automatic Photo Pop-up , Derek Hoiem, Alexei A. Efros, Martial Hebert.

30. THANK YOU