Survey vision based model generation of 3 d real world scene
1 / 28

Survey: Vision-based Model generation of - PowerPoint PPT Presentation

  • Uploaded on

Survey: Vision-based Model generation of 3-D real world scene 김준환 , Marc Nguyen , 설창환 Nov. 05 Introduction Building 3-D Model without using wrestling with CAD tools for months ? Labor-intensive time-consuming resulting models is apparently computer-generated

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Survey: Vision-based Model generation of' - omer

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Survey vision based model generation of 3 d real world scene l.jpg

Survey: Vision-based Model generation of 3-D real world scene

김준환, Marc Nguyen, 설창환

Nov. 05

Introduction l.jpg

  • Building 3-D Model without using wrestling with CAD tools for months ?

    • Labor-intensive

    • time-consuming

    • resulting models is apparently computer-generated

    • can’t be sure about the accuracy of the model

  • The Alternatives

    • vision-based approach

      • take some photos, process them, ready-to-go

    • 3-D scanning

      • not suitable for outdoor scene

Modeling approaches l.jpg
Modeling Approaches

Geometry(CAD) -based


Decisions to make l.jpg
Decisions to make

  • Model: polygon or image?

    • Tightly coupled to utilization of the model

    • 3-D polygonal model

      • conventional VR walk-thru / fly-by

    • image-based model

      • image-based renderering/VR (sort of QuickTime VR™)

  • User input?

    • Fully automatic

    • User input as needed

Getting polygonal models l.jpg
Getting Polygonal models

  • Existing works

    • Depth map + textures

    • Hybrid approach [Devebec96]

  • Issues

    • shape-from

      • Stereo

      • Motion

      • something else ?

    • which feature to use

      • pixel, line, face, …

Basic principles stereo vision 1 5 l.jpg
Basic PrinciplesStereo Vision(1/5)

  • Basic formula

    • reconstruction of the 3-D coordinates of a number of points in a scene for given 2 (or more) images obtained by cameras of known relative positions and orientations

  • Correspondence problem

    • given a token in image 1, what is the corresponding token in image 2?

Basic principles stereo vision 2 5 l.jpg
Basic PrinciplesStereo Vision(2/5)

  • Constraints

    • epipolar constraint

      • for a given point in the plane 1, its possible matches in the plane 2 all lie on a line, therefore search space is reduced from 2D to 1D

Basic principles stereo vision 3 5 l.jpg
Basic PrinciplesStereo Vision(3/5)

  • Ordering constraints

    • the orders of tokens in one image is preserved in the other image (not true when one token is in the forbidden region of the other token)

Basic principles stereo vision 4 5 l.jpg
Basic PrinciplesStereo Vision(4/5)

  • Planarity constraint

    • if the surfaces of the objects are planar, there exists an analytic transformation from the left image coordinates to the right image coordinates.

Basic principles stereo vision 5 5 l.jpg
Basic PrinciplesStereo Vision(5/5)

  • Limitation

    • still exist ambiguity

    • the distance between of the two camera must be sufficiently small

Basic principles model based vision 1 3 l.jpg
Basic PrinciplesModel-based Vision(1/3)

  • Basic principle

    • to recognize 3D objects, compare a scene model (constructed by processing images obtained from sensors) against entities in a model database (containing a discription of each object the system is expected to recognize).

Basic principles model based vision 2 3 l.jpg
Basic PrinciplesModel-based Vision(2/3)

  • Related works

    • Hanson and Henderson[Hans89]

      • the automatic synthesis of a specialized recognition scheme, called a strategy tree based on CAGD(computer aided geometric design) model.

      • Strategy tree

        • describe the search process used for recognition and localization of a particular objects in the given scene

        • consist of selected 3D features which satisfy system constraints and corroborating evidence subtrees which are used in the formation of hypothesis.

Basic principles model based vision 3 3 l.jpg
Basic PrinciplesModel-based Vision(3/3)

  • Flynn and Jain [Flyn91]

    • develop a system which uses 3D object descriptions created on a commercial CAD system

    • express in both the industry-standard IGES (initial graphics exchange specification) form and a polyhedral approximation

    • perform geometric inferencing to obtain a relational graph representation of the object which can be stored in a database of models for object recognition

Depth map textures l.jpg
Depth map + Textures

  • Not provide polygonal representation

    • need further processing(e.g mesh construction)

  • Need special H/W

    • 3D scanner

    • laser range finder

    • video-rate stereo machine


Depth map texture t kanade at cmu 1 3 l.jpg
Depth map & Texture: T.Kanade at CMU (1/3)

  • MBV(Modeling by Videotaping)

    • “Walking around the room with camcorder, and get the 3-D model of the room and the trajectory of camera”

    • Based on shape-from-motion

    • factorization technique



Related works at cmu 2 3 l.jpg
Related works at CMU (2/3)

  • Z-key

    • generation of depth map in real time using special purpose H/W

Related works at cmu 3 3 l.jpg











Related works at CMU (3/3)

  • Virtualized Reality

    • create virtual models of real-world events (e.g. sports)

Hybrid approach for architectural scene l.jpg
Hybrid approach for architectural scene

  • Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-based Approach,” Proc. SIGGRAPH ‘96

  • For architectural scene

  • Hierarchy of Block primitives

    • parameter reduction in first phase

    • affordable level in amount of user input

  • Model-based stereo

    • refine rough model to recover the details

SIGGRAPH ‘96 Conference Proceedings

Hybrid approach snu l.jpg
Hybrid approach신영길, SNU

  • Road and surrouding environments

  • Simplified case of [Debevec96]

  • Face feature instead of edge


Image based model l.jpg
Image-based model

  • Existing works

    • Hirose95

    • View mosaicing

  • Issues

    • how to acquire / store the 2D images ?

    • How to generate seamless image sequence

      • morphing, stitching

      • tightly related to image-based rendering

Hirose 95 1 4 l.jpg
Hirose 95 (1/4)

  • Purpose

    • generation of virtual words by processing 2D real images taken by video cameras

  • Basic concept

    • image recording

    • position recording

    • image generation for user’s viewpoint

Presence, Vol 5, No 1,

Hirose 95 2 4 l.jpg
Hirose 95 (2/4)

Image Recording H/W

Hirose 95 3 4 l.jpg
Hirose 95 (3/4)

  • Image synthesis system

    • Search for nearby images in the database

    • Basic operations : shift, scale, rotation of recorded image

    • Combination of basic operations

  • Enhanced system

    • Use multi-images interpolation

    • Reduce the the feeling of abrupt switch from one image to another

Hirose 95 4 4 l.jpg
Hirose 95 (4/4)

  • Advantages of the method

    • easy way to generate virtual worlds

    • very realistic appearance

  • Drawbacks

    • No possibility of user’s interaction

    • Archiving volume very large

    • Image processing problems (speed, distortion,…)

  • Future works

    • Use of new technology for archiving virtual worlds

    • Generation of wide virtual worlds (world database)

    • evolved into CABIN?

View mosaicing l.jpg
View mosaicing

  • process of registering several images to obtain a single coherent image

  • Suitable for “looking around” style VR

Szeliski96 1 2 l.jpg
Szeliski96 (1/2)

  • Purpose

    • Set of techniques for building image mosaics

    • Virtual reality applications

  • Planar image mosaics

    • Different pictures are used to generate one wide planar image

  • Panoramic image mosaics

    • Set of images taken from one viewpoint with a rotation of the camera

    • Panoramic effect -> illusion of real view and scene

    • Used for outdoor scenic view, building interior in virtual reality applications

IEEE CG&A 1996

Szeliski96 2 2 l.jpg
Szeliski96 (2/2)

  • Projective depth recovery

    • Necessary for illusion of 3D

  • Conclusions

    • These techniques can be used as vision based generation of virtual worlds

    • Photorealistic appearance and try to restore 3D effect

    • But now only static applications

    • May be used as a part of more complete vision-based system

Summary l.jpg

  • Getting realistic model of real world object / scene without CAD

    • indoor, human-scale object : 3D scanning

    • outdoor scene : vision-based approach

      • Based on computer vision techniques

    • human input to some degree might be very helpful

      • Hybrid approach

    • towards moving objects / realtime modeling