Survey vision based model generation of 3 d real world scene
1 / 28

Survey: Vision-based Model generation of 3-D real world scene - PowerPoint PPT Presentation

Survey: Vision-based Model generation of 3-D real world scene 김준환 , Marc Nguyen , 설창환 Nov. 05 Introduction Building 3-D Model without using wrestling with CAD tools for months ? Labor-intensive time-consuming resulting models is apparently computer-generated

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Survey: Vision-based Model generation of 3-D real world scene

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Survey: Vision-based Model generation of 3-D real world scene

김준환, Marc Nguyen, 설창환

Nov. 05


  • Building 3-D Model without using wrestling with CAD tools for months ?

    • Labor-intensive

    • time-consuming

    • resulting models is apparently computer-generated

    • can’t be sure about the accuracy of the model

  • The Alternatives

    • vision-based approach

      • take some photos, process them, ready-to-go

    • 3-D scanning

      • not suitable for outdoor scene

Modeling Approaches

Geometry(CAD) -based


Decisions to make

  • Model: polygon or image?

    • Tightly coupled to utilization of the model

    • 3-D polygonal model

      • conventional VR walk-thru / fly-by

    • image-based model

      • image-based renderering/VR (sort of QuickTime VR™)

  • User input?

    • Fully automatic

    • User input as needed

Getting Polygonal models

  • Existing works

    • Depth map + textures

    • Hybrid approach [Devebec96]

  • Issues

    • shape-from

      • Stereo

      • Motion

      • something else ?

    • which feature to use

      • pixel, line, face, …

Basic PrinciplesStereo Vision(1/5)

  • Basic formula

    • reconstruction of the 3-D coordinates of a number of points in a scene for given 2 (or more) images obtained by cameras of known relative positions and orientations

  • Correspondence problem

    • given a token in image 1, what is the corresponding token in image 2?

Basic PrinciplesStereo Vision(2/5)

  • Constraints

    • epipolar constraint

      • for a given point in the plane 1, its possible matches in the plane 2 all lie on a line, therefore search space is reduced from 2D to 1D

Basic PrinciplesStereo Vision(3/5)

  • Ordering constraints

    • the orders of tokens in one image is preserved in the other image (not true when one token is in the forbidden region of the other token)

Basic PrinciplesStereo Vision(4/5)

  • Planarity constraint

    • if the surfaces of the objects are planar, there exists an analytic transformation from the left image coordinates to the right image coordinates.

Basic PrinciplesStereo Vision(5/5)

  • Limitation

    • still exist ambiguity

    • the distance between of the two camera must be sufficiently small

Basic PrinciplesModel-based Vision(1/3)

  • Basic principle

    • to recognize 3D objects, compare a scene model (constructed by processing images obtained from sensors) against entities in a model database (containing a discription of each object the system is expected to recognize).

Basic PrinciplesModel-based Vision(2/3)

  • Related works

    • Hanson and Henderson[Hans89]

      • the automatic synthesis of a specialized recognition scheme, called a strategy tree based on CAGD(computer aided geometric design) model.

      • Strategy tree

        • describe the search process used for recognition and localization of a particular objects in the given scene

        • consist of selected 3D features which satisfy system constraints and corroborating evidence subtrees which are used in the formation of hypothesis.

Basic PrinciplesModel-based Vision(3/3)

  • Flynn and Jain [Flyn91]

    • develop a system which uses 3D object descriptions created on a commercial CAD system

    • express in both the industry-standard IGES (initial graphics exchange specification) form and a polyhedral approximation

    • perform geometric inferencing to obtain a relational graph representation of the object which can be stored in a database of models for object recognition

Depth map + Textures

  • Not provide polygonal representation

    • need further processing(e.g mesh construction)

  • Need special H/W

    • 3D scanner

    • laser range finder

    • video-rate stereo machine


Depth map & Texture: T.Kanade at CMU (1/3)

  • MBV(Modeling by Videotaping)

    • “Walking around the room with camcorder, and get the 3-D model of the room and the trajectory of camera”

    • Based on shape-from-motion

    • factorization technique



Related works at CMU (2/3)

  • Z-key

    • generation of depth map in real time using special purpose H/W











Related works at CMU (3/3)

  • Virtualized Reality

    • create virtual models of real-world events (e.g. sports)

Hybrid approach for architectural scene

  • Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-based Approach,” Proc. SIGGRAPH ‘96

  • For architectural scene

  • Hierarchy of Block primitives

    • parameter reduction in first phase

    • affordable level in amount of user input

  • Model-based stereo

    • refine rough model to recover the details

SIGGRAPH ‘96 Conference Proceedings

Hybrid approach신영길, SNU

  • Road and surrouding environments

  • Simplified case of [Debevec96]

  • Face feature instead of edge


Image-based model

  • Existing works

    • Hirose95

    • View mosaicing

  • Issues

    • how to acquire / store the 2D images ?

    • How to generate seamless image sequence

      • morphing, stitching

      • tightly related to image-based rendering

Hirose 95 (1/4)

  • Purpose

    • generation of virtual words by processing 2D real images taken by video cameras

  • Basic concept

    • image recording

    • position recording

    • image generation for user’s viewpoint

Presence, Vol 5, No 1,

Hirose 95 (2/4)

Image Recording H/W

Hirose 95 (3/4)

  • Image synthesis system

    • Search for nearby images in the database

    • Basic operations : shift, scale, rotation of recorded image

    • Combination of basic operations

  • Enhanced system

    • Use multi-images interpolation

    • Reduce the the feeling of abrupt switch from one image to another

Hirose 95 (4/4)

  • Advantages of the method

    • easy way to generate virtual worlds

    • very realistic appearance

  • Drawbacks

    • No possibility of user’s interaction

    • Archiving volume very large

    • Image processing problems (speed, distortion,…)

  • Future works

    • Use of new technology for archiving virtual worlds

    • Generation of wide virtual worlds (world database)

    • evolved into CABIN?

View mosaicing

  • process of registering several images to obtain a single coherent image

  • Suitable for “looking around” style VR

Szeliski96 (1/2)

  • Purpose

    • Set of techniques for building image mosaics

    • Virtual reality applications

  • Planar image mosaics

    • Different pictures are used to generate one wide planar image

  • Panoramic image mosaics

    • Set of images taken from one viewpoint with a rotation of the camera

    • Panoramic effect -> illusion of real view and scene

    • Used for outdoor scenic view, building interior in virtual reality applications

IEEE CG&A 1996

Szeliski96 (2/2)

  • Projective depth recovery

    • Necessary for illusion of 3D

  • Conclusions

    • These techniques can be used as vision based generation of virtual worlds

    • Photorealistic appearance and try to restore 3D effect

    • But now only static applications

    • May be used as a part of more complete vision-based system


  • Getting realistic model of real world object / scene without CAD

    • indoor, human-scale object : 3D scanning

    • outdoor scene : vision-based approach

      • Based on computer vision techniques

    • human input to some degree might be very helpful

      • Hybrid approach

    • towards moving objects / realtime modeling

  • Login