Symmetric architecture modeling with a single image
This presentation is the property of its rightful owner.
Sponsored Links
1 / 31

Symmetric Architecture Modeling with a Single Image PowerPoint PPT Presentation


  • 69 Views
  • Uploaded on
  • Presentation posted in: General

Symmetric Architecture Modeling with a Single Image. Author: Nianjuan Jiang, Ping Tan, Loong-Fah Cheong Department of Electrical & Computer Engineering, National University of Singapore. Presenter: Feilong Yan. Motivation.

Download Presentation

Symmetric Architecture Modeling with a Single Image

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Symmetric architecture modeling with a single image

Symmetric Architecture Modeling with a Single Image

Author: Nianjuan Jiang, Ping Tan, Loong-Fah Cheong

Department of Electrical & Computer Engineering, National University of Singapore

Presenter: Feilong Yan


Motivation

Motivation

Model architecture from single image is common task in 3D creation due to the lack of the more images.

Historic Photo:

Internet Photo:


Motivation1

Motivation

Single image based modeling is very difficult!

Due to the trouble on camera calibration and texture loss

The recent methods only can handle simple and planar façade

Pascal Mulleet al. Image-based procedural modeling of facades

Changchang Wu et al. Repetition-based Dense Single-View Reconstruction


Motivation2

Motivation

But what about this one?

And If we only have this single photo.

Complex and not planar


Motivation3

Motivation

Fortunately , the symmetry is very prevalent in the architecture

Symmetry is a breakthrough which magically can generate

more images from the input

Complex and not planar

This is reasonable, but exciting to me


Main idea

Main Idea

Bilateral Symmetry


Main idea1

Main Idea

Rotational Symmetry


Main idea2

Main Idea

2 even more views Reconstruction


Modeling pipeline

Modeling Pipeline

3D Reconstruction

Surface Modeling

Texture Enhancement

Input Image and Frustum Vertices

Model Refinement

Model Initialization

Calibration and 3D Reconstruction


3d reconstruction

3D Reconstruction

Camera Calibration

3D points Reconstruction


Camera calibration

Camera Calibration

Previous Methods

Calibrate the camera from the vanishing points of 3 mutually orthogonal directions in a single image.

HARTLEY, R., AND ZISSERMAN

Multiple View Geometry in Computer Vision

But many photos do not have 3 vanishing points, and this method is often numerical unstable


Camera calibration1

Camera Calibration

Previous Methods

If enough(>=6) correspondences between spatial vertices and the image pixels are known, the camera calibration may be immediately computed.

WILCZKOWIAK, M. et. al Using geometric constraints through parallelepipeds for calibration and 3d modeling

Parallelipiped is used to represent a building block. Under the constraint of parallelipiped, the visible 6 spatial vertices may be estimated.

This method is stable but not very suitable for some architecture


Camera calibration2

Camera Calibration

New Method:

Inspired by parallelipiped method, the author found the frustum more general to represent the architecture


Camera calibration3

Camera Calibration

Demo:

Frustum is symmetric


Camera calibration4

Camera Calibration

6

4

5

3

1

2

Coordinate represented in world:

Of this example


Camera calibration5

Camera Calibration

6

4

5

3

1

2

K=

=

= Quaternion( unit vector(x,y,z),)

t =t(x, y, z)

15 parameters to estmate

=1


Camera calibration6

Camera Calibration

6

4

5

3

1

2

Simplification:

K=

11 parameters to estimate, now the calibration is formulated as a non-linear optimization


Camera calibration7

Camera Calibration

Optimization Initialization:

=

=

The Quadratic :

Extend the right multiplication, since the R is unit orthogonal matrix, then we obtain:

User gives the


3d points reconstruction

3D Points Reconstruction

Symmetry-Based Triangulation:


3d points reconstruction1

3D Points Reconstruction

Symmetry-Based Triangulation:


Surface modeling

Surface Modeling

User-Interaction Assisted Modeling

Geometry Modeling

Model Refinement

Texture Mapping


Geometry modeling

Geometry Modeling

Roof

Planar Structure


Model refinement

Model Refinement


Texture mapping

Texture Mapping

Single image inevitably lack texture samples due to the foreshortening and occlusion.

But to achieve a good texture effect, there are 2 requirements:

1. the final texture should be consistent with the foreshortened image ;

2, the final texture should have consistent weathering pattern.

We need to know where is well textured and where not

Refine low quality region

Detect Texture Quality

Texture the occluded region


Texture quality detection

Texture Quality Detection

Back Project

Ratio = Triangle.size / imageProjection.size

Ratio > Threshold and Ratio is finite: large texture distortion

Ratio<Threshold: distortion free

Ratio is infinite: occluded

Texture in distortion free region will be used as the texture sample


Refinement for low quality

Refinement for Low-Quality

Super- Resolution Problem


Occluded region texturing

Occluded Region Texturing

The simplest way is to repeat the same texture as those of their symmetric counterparts, but this makes the model look artificial.

It is better to synthesize the texture in these region with common method

Another feature of the texture is weathering pattern, a constraint texture map is used according to the height of the architecture.

Synthesized

Input sample


Result

Result


Result1

Result


Conclusion

Conclusion

  • Contribution:

    • Novel Calibration Method

    • Texture Enhance Method

  • Limitations:

    • Strong assumption for simplification of camera calibration


Message from this paper

Message from this Paper


  • Login