Symmetric Architecture Modeling with a Single Image. Author: Nianjuan Jiang, Ping Tan, Loong-Fah Cheong Department of Electrical & Computer Engineering, National University of Singapore. Presenter: Feilong Yan. Motivation.
Author: Nianjuan Jiang, Ping Tan, Loong-Fah Cheong
Department of Electrical & Computer Engineering, National University of Singapore
Presenter: Feilong Yan
Model architecture from single image is common task in 3D creation due to the lack of the more images.
Single image based modeling is very difficult!
Due to the trouble on camera calibration and texture loss
The recent methods only can handle simple and planar façade
Pascal Mulleet al. Image-based procedural modeling of facades
Changchang Wu et al. Repetition-based Dense Single-View Reconstruction
But what about this one?
And If we only have this single photo.
Complex and not planar
Fortunately , the symmetry is very prevalent in the architecture
Symmetry is a breakthrough which magically can generate
more images from the input
Complex and not planar
This is reasonable, but exciting to me
2 even more views Reconstruction
Input Image and Frustum Vertices
Calibration and 3D Reconstruction
3D points Reconstruction
Calibrate the camera from the vanishing points of 3 mutually orthogonal directions in a single image.
HARTLEY, R., AND ZISSERMAN
Multiple View Geometry in Computer Vision
But many photos do not have 3 vanishing points, and this method is often numerical unstable
If enough(>=6) correspondences between spatial vertices and the image pixels are known, the camera calibration may be immediately computed.
WILCZKOWIAK, M. et. al Using geometric constraints through parallelepipeds for calibration and 3d modeling
Parallelipiped is used to represent a building block. Under the constraint of parallelipiped, the visible 6 spatial vertices may be estimated.
This method is stable but not very suitable for some architecture
Inspired by parallelipiped method, the author found the frustum more general to represent the architecture
Frustum is symmetric
Coordinate represented in world:
Of this example
= Quaternion( unit vector(x,y,z),)
t =t(x, y, z)
15 parameters to estmate
11 parameters to estimate, now the calibration is formulated as a non-linear optimization
The Quadratic :
Extend the right multiplication, since the R is unit orthogonal matrix, then we obtain:
User gives the
User-Interaction Assisted Modeling
Single image inevitably lack texture samples due to the foreshortening and occlusion.
But to achieve a good texture effect, there are 2 requirements:
1. the final texture should be consistent with the foreshortened image ;
2, the final texture should have consistent weathering pattern.
We need to know where is well textured and where not
Refine low quality region
Detect Texture Quality
Texture the occluded region
Ratio = Triangle.size / imageProjection.size
Ratio > Threshold and Ratio is finite: large texture distortion
Ratio<Threshold: distortion free
Ratio is infinite: occluded
Texture in distortion free region will be used as the texture sample
Super- Resolution Problem
The simplest way is to repeat the same texture as those of their symmetric counterparts, but this makes the model look artificial.
It is better to synthesize the texture in these region with common method
Another feature of the texture is weathering pattern, a constraint texture map is used according to the height of the architecture.