NUDT
This presentation is the property of its rightful owner.
Sponsored Links
1 / 87

Model-Driven 3D Content Creation as Variation PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on
  • Presentation posted in: General

NUDT. TAU. ZJU. SFU. Model-Driven 3D Content Creation as Variation. Hao (Richard) Zhang – 张皓 GrUVi Lab, Simon Fraser University (SFU) Talk @ HKUST, 04/21/11. 3D content creation. Inspiration?. Inspiration  a readily usable digital 3D model. Realistic reconstruction.

Download Presentation

Model-Driven 3D Content Creation as Variation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Model driven 3d content creation as variation

NUDT

TAU

ZJU

SFU

Model-Driven 3D Content Creation as Variation

Hao (Richard) Zhang – 张皓

GrUVi Lab, Simon Fraser University (SFU)

Talk @ HKUST, 04/21/11


3d content creation

3D content creation

Inspiration?

Inspiration  a readily usable digital 3D model


Realistic reconstruction

Realistic reconstruction

Inspiration = real-world data

[Nan et al., SIGGRAPH 2010]


Creative inspiration

Creative inspiration

Creation of novel 3D shapes

Inspiration = design concept, mental picture, …

High demand in VFX, games, simulation, VR, …

sketch


3d content creation is hard

3D content creation is hard

2D-to-3D: an ill-posed problem

Shape from shading, sketch-based modeling, …

Creation from scratch is hard: job for skilled artists

One of the most central problems in graphics; One of the most discussed at SIG’10 panel


Usable 3d content even harder

Usable 3D content even harder

Models created are meant for subsequent use

Creation of readily usable 3D models


Usable 3d content even harder1

Usable 3D content even harder

Models created are meant for subsequent use

Creation of readily usable 3D models

Higher-level information beyond low-level mesh

Part or segmentation information

Structural relations between parts

Correspondence to relevant models, etc.

Hard shape analysis problems!


Key model reuse

Key: model reuse

Reuse existing 3D models and associated info

Model-driven approach: creation is driven by or based on existing (pre-analyzed) models


Key model reuse1

Key: model reuse

Reuse existing 3D models and associated info

Model-driven approach: creation is driven by or based on existing (pre-analyzed) models

Two primary modes of reuse:

New creation via part composition


Key model reuse2

Key: model reuse

Reuse existing 3D models and associated info

Model-driven approach: creation is driven by or based on existing (pre-analyzed) models

Two primary modes of reuse:

New creation via part composition

New creation as variationor modification of existing model(s), e.g.,a warp or a deformation


Modeling by example

Modeling by example

New models composed by parts retrieved from an existing data repository

Key: retrieve relevant parts

Many variants …

[Funkhouser et al., SIGGRAPH 2004]


Pros and cons

Pros and cons

Pros:

Significant deviation from existing models

Exploratory modeling via part suggestions

[Chaudhuri & Koltun., SIG Asia 2010]


Pros and cons1

Pros and cons

Pros:

Significant deviation from existing models

Exploratory modeling with part suggestions

Cons:

Are models composed by parts readily usable?


Pros and cons2

Pros and cons

Pros:

Significant deviation from existing models

Exploratory modeling with part suggestions

Cons:

Are models composed by parts readily usable?  structure lost by part composition; how to stitch?


Pros and cons3

Pros and cons

Pros:

Significant deviation from existing models

Exploratory modeling with part suggestions

Cons:

Are models composed by parts readily usable?  structure lost by part composition; how to stitch?

Does part exploration always reflect user design intent?


Model driven creation as variation

Model-driven creation as variation

New creation as variationof existing model(s)

Inspiration = a model set

Inspiration = photographs

Photo-inspired 3D model creation

Enrich a set; generate “more of the same” …


Model driven creation as variation1

Model-driven creation as variation

New creation as variationof existing model(s)

Enrich a set; generate “more of the same” …

Inspiration = a model set


Style content separation by anisotropic part scales

Style-Content Separation by Anisotropic Part Scales

Kai Xu1,2, Honghua Li2, Hao Zhang2, Daniel Cohen-Or3

Yueshan Xiong2, and Zhi-Quan Cheng2

1Simon Fraser Universtiy 2National Univ. of Defense Tech. 3Tel-Aviv University


Motivation

Motivation

Enrich a set of 3D models with their derivatives

Set belongs to the same family or class


Variations in shape parts in the set

Variations in shape parts in the set

Geometric or content difference

Part proportion (= style) difference


Style transfer as a derivative

Style transfer as a derivative

?

Part proportion style


Style transfer as a derivative1

Style transfer as a derivative

Part proportion style

?


Difficulty with style transfer

Difficulty with style transfer

Style transfer needs part correspondence

Part correspondence is difficult

Unsupervised problem

Both content and style variations

Variations can be significant!


Work at part and obb level

Work at part and OBB level

Parts enclosed and characterized by tight oriented bounding boxes (OBBs)


Style content separation

Style content separation

To address both shape variations in the set

Separate treatment of “style” and “content”

Content

Style 1

Style

Style 2

Style 3


Style transfer as a derivative2

Style transfer as a derivative

Creation = filling in the style-content table


Style vs content

Style vs. content

Fundamental to human perception


Style content separation1

Style content separation

Previous works on faces, motion, etc.

Prerequisite: data correspondence

Correspondence dealt with independently

Correspondence itself is the very challenge!


Our approach

Our approach

  • One particular style:

    • Anisotropic part scales or part proportions


Our approach1

Our approach

  • One particular style:

    • Anisotropic part scales or part proportions

  • The approach:

    • Style-content separation with style clustering inacorrespondence-free way


Algorithm overview

Algorithm overview

Pipeline

Style clustering

Co-segmentation

Inter-style part

correspondence

Content

classification


Anisotropic part scales

Anisotropic part scales

Measure style distance between two shapes


Anisotropic part scales1

Anisotropic part scales

Measure style distance between two shapes

Part OBB graphs ofgiven segmentation


Anisotropic part scales2

Anisotropic part scales

Measure style distance between two shapes

Part OBB graphs ofgiven segmentation

Compute

style signatures


Anisotropic part scales3

Anisotropic part scales

Measure style distance between two shapes

Part OBB graphs ofgiven segmentation

Compute

style signatures

Euclidean

distance


Style distance issues

Style distance issues

Unknown segmentation

Unknown correspondence

?

?


Style distance

Style distance

Search over all part compositions and part counts

……

……


Style distance1

Style distance

For each part count, find minimal distance

……

A good signature will return min distance across all part counts to reflect corresponding part decompositions …

……


Correspondence free style signature

Correspondence-free style signature

UseLaplacian graph spectra:

Binary relations: difference of part scales between adjacent OBBs

OBB graph


Correspondence free style signature1

Correspondence-free style signature

Use Laplacian graph spectra:

Unary attributes: anisotropy of parts

Graph spectra is permutation-free

spherical

linear

planar

OBB graph


Style clustering

Style clustering

Spectral clustering using style distances


Pipeline

Pipeline

Style clustering

Co-segmentation

Inter-style part

correspondence

Content

classification


Co segmentation

Co-segmentation

Approach:

Consistent segmentation [Golovinskiy & Funkhouser, SMI 09]

Initial guess: global alignment (ICP)

[Golovinskiy & Funkhouser 09]


Co segmentation1

Co-segmentation

Approach:

Consistent segmentation [Golovinskiy & Funkhouser, SMI 09]

Initial guess: global alignment (ICP)

We co-segment within a style cluster

Removing non-homogeneous part scaling from analysis

[Golovinskiy & Funkhouser 09]


Co segmentation2

Co-segmentation

Approach:

Consistent segmentation [Golovinskiy & Funkhouser, SMI 09]

Initial guess: global alignment (ICP)

We co-segment within a style cluster

Removing non-homogeneous part scaling from analysis

[Golovinskiy & Funkhouser 09]

After style separation


Pipeline1

Pipeline

Style clustering

Co-segmentation

Inter-style part

correspondence

Content

classification


Inter style part correspondence

Inter-style part correspondence

Approach: deform-to-fit

Deformation-driven correspondence [Zhang et al., SGP 08]

Consider common interactions between OBBs

1D-to-1D

2D-to-3D

1D-to-2D

2D-to-2D


Inter style part correspondence1

Inter-style part correspondence

Deform-to-fit: appropriate deformation energy

Pruned priority-driven search


Pipeline2

Pipeline

Style clustering

Co-segmentation

Inter-style part

correspondence

Content

classification


Content classification

Content classification

Use Light Field Descriptor [Chen et al. 2003]

Compare corresponding parts

Part-level LFD

Global LFD


Synthesis by style transfer

Synthesis by style transfer

  • OBBs are scaled

  • Underlying geometry via space deformation

content

style transfer

style


Results hammers

Results: hammers


Results goblets

Results: goblets


Results humanoids

Results: humanoids


Results chairs

Results: chairs


Pros and cons4

Pros and cons

Pros:

Automatic generation of many variations

Unsupervised

Deals with anisotropic part scales

Variation = part scaling: structure preservation


Pros and cons5

Pros and cons

Pros:

Automatic generation of many variations

Unsupervised

Deals with anisotropic part scales

Variation = part scaling: structure preservation

Cons:

Rely on sufficiently good initial segmentations

Variation does not createnew content


Interesting future work

Interesting future work

Learn and synthesize with generic styles


Model driven creation as variation2

Model-driven creation as variation

New creation as variationof existing model(s)

Photo-inspired 3D model creation

Inspiration = photographs


Photo inspired 3d modeling

Photo-inspired 3D modeling

Photo-Inspired Model-Driven 3D Object Modeling

Kai Xu1,2, Hanlin Zheng4, Hao Zhang2, Daniel Cohen-Or3

Ligang Liu4, and Yueshan Xiong2

Conditionally accepted

1NUDT 2SFU3TAU 4ZJU


Overview

Overview

Input: single photograph + pre-analyzed dataset


Overview1

Overview

1. Model-driven labelled segmentation of photographed object


Overview2

Overview

2. Choosing of a candidate model from the database


Overview3

Overview

3. Silhouette-constrained deform-to-fit of candidate


Overview4

Overview

Output


Structure preservation

Structure preservation

Any higher-level structural info in the candidate models is preserved during deform-to-fit

Symmetry relations

Part-level correspondence in the set

Controller structures [Zheng et al. @ HKUST, EG 11]


Structure preservation1

Structure preservation

Any higher-level structural info in the candidate models is preserved during deform-to-fit

Symmetry relations

Part-level correspondence in the set

Controller structures [Zheng et al. @ HKUST, EG 11]

Structures also serve to constrain deformation of candidate model


Controller representations

Controller representations

Controllers: cuboids and generalized cylinders

Relations: symmetry, proximity, etc.

Fitting primitives


Controller representations1

Controller representations

Controllers: cuboids and generalized cylinders

Relations: symmetry, proximity, etc.

Fitting primitives


Deformation of controllers

Deformation of controllers

photo


Deformation of controllers1

Deformation of controllers

Controller primitives

photo

candidate model


Deformation of controllers2

Deformation of controllers

Controller primitives

Result of silhouette-driven deform-to-fit

photo

candidate model


Structure preservation at work

Structure preservation at work

symmetry


Structure preservation at work1

Structure preservation at work

proximity

symmetry


Structure preservation at work2

Structure preservation at work

proximity

symmetry

optimization


Structure preservation at work3

Structure preservation at work

proximity

symmetry

output

optimization

Short video


Results

Results

Guidance in single view but coherent 3D results


Results1

Results


The google chair challenge

The Google chair challenge


Not just chairs

Not just chairs …


Pros and cons6

Pros and cons

Pros:

Photos: immensely rich source of inspiration

Silhouette-driven deformation

Variation is less “intrusive” to retain high-level info of source model readily usable


Pros and cons7

Pros and cons

Pros:

Photos: immensely rich source of inspiration

Silhouette-driven deformation

Variation is less “intrusive” to retain high-level info of source model  more readily usable

Cons

Variation does not create new structures


Future work

Future work

Photo-inspired model deformation only a start

Further model refinement, e.g., via sketches


Future work1

Future work

Photo-inspired model deformation only a start

Further model refinement, e.g., via sketches

Model-driven structure modification


Future work2

Future work

Photo-inspired model deformation only a start

Further model refinement, e.g., via sketches

Model-driven structure modification

Other inspirations for 3D content creation

Sketch-inspired model variation


Future work3

Future work

Photo-inspired model deformation only a start

Further model refinement, e.g., via sketches

Model-driven structure modification

Other inspirations for 3D content creation

Sketch-inspired model variation

Style transfer with unknown style in a set


Thank you

Thank you, 谢谢

NUDT

TAU

ZJU

SFU


  • Login