interactive texturing on objects in images via a sketching interface n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Interactive Texturing on Objects in Images via a Sketching Interface PowerPoint Presentation
Download Presentation
Interactive Texturing on Objects in Images via a Sketching Interface

Loading in 2 Seconds...

play fullscreen
1 / 47

Interactive Texturing on Objects in Images via a Sketching Interface - PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on

Interactive Texturing on Objects in Images via a Sketching Interface. Kwanrattana Songsathaporn. The University of Tokyo. Henry Johan. Nanyang Technological University. Tomoyuki Nishita. The University of Tokyo. Overview. Background & System Overview Related Work

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Interactive Texturing on Objects in Images via a Sketching Interface' - hagen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
interactive texturing on objects in images via a sketching interface

Interactive Texturing on Objects in Images via a Sketching Interface

KwanrattanaSongsathaporn

The University of Tokyo

Henry Johan

Nanyang Technological University

TomoyukiNishita

The University of Tokyo

overview
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
overview1
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
research background
Research Background

Paste textures on textured area

Paste textures on blank area

  • Paste textures on images, and maintain underlying shapes of objects
  • Applications in digital image/video manipulation
scope features 1 3
Scope & Features(1/3)

Drawing

Photograph

Interactive system for pasting textures

Input image: Drawings and Photographs

scope features 2 3
Scope & Features(2/3)

http://en.wikipedia.org/wiki/Texture_synthesis

(upper) Textures with pattern

(lower) Texture without pattern

Our system supports textures regardless of their patterns

scope features 3 3
Scope & Features(3/3)

Perspective

Occlusion

  • Two features to make texturing easier
    • Systematic occlusion handling: create occlusion with one piece of texture
    • Systematic perspective handling: consistency to surrounding
system framework
System Framework

Specifications using interface

Objects’ Normal Vector Field Construction

Texture Coordinate Calculation

Input image

Output image

Texture image

User specifications

Video

Texture mapping is based on normal vector field=> independent of patterns

overview2
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Parameterization
  • Results
  • Conclusions and Future Work
texturing in images 1 3
Texturing in Images(1/3)

(a) Default

grid mesh

(b) Aligned vertices

to texels

  • Liu et al., SIGGRAPH 2004
    • Users align vertices on grid mesh with texels to replace textures
    • Not support input objects without textures
    • A lot of user interactions
texturing in images 2 3
Texturing in Images(2/3)

Output image

Input image

Constructed normal field

  • Fang and Hart, SIGGRAPH 2004
    • Synthesize texture on objects in images
    • Construct normal vector field with shape-from-shading
    • Constrained by the limitation of shape-from-shading
    • Not interactive
texturing in images 3 3
Texturing in Images(3/3)

Designed normal vector field

Paste texture

Add shading

  • Winnemöller et al., EGSR 2009
    • Use Diffusion Curves (DC) to design
      • Normal vector field
      • uv map
    • Construct normal vector field without shading
    • Require a lot of user interactions
comparison of methods
Comparison of Methods

support

not support

with limitations

normal construction interface
Normal Construction Interface

Specifications to transfer normal vectors

3D model from normal map

  • Wu et al., SIGGRAPH 2007
    • Sketch-based interface: Shape palette
    • Transfer normal vectors from shape palettes
    • Easy-to-understand interface
overview3
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
two phase user interface
Two-Phase User Interface

Objects’ Normal Vector Field Construction

Texture Coordinate Calculation

sparse normal vector specification
Sparse Normal Vector Specification

Convex palette

Concave palette

Input image

Normal vector field construction using our interface

meaning of palettes
Meaning of Palettes

z

y

x

  • Convex palette
  • Imaginary 3D image of convex palette

Imagine that texturing region and shape palettes are 3D objects

Monitor screen is image plane

split stroke specification
Split Stroke Specification

(a) Specify texturing region

(b),(c) Subregions

  • Discontinuity on the normal vector field
    • Occlusion and edge
  • Split single region into subregions
    • Initial normal vector field of each subregion can be specified separately
distance between subregions
Distance Between Subregions

(a) split region

(b) Distance between subregions = 0

(c) Distance between subregions > 0

(d) Black strip shows occluded part

(e) Drag slide bar to increase distance

Imaginary top view of the curtain

Approximate occluded region by specifying the distance between subregions > 0

correcting perspective projection
Correcting Perspective Projection
  • Furthest and nearest points (pink points)
  • Interface for specifying relative depth
  • Result from default perspective projection matrix
  • Result from corrected perspective projection matrix

Specify depths at furthest and nearest point (pink points)

Pink points are estimated by system

overview4
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
overview5
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
normal vector field construction
Normal Vector Field Construction

z

y

x

Direction of x, y and z components

normal vector field construction1
Normal Vector Field Construction

: set of normal specified pixels

: subregion which pixel i is a member

: adjacent pixels of i

: set of all pixels in texturing region

normal vector field construction2
Normal Vector Field Construction

Aim to find this value of every pixel

: set of normal specified pixels

: subregion which pixel i is a member

: adjacent pixels of i

: set of all pixels in texturing region

normal vector field construction3
Normal Vector Field Construction

Sparsely specified normal vectors

: set of normal specified pixels

: subregion which pixel i is a member

: adjacent pixels of i

: set of all pixels in texturing region

normal vector field construction4
Normal Vector Field Construction

Minimize error from the specified value

: set of normal specified pixels

: subregion which pixel i is a member

: adjacent pixels of i

: set of all pixels in texturing region

normal vector field construction5
Normal Vector Field Construction

Enforce smoothness

: set of normal specified pixels

: subregion which pixel i is a member

: adjacent pixels of i

: set of all pixels in texturing region

normal vector field construction6
Normal Vector Field Construction

Adjacent pixels on the same subregion

: set of normal specified pixels

: subregion which pixel i is a member

: adjacent pixels of i

: set of all pixels in texturing region

normal vector field construction7
Normal Vector Field Construction

Minimize curvature

: set of normal specified pixels

: subregion which pixel i is a member

: adjacent pixels of i

: set of all pixels in texturing region

overview6
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
what we know
What We Know?

Next, texture mapping

Normal vector field of the objects (Normal vector field construction)

Corrected perspective projection (Relative depth specification)

texture coordinate calculation
Texture Coordinate Calculation

p: (u,v) known

r: (u,v) unknown

Goal: Find texture coordinate (u,v) of every pixels in texturing region

Subregion-by-subregion, flood-filled manner from pinpoint

as the calculation continues
As the calculation continues

Initial state

Flood fill

p: (u,v) known

r: (u,v) unknown

Arithmetic weighted mean

overview7
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
results 1 2
Results(1/2)

(b) Retextured

image

(a) Input

(c) Texture

  • Interaction time (mins)
    • Winnemölleret al. : 60
    • Our system: 30
results 2 2
Results(2/2)

(a) Input

(b,c) Retextured image

  • Interaction time (mins)
    • Winnemölleret al.: 30
    • Our system: 16
limitation
Limitation

Distortion is visible in the green box area

  • Distortion cumulates at distant area from pinpoint (red point)
overview8
Overview
  • Background & System Overview
  • Related Work
  • Proposed User Interface for Texturing
  • Underlying Algorithms
    • Normal Vector Field Construction
    • Texture Coordinate Calculation
  • Results
  • Conclusions and Future Work
conclusions
Conclusions
  • Interactive system for texturing objects in images
    • Independent of texture patterns
    • Systematically handle occlusion
    • Systematically handle perspective
  • Reduce user interactions
  • Produce results with reasonable shorter time
future work
Future Work

Photograph shows complex occlusion

  • Intuitive user interface that handles complex occlusion systematically
  • User study
    • How precisely could users perceive normal vectors on objects?
    • How precisely could users specify normal vectors using our interface?
as the calculation continues1
As the calculation continues

Position of adjacent pixels

Position of pinpoint

p: (u,v) known

r: (u,v) unknown