Interactive editing and modeling of bidirectional texture functions
Download
1 / 51

Talk PPT21MB - PowerPoint PPT Presentation


  • 269 Views
  • Uploaded on

Interactive Editing and Modeling of Bidirectional Texture Functions. Fr é do Durand MIT – CSAIL. Jan Kautz University College London. Solomon Boulos University of Utah. Introduction. Bidirectional Texture Functions (BTFs) A representation for complex materials [Dana99]

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Talk PPT21MB' - Gabriel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Interactive editing and modeling of bidirectional texture functions

Interactive Editing and Modeling of Bidirectional Texture Functions

Frédo Durand

MIT – CSAIL

Jan Kautz

University College London

Solomon Boulos

University of Utah


Introduction
Introduction Functions

  • Bidirectional Texture Functions (BTFs)

    • A representation for complex materials [Dana99]

    • Collection of textures under many different viewand light directions

      • 6D data structure: 2D spatial, 2D view, 2D light

view: top

light: top

view: top

light: 50º

view: 45º

light: 50º

view: 50º

light: 60º


Introduction1
Introduction Functions

  • Bidirectional Texture Functions

    • Usually acquired from real materials, e.g. [Sattler03]

    • Include real-world effects (scattering, occlusions, transparencies, parallax, …)

    • Very realistic

Lamp

Camera

Sample(on robotic arm)


Introduction2
Introduction Functions

  • Acquisition of Bidirectional Texture Functions

    • Is very expensive (time, setup, …)

    • University of Bonn BTFs have:

      • 81 view x 81 light images (6561 images)

      • Takes several hours to capture

    • Only 10 publicly available


Introduction3
Introduction Functions

  • Acquisition of Bidirectional Texture Functions

    • Is very expensive (time, setup, …)

    • University of Bonn BTFs have:

      • 81 view x 81 light images (6561 images)

      • Takes several hours to capture

    • Only 10 publicly available

Want better return on investment:

 Editing of BTFs

Allows to create variations from single BTF

Create BTF from flat texture


Introduction4
Introduction Functions

  • How to edit BTFs?


Introduction5
Introduction Functions

  • How to edit BTFs?

    • Everything is baked in

    • No explicit knowledge of

      • BRDFs

      • Geometry

      • Transparencies , etc.

  • Extraction of these properties is

    • Difficult

    • How to support editing of high-level properties


Related work
Related Work Functions

  • Most work focuses on

    • BTF Acquisition [Dana99,Sattler03,Koudelka03,…]

    • BTF Rendering/Compression [Sattler03, Meseth04,Vasilescu04,…]

  • Very little on BTF editing

    • Pasting BTFs onto surfaces [Zhou05]

    • Self-similarity-based editing [Dong05]

    • Structural editing of BTFs [Mueller07]

    • BRDF editing [Lawrence06, Colbert06]


Our btf editing approach
Our BTF Editing Approach Functions

  • Simple operators

    • Can be applied selectively (with selections)

  • Enables powerful editing: “BTFShop”

VIDEO


Our btf editing approach1
Our BTF Editing Approach Functions

  • Philosophy

    • Simple operators → change material appearance

  • Rely on simple aspects of material perception:

    • Contrast ↔ roughness [Koenderink02]

    • Histograms ↔ roughness [Ginneken99, Leung97]

    • Sharpness ↔ subsurface scattering [Fleming04]

      → Enables simple operators to be effective


Our btf editing operators
Our BTF Editing Operators Functions

Operators

  • E.g., color manipulation, blur/sharpen

  • Work on certain dimensions

  • The simpler the better

    • Use raw data, avoid big inverse problems

    • Exploit shape information, when appropriate

  • Constrained to user selections

    • E.g., highlight pixels, directional range


  • Overview
    Overview Functions

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

      Due to time: only highlight one example each


    Overview1
    Overview Functions

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • Low-level operators

      • Blur / sharpen

      • Color manipulation

    • Give direct control of simple material properties

      • Shading & roughness


    Shading changing specularity

    lower- Functionsfrequency

    higher-frequency

    Shading – Changing Specularity

    • Goal:

    • Specularity relates to frequencies in light-space [Ramamoorthi01]:

    blur/sharpen in light-space

    Light-dependencefor given view and location

    Light-dependencefor given view and location


    Shading changing specularity results part 1

    Original Functions

    Blur in Light-Space

    Reference

    Shading – Changing SpecularityResults – Part 1

    (BTF computed from synthetic micro-geometry)

    (BTF computed from synthetic micro-geometry)


    Shading changing specularity results part 2

    Blur in Light-Space Functions

    Original

    Sharpen in Light-Space

    Shading – Changing SpecularityResults – Part 2

    (Measured BTF)


    Shading changing roughness
    Shading – Changing Roughness Functions

    • Goal: modify roughness

    ?


    Shading changing roughness1

    Output Intensity Functions

    Input Intensity

    Shading – Changing Roughness

    • Observation:

      • Distribution of light vs. dark pixels relates to roughness [Pont and Koenderink 02]

    • Distribution can be changed with curve tool

      • User-controlledcurve

      • Remaps input to output intensities


    remapping Functionscurve

    BTFShop

    Reference

    (BTF: )

    synthetic micro-geometry

    Original

    (BTF: )

    synthetic micro-geometry


    Overview2
    Overview Functions

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • More complex editing operations

      • Requiring selections for effective modification


    Shadow removal
    Shadow Removal Functions

    • Goal:

    • Challenges:

      • How to find shadow areas?

      • How to fill them in automatically?


    Shadow removal finding shadow areas
    Shadow Removal – FunctionsFinding Shadow Areas

    • User-assisted selection

      • Fully-lit, frontal texture slice: always shadow-free

      • Check ratio of: fully-lit frontal slice / each of light-varying frontal slices

      • If (scaled) ratio < user-threshold: pixel in shadow

    /

    =

    fully-lit, frontal

    side-lit, frontal

    ratio

    thresholdapplied


    Shadow removal finding shadow areas1

    fully-lit, frontal Functions

    fully-lit, frontal

    side-lit, frontal

    side-lit, frontal

    ratio

    ratio

    thresholded

    thresholded

    Shadow Removal –Finding Shadow Areas

    /

    =

    /

    =


    Shadow removal finding shadow areas2
    Shadow Removal – FunctionsFinding Shadow Areas

    Found Shadow Areas


    Shadow removal automatically filling in shadows
    Shadow Removal – FunctionsAutomatically Filling in Shadows

    Two-stage process

    • Fill in selected areas with data from fully-lit frontal texture (never contains shadows)

    • Adjust average brightness and saturation to match surrounding area


    Shadow removal automatically filling in shadows1
    Shadow Removal – FunctionsAutomatically Filling in Shadows

    Original


    Shadow removal automatically filling in shadows2
    Shadow Removal – FunctionsAutomatically Filling in Shadows

    Selected Shadow Areas


    Shadow removal automatically filling in shadows3
    Shadow Removal – FunctionsAutomatically Filling in Shadows

    Shadow Areas Filled In From Fully-Lit, Frontal Texture


    Shadow removal automatically filling in shadows4
    Shadow Removal – FunctionsAutomatically Filling in Shadows

    Brightness/Saturation Adjusted


    Shadow removal results

    Label Functions

    Shadow Removal – Results

    Shading is

    preserved

    Original

    Shadows Removed


    Overview3
    Overview Functions

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • Now:

      • Editing operator requiring inferred information


    Geometry parallax

    Rendering Functions

    (x1,y1)

    (x2,y2)

    Geometry – Parallax

    • Parallax is common in complex materials

    • Geometric point → different (x,y)


    Geometry parallax1
    Geometry – Parallax Functions

    • Shows up in BTF slices:

    • Consequences:

      • Spatial editing such as painting needs to take it into account

      • Change of geometric structure → warp in texture space

    Frontal Slice

    Oblique Slice

    Frontal Slice

    Oblique Slice

    Original Height

    Increased Height


    Parallax spatial editing
    Parallax – Spatial Editing Functions

    Extract Heightfield

    (Shape from Shadows)

    Unwarp Texture Slices

    (Removes Parallax, Aligns Texture Slices)

    Spatial Editing

    Warp Texture Slices

    (Reintroduce Parallax)


    Parallax spatial editing result
    Parallax – Spatial Editing Result Functions

    No Correction

    Parallax Correction


    Geometric modification
    Geometric Modification Functions

    • Parallax Warp can also be used to impose different geometric structure

    Heightfield


    Local geometric appearance
    Local Geometric Appearance Functions

    • Parallax Warp

      • Does not modify underlying shading

    • Bump mapping [Blinn78]: modify local view and light direction

    +

    =

    Diffuse

    Heightfield

    Bump Mapped


    Local geometric appearance1
    Local Geometric Appearance Functions

    • Use same idea for BTFs

      • Rotate lookup directions w.r.t. heightfield

      • Lookup values with rotated directions

      • Store values again as BTF


    Local geometric appearance2

    Bump Mapped BTF Functions

    Bump Mapped + Warped BTF

    Local Geometric Appearance

    Original BTF

    Heightfield


    Overview4
    Overview Functions

    • Shading

    • Shadowing

    • Geometry

    • Global Effects


    Global effects

    BTFShop Functions

    Global Effects

    • Asperity Scattering [Koenderink03]

      • “Glow” at grazing angles

      • Simulate using angular selection and curve tool

    No Fuzz

    (BTF: synth. micro-geo.)

    Fuzz – Reference

    (BTF: synth. micro-geo.)


    Overview5

    Use to Functions

    modifyBTFs

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • More tools / selections: see paper

    Edit Albedos

    Modify Shadows

    Edit Thickness

    Edit Subsur. Scat.


    More results
    More Results Functions

    VIDEO


    Overview6

    Use to Functions

    createBTFs

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • More tools / selections: see paper


    Modeling of btfs
    Modeling of BTFs Functions

    Single Texture

    BTF Created


    Modeling of btfs1
    Modeling of BTFs Functions

    Single Texture

    Created BTF

    Reference BTF


    Modeling of btfs2
    Modeling of BTFs Functions

    VIDEO


    Btfshop system
    BTFShop – System Functions

    • Multi-threaded, tile-based architecture with efficient out-of-core caching

    • Visual feedback in ~2s (touch 5% data)

    • Complete BTF (1.6GB data): ~minutes


    Discussion
    Discussion Functions

    • Approach works well for tweaking BTF appearance:

      • Shading, Shadowing, Geometry, and Global Effects

    • Limitations

      • Cannot turn bark into wool

      • Resulting BTFs not physically correct

      • Operators edit one effect at a time


    Conclusions
    Conclusions Functions

    • Simple operators and selections → surprisingly effective

    • Mostly work on raw 6D data

      • Infer properties when necessary

    • BTFShop: out-of-core (6D dataset)

    • BTF

      • Modification

      • Creation from single textures


    Acknowledgements
    Acknowledgements Functions

    • Pete Shirley

    • Mary Williamson and Omari Dennis for PBRT tools

    • Reviewers and referees

    • University of Bonn and UCSD for BTF data

    • NSF Career Award 0447561, Microsoft Research New Faculty Fellowship, and Sloan Fellowship

    • University of Utah Brown Fellowship

    • German Research Foundation Emmy-Noether Fellowship

    • Code will be available at http://btfshop.sourceforge.net


    ad