Interactive Editing and Modeling of Bidirectional Texture Functions - PowerPoint PPT Presentation

Interactive editing and modeling of bidirectional texture functions
Download
1 / 51

Interactive Editing and Modeling of Bidirectional Texture Functions. Fr é do Durand MIT – CSAIL. Jan Kautz University College London. Solomon Boulos University of Utah. Introduction. Bidirectional Texture Functions (BTFs) A representation for complex materials [Dana99]

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Interactive Editing and Modeling of Bidirectional Texture Functions

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Interactive editing and modeling of bidirectional texture functions

Interactive Editing and Modeling of Bidirectional Texture Functions

Frédo Durand

MIT – CSAIL

Jan Kautz

University College London

Solomon Boulos

University of Utah


Introduction

Introduction

  • Bidirectional Texture Functions (BTFs)

    • A representation for complex materials [Dana99]

    • Collection of textures under many different viewand light directions

      • 6D data structure: 2D spatial, 2D view, 2D light

view: top

light: top

view: top

light: 50º

view: 45º

light: 50º

view: 50º

light: 60º


Introduction1

Introduction

  • Bidirectional Texture Functions

    • Usually acquired from real materials, e.g. [Sattler03]

    • Include real-world effects (scattering, occlusions, transparencies, parallax, …)

    • Very realistic

Lamp

Camera

Sample(on robotic arm)


Introduction2

Introduction

  • Acquisition of Bidirectional Texture Functions

    • Is very expensive (time, setup, …)

    • University of Bonn BTFs have:

      • 81 view x 81 light images (6561 images)

      • Takes several hours to capture

    • Only 10 publicly available


Introduction3

Introduction

  • Acquisition of Bidirectional Texture Functions

    • Is very expensive (time, setup, …)

    • University of Bonn BTFs have:

      • 81 view x 81 light images (6561 images)

      • Takes several hours to capture

    • Only 10 publicly available

Want better return on investment:

 Editing of BTFs

Allows to create variations from single BTF

Create BTF from flat texture


Introduction4

Introduction

  • How to edit BTFs?


Introduction5

Introduction

  • How to edit BTFs?

    • Everything is baked in

    • No explicit knowledge of

      • BRDFs

      • Geometry

      • Transparencies , etc.

  • Extraction of these properties is

    • Difficult

    • How to support editing of high-level properties


Related work

Related Work

  • Most work focuses on

    • BTF Acquisition [Dana99,Sattler03,Koudelka03,…]

    • BTF Rendering/Compression [Sattler03, Meseth04,Vasilescu04,…]

  • Very little on BTF editing

    • Pasting BTFs onto surfaces [Zhou05]

    • Self-similarity-based editing [Dong05]

    • Structural editing of BTFs [Mueller07]

    • BRDF editing [Lawrence06, Colbert06]


Our btf editing approach

Our BTF Editing Approach

  • Simple operators

    • Can be applied selectively (with selections)

  • Enables powerful editing: “BTFShop”

VIDEO


Our btf editing approach1

Our BTF Editing Approach

  • Philosophy

    • Simple operators → change material appearance

  • Rely on simple aspects of material perception:

    • Contrast ↔ roughness [Koenderink02]

    • Histograms ↔ roughness [Ginneken99, Leung97]

    • Sharpness ↔ subsurface scattering [Fleming04]

      → Enables simple operators to be effective


Our btf editing operators

Our BTF Editing Operators

Operators

  • E.g., color manipulation, blur/sharpen

  • Work on certain dimensions

  • The simpler the better

    • Use raw data, avoid big inverse problems

    • Exploit shape information, when appropriate

  • Constrained to user selections

    • E.g., highlight pixels, directional range


  • Overview

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

      Due to time: only highlight one example each


    Overview1

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • Low-level operators

      • Blur / sharpen

      • Color manipulation

    • Give direct control of simple material properties

      • Shading & roughness


    Shading changing specularity

    lower-frequency

    higher-frequency

    Shading – Changing Specularity

    • Goal:

    • Specularity relates to frequencies in light-space [Ramamoorthi01]:

    blur/sharpen in light-space

    Light-dependencefor given view and location

    Light-dependencefor given view and location


    Shading changing specularity results part 1

    Original

    Blur in Light-Space

    Reference

    Shading – Changing SpecularityResults – Part 1

    (BTF computed from synthetic micro-geometry)

    (BTF computed from synthetic micro-geometry)


    Shading changing specularity results part 2

    Blur in Light-Space

    Original

    Sharpen in Light-Space

    Shading – Changing SpecularityResults – Part 2

    (Measured BTF)


    Shading changing roughness

    Shading – Changing Roughness

    • Goal: modify roughness

    ?


    Shading changing roughness1

    Output Intensity

    Input Intensity

    Shading – Changing Roughness

    • Observation:

      • Distribution of light vs. dark pixels relates to roughness [Pont and Koenderink 02]

    • Distribution can be changed with curve tool

      • User-controlledcurve

      • Remaps input to output intensities


    Talk ppt21mb

    remappingcurve

    BTFShop

    Reference

    (BTF: )

    synthetic micro-geometry

    Original

    (BTF: )

    synthetic micro-geometry


    Overview2

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • More complex editing operations

      • Requiring selections for effective modification


    Shadow removal

    Shadow Removal

    • Goal:

    • Challenges:

      • How to find shadow areas?

      • How to fill them in automatically?


    Shadow removal finding shadow areas

    Shadow Removal –Finding Shadow Areas

    • User-assisted selection

      • Fully-lit, frontal texture slice: always shadow-free

      • Check ratio of: fully-lit frontal slice / each of light-varying frontal slices

      • If (scaled) ratio < user-threshold: pixel in shadow

    /

    =

    fully-lit, frontal

    side-lit, frontal

    ratio

    thresholdapplied


    Shadow removal finding shadow areas1

    fully-lit, frontal

    fully-lit, frontal

    side-lit, frontal

    side-lit, frontal

    ratio

    ratio

    thresholded

    thresholded

    Shadow Removal –Finding Shadow Areas

    /

    =

    /

    =


    Shadow removal finding shadow areas2

    Shadow Removal –Finding Shadow Areas

    Found Shadow Areas


    Shadow removal automatically filling in shadows

    Shadow Removal –Automatically Filling in Shadows

    Two-stage process

    • Fill in selected areas with data from fully-lit frontal texture (never contains shadows)

    • Adjust average brightness and saturation to match surrounding area


    Shadow removal automatically filling in shadows1

    Shadow Removal –Automatically Filling in Shadows

    Original


    Shadow removal automatically filling in shadows2

    Shadow Removal –Automatically Filling in Shadows

    Selected Shadow Areas


    Shadow removal automatically filling in shadows3

    Shadow Removal –Automatically Filling in Shadows

    Shadow Areas Filled In From Fully-Lit, Frontal Texture


    Shadow removal automatically filling in shadows4

    Shadow Removal –Automatically Filling in Shadows

    Brightness/Saturation Adjusted


    Shadow removal results

    Label

    Shadow Removal – Results

    Shading is

    preserved

    Original

    Shadows Removed


    Overview3

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • Now:

      • Editing operator requiring inferred information


    Geometry parallax

    Rendering

    (x1,y1)

    (x2,y2)

    Geometry – Parallax

    • Parallax is common in complex materials

    • Geometric point → different (x,y)


    Geometry parallax1

    Geometry – Parallax

    • Shows up in BTF slices:

    • Consequences:

      • Spatial editing such as painting needs to take it into account

      • Change of geometric structure → warp in texture space

    Frontal Slice

    Oblique Slice

    Frontal Slice

    Oblique Slice

    Original Height

    Increased Height


    Parallax spatial editing

    Parallax – Spatial Editing

    Extract Heightfield

    (Shape from Shadows)

    Unwarp Texture Slices

    (Removes Parallax, Aligns Texture Slices)

    Spatial Editing

    Warp Texture Slices

    (Reintroduce Parallax)


    Parallax spatial editing result

    Parallax – Spatial Editing Result

    No Correction

    Parallax Correction


    Geometric modification

    Geometric Modification

    • Parallax Warp can also be used to impose different geometric structure

    Heightfield


    Local geometric appearance

    Local Geometric Appearance

    • Parallax Warp

      • Does not modify underlying shading

    • Bump mapping [Blinn78]: modify local view and light direction

    +

    =

    Diffuse

    Heightfield

    Bump Mapped


    Local geometric appearance1

    Local Geometric Appearance

    • Use same idea for BTFs

      • Rotate lookup directions w.r.t. heightfield

      • Lookup values with rotated directions

      • Store values again as BTF


    Local geometric appearance2

    Bump Mapped BTF

    Bump Mapped + Warped BTF

    Local Geometric Appearance

    Original BTF

    Heightfield


    Overview4

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects


    Global effects

    BTFShop

    Global Effects

    • Asperity Scattering [Koenderink03]

      • “Glow” at grazing angles

      • Simulate using angular selection and curve tool

    No Fuzz

    (BTF: synth. micro-geo.)

    Fuzz – Reference

    (BTF: synth. micro-geo.)


    Overview5

    Use to

    modifyBTFs

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • More tools / selections: see paper

    Edit Albedos

    Modify Shadows

    Edit Thickness

    Edit Subsur. Scat.


    More results

    More Results

    VIDEO


    Overview6

    Use to

    createBTFs

    Overview

    • Shading

    • Shadowing

    • Geometry

    • Global Effects

    • More tools / selections: see paper


    Modeling of btfs

    Modeling of BTFs

    Single Texture

    BTF Created


    Modeling of btfs1

    Modeling of BTFs

    Single Texture

    Created BTF

    Reference BTF


    Modeling of btfs2

    Modeling of BTFs

    VIDEO


    Btfshop system

    BTFShop – System

    • Multi-threaded, tile-based architecture with efficient out-of-core caching

    • Visual feedback in ~2s (touch 5% data)

    • Complete BTF (1.6GB data): ~minutes


    Discussion

    Discussion

    • Approach works well for tweaking BTF appearance:

      • Shading, Shadowing, Geometry, and Global Effects

    • Limitations

      • Cannot turn bark into wool

      • Resulting BTFs not physically correct

      • Operators edit one effect at a time


    Conclusions

    Conclusions

    • Simple operators and selections → surprisingly effective

    • Mostly work on raw 6D data

      • Infer properties when necessary

    • BTFShop: out-of-core (6D dataset)

    • BTF

      • Modification

      • Creation from single textures


    Acknowledgements

    Acknowledgements

    • Pete Shirley

    • Mary Williamson and Omari Dennis for PBRT tools

    • Reviewers and referees

    • University of Bonn and UCSD for BTF data

    • NSF Career Award 0447561, Microsoft Research New Faculty Fellowship, and Sloan Fellowship

    • University of Utah Brown Fellowship

    • German Research Foundation Emmy-Noether Fellowship

    • Code will be available at http://btfshop.sourceforge.net


  • Login