image completion using global optimization l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Image Completion using Global Optimization PowerPoint Presentation
Download Presentation
Image Completion using Global Optimization

Loading in 2 Seconds...

play fullscreen
1 / 33

Image Completion using Global Optimization - PowerPoint PPT Presentation


  • 252 Views
  • Uploaded on

Image Completion using Global Optimization. Presented by Tingfan Wu. The Image Inpainting Problem. Outline. Introduction History of Inpainting Camps – Greedy & Global Opt. Model and Algorithm Markov Random Fields (MRF) & Inpainting Belief Propagation (BP) Priority BP Results

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Image Completion using Global Optimization' - Antony


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
outline
Outline
  • Introduction
    • History of Inpainting
    • Camps – Greedy & Global Opt.
  • Model and Algorithm
    • Markov Random Fields (MRF) & Inpainting
    • Belief Propagation (BP)
    • Priority BP
  • Results
  • Structural Propagation
method type
Method Type

PriorityTexture Synth.

Need User Guidance

method type6
Method Type

PriorityTexture Synth.

Need User Guidance

greedy v s global optmization

Ooops

Greedy v.s Global Optmization

Greedy Method

Global Optimization

Refine Globally 

Cannot go back 

outline8
Outline
  • Introduction
    • History of Inpainting
    • Camps – Greedy & Global Opt.
  • Model and Algorithm
    • Markov Random Fields (MRF) & Inpainting
    • Belief Propagation (BP)
    • Priority BP
  • Results
  • Structural Propagation
random fields belief network
Random Fields / Belief Network

Random Variable(Observation)

  • RF:Random Variables on Graph
  • Node : Random Var. (Hidden State)
  • Belief : from Neighbors, and Observation

Good Project Writer?(High Project grade)

Smart Student?(High GPA)

Good Test Taker?(High test score)

Good Employee

(No Observation yet)

Edge: Dependency

story about mrf
Story about MRF
  • (Bayesian) Belief Network (DAG)
  • Markov Random Fields (Undirected, Loopy)
  • Special Case:
    • 1D - Hidden Markov Model (HMM)

Hidden Markov Model (HMM)

Office Helper Wizard

inpainting as mrf optimization
Inpainting as MRF optimization
  • Node : Grid on target region, overlapped patches
  • Edge : A node depends only on its neighbors
  • Optimal labeling (hidden state) that minimizing mismatch energy
mrf potential functions
MRF Potential Functions

Mismatch (Energy) between ..

  • Vp (Xp ) : Source Image vs. New Label
  • Vpq(Xp, Xq) : Adjacent Labels
  • Sum of Square Distances (SSD) in Overlapping Region
outline14
Outline
  • Introduction
    • History of Inpainting
    • Camps – Greedy & Global Opt.
  • Model and Algorithm
    • Markov Random Fields (MRF) & Inpainting
    • Belief Propagation (BP)
    • Priority BP
  • Results
  • Structural Propagation
belief propagation 1 3
Belief Propagation(1/3)

Good Project Writer?(High Project grade)

Smart Student?(High GPA)

Good Test Taker?(High test score)

Good Employee

(No Observation yet)

  • Undirected and Loopy
  • Propagate forward and backward
belief propagation 2 3

X

X

q

p

Belief Propagation(2/3)
  • Message Forwarding
  • Iterative algorithm until converge

O(|Candidate|2)

Candidates at Node Q

Candidates at Node P

Neighbors (P)

priority bp
Priority BP
  • BP too slow:
    • Huge #candidates  Timemsg = O(|Candidates|2)
    • Huge #Pairs Cannot cache pairwise SSDs.
  • Observations
    • Non-Informative messages in unfilled regions
    • Solution to some nodes is obvious (fewer candidates.)
human wisdom
Human Wisdom

Candidates

Start from non-ambiguous part

And

Search for

Brown feather+green grass

Nobody start from here

priority bp20
Priority BP
  • Observations
    • needless messages in unfilled regions
    • Solution to some nodes is obvious (fewer candidates.)
  • Solution: Enhanced BP:
    • Easy nodes goes first (priority message scheduling)
    • Keep only highly possible candidates (maintain a Active Set)
priority pruning

?

?

?

?

?

?

?

?

Priority & Pruning

Discard Blue Points

High Priorityprune a lot

Low Priority

Candidates sorted by relative belief

Pruning may miss correct label

candidates after pruning
#Candidates after Pruning

Active Set (Darker means smaller)

Histogram of #candidates

Similar candidates

a closer look at priority bp
A closer look at Priority BP
  • Priority Calculation
    • Priority : 1/(#significant candidate)
  • Pruning (on the fly )
    • Discard Low Confidence Candidates
    • Similar patches  One representative (by clustering)
  • Result
    • More Confident More Pruning
    • Confident node helps increase neighbor’s confidence.
  • Warning:
    • PBP and Pruning must be used together
outline24
Outline
  • Introduction
    • History of Inpainting
    • Camps – Greedy & Global Opt.
  • Model and Algorithm
    • Markov Random Fields (MRF) & Inpainting
    • Belief Propagation (BP)
    • Priority BP
  • Results
  • Conclusion
  • Structural Propagation
results inpainting 1 3
Results-Inpainting(1/3)

Darker pixels  higher priority

Automatically start from salient parts.

slide27

Results-Inpainting(3/3)

  • Up to 2minutes / image (256x170) on P4-2.4G
more texture synthesis
More : Texture Synthesis
  • Interpolation as well as extrapolation
conclusion
Conclusion
  • Priority BP
    • {Confident node first} + {candidate pruning}
    • Generic – applicable to other MRF problems.
    • Speed up
  • MRF for Inpainting
    • Global optimization
      • avoid visually inconsistence by greedy
  • Priority BP for Inpainting
    • Automatically start from salient point.
sometimes
Sometimes …
  • Image contains hard high-level structure
    • Hard for computers
    • Interactive completion guided by human.
potential func for structural propagation
Potential Func. For Structural Propagation
  • User input a guideline by human region.
  • Potential Function respect distance between lines

Jian Sun et al, SIGGRAPH 2005

video
Video
  • Link:Microsoft Research