CS 691 Computational Photography. Instructor: Gianfranco Doretto Cutting Images. This Lecture: Finding Seams and Boundaries. Segmentation. This Lecture: Finding Seams and Boundaries. Retargetting. http://swieskowski.net/carve/. This Lecture: Finding Seams and Boundaries. Stitching.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
CS 691 Computational Photography
Instructor: Gianfranco Doretto
Cutting Images
Segmentation
Retargetting
http://swieskowski.net/carve/
Stitching
Fundamental Concept: The Image as a Graph
User provides imprecise and incomplete specification of region – your algorithm has to read his/her mind.
Key problems
What groups of pixels form cohesive regions?
What pixels are likely to be on the boundary of regions?
Which region is the user trying to select?
Node: pixel
Edge: cost of path or cut between two pixels
Mortenson and Barrett (SIGGRAPH 1995)
Mortenson and Barrett (SIGGRAPH 1995)
A good image boundary has a short path through the graph.
1
2
1
Start
2
1
9
1
5
4
4
3
3
3
6
1
1
1
End
Gradient Magnitude
Path Cost
Edge Image
Initialize, given seed s:
Loop while A is not empty
1. q = pixel in A with lowest cost
Magic Wand(198?)
Intelligent ScissorsMortensen and Barrett (1995)
GrabCut
User Input
Result
Regions
Regions & Boundary
Boundary
Source: Rother
Source (Label 0)
Cost to assign to 0
Cost to split nodes
Cost to assign to 1
Sink (Label 1)
Source (Label 0)
Cost to assign to 0
Cost to split nodes
Cost to assign to 1
Sink (Label 1)
Foreground (source)
Min Cut
Background(sink)
Cut:separatingsource and sink; Energy: collection of edges
Min Cut: Global minimal enegry in polynomial time
Image
constraints
Gaussian Mixture Model (typically 5-8 components)
R
R
Iterated graph cut
Foreground &Background
Foreground
G
Background
G
Background
Source: Rother
GrabCut – Interactive Foreground Extraction10
GrabCut – Interactive Foreground Extraction11
Camouflage &
Low Contrast
Fine structure
Harder Case
Initial Rectangle
InitialResult
Seam Carving – Avidan and Shamir (2007)
Demo: http://swieskowski.net/carve/
http://www.youtube.com/watch?v=6NcIJXTlugc
Seam Carving – Avidan and Shamir (2007)
Demo: http://swieskowski.net/carve/
DP is used to solve problems with the following characteristics:
2. Recursively define the value of an optimal solution
3. Compute the value bottom up
4. (if needed) Construct an optimal solution
Input: Matrices Ap×q and Bq×r (with dimensions p×q and q×r)
Result: Matrix Cp×r resulting from the product A·B
MATRIX-MULTIPLY(Ap×q , Bq×r)
1.for i ← 1 top
2.for j ← 1 tor
3.C[i, j]← 0
4.for k ← 1 toq
5.C[i, j]← C[i, j] + A[i, k]· B[k, j]
6.returnC
Scalar multiplication in line 5 dominates time to compute CNumber of scalar multiplications = pqr
Total: 7,500
Total: 75,000
0 if i=j
m[i, j ] =
min {m[i, k] + m[k+1, j ] + pi-1pk pj}if i<j
i≤ k< j
Input: Array p[0…n] containing matrix dimensions and n
Result: Minimum-cost table m and split table s
MATRIX-CHAIN-ORDER(p[ ], n)
for i← 1 ton
m[i, i]← 0
for l← 2 ton
for i← 1 ton-l+1
j← i+l-1
m[i, j]← ∞
for k← itoj-1
q ← m[i, k] + m[k+1, j] + p[i-1] p[k] p[j]
ifq < m[i, j]
m[i, j]← q
s[i, j]← k
returnm and s
Takes O(n3) time
Requires O(n2) space
2
_
=
overlap error
min. error boundary
overlapping blocks
vertical boundary
Graphcut Textures – Kwatra et al. SIGGRAPH 2003
Graphcut Textures – Kwatra et al. SIGGRAPH 2003
+
Ideal boundary:
Similar color in both images
High gradient in both images