- 114 Views
- Uploaded on
- Presentation posted in: General

Intelligent Scissors for Image Composition

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Intelligent Scissors for Image Composition

Anthony Dotterer

01/17/2006

- Title
- Intelligent Scissors for Image Composition

- Author
- Eric N. Mortensen
- William A. Barrett

- Publication
- 1995

- Interactive image segmentation and composition tool
- Easy to use
- Quick
- Quality output

- Features
- Best path along image edges
- Cooling
- On-the-fly training
- Source to destination warping and composition
- Destination matching

- Need
- Optimal path along edges starting at a ‘seed’ point
- Optimal path creation must be quick

- Solution
- Use dynamic programming to create path reference
- Local cost definition
- Path reference creation

- Use dynamic programming to create path reference

- Define l(p,q) as the cost for going from pixel p to pixel q
- Incorporate the several edge functions into the cost
- Laplacian Zero-Crossing, fZ(q)
- Gradient Magnitude, fG(q)
- Gradient Direction, fD (p,q)

- Relate edge functions to the cost function
- Use ωZ,ωD,ωGas constants to weight features
l(p,q) = ωZ · fZ(q) + ωD · fD (p,q) + ωG · fG(q)

- Use ωZ,ωD,ωGas constants to weight features

- Properties
- Approximate 2nd partial derivative of Image
- Zero-crossings represent maxima and minima
- Good image edges

- Cost
- Define IL(q) as Laplacian at pixel q
- Get low cost by defining Laplacian as a binary
fZ(q) = { 0; if IL(q) = 0, 1; if IL(q) ≠ 0

- Issue
- Zeros rarely occur

- Solution
- Use pixel closest to zero

- Examples
- Image (top)
- Laplacian (bottom)

- Properties
- Magnitude of 1st partial derivatives of an image
- Direct correlation between edge strength and local cost

- Cost
- Define G as gradient magnitude
G = √(Ix² + Iy²)

- Get low cost by inverting and scaling
fG = 1 – G / (max(G))

- Also factor in Euclidean distance
- Scale adjacent pixels cost by 1
- Scale diagonal pixels cost by 1/√2

- Define G as gradient magnitude

- Examples
- Original (top left)
- Gradient Magnitude (top right)
- Inverted & Scaled Gradient Magnitude (bottom)

- Properties
- Vectors created by the 1st derivatives of an image
- High cost for shape changes
- Adds smoothing constraint

- Cost
- Give low costs to gradients in the same direction
- Define D(p) as the unit vector perpendicular to the gradient vector at point p
D(p) = norm(Iy(p), -Ix(p))

- Cost
- Define L(p, q) to be the link between point q and p, such that
L(p, q) = { q – p; if D(p) · (q – p) ≥ 0,

p – q; if D(p) · (q – p) < 0

- Let dp(p, q) and dq(p, q) as follows
dp(p, q) = D(p) · L(p, q)

dq(p, q) = L(p, q) · D(q)

- Finally, the cost function
fD(p, q) = 1/π ( cos-1(dp(p, q)) + cos-1(dq(p, q)) )

- Define L(p, q) to be the link between point q and p, such that

Let

p = (3, 3)

q = (3, 4)

D(p) = (0, 1)

D(q) = (0, 1)

Calculate L(p, q)

L(p, q) = ((3, 4) – (3, 3)) = (0, 1)

Determine d(p, q)

dp(p, q) = (0, 1) · (0, 1) = 1

dq(p, q) = (0, 1) · (0, 1) = 1

Finally fD(p, q)

fD(p, q) = 1/π ( 0 + 0 ) = 0

Low Cost!

Let

p = (3, 3)

q = (4, 3)

D(p) = (0, 1)

D(q) = (0, 1)

Calculate L(p, q)

L(p, q) = ((4, 3) – (3, 3)) = (1, 0)

Determine d(p, q)

dp(p, q) = (0, 1) · (1, 0) = 0

dq(p, q) = (1, 0) · (0, 1) = 0

Finally fD(p, q)

fD(p, q) = 1/π ( π/2 + π/2 ) = 1

High Cost!

- Differs from method studied in class
- No stages
- Link cost between nodes changes
- No destination

- Inputs
- Seed point, s
- Local cost function, l(q, r)

- Data structures
- Sorted list of active pixels, L
- Neighborhood of pixel q, N(q)
- Flag map of expanded pixels, e(q)
- Cumulative cost from seed point, g(q)

- Output
- Path reference map, p

- Start at seed point
- Cost is adjusted for Euclidean distance

- Put all neighbor pixels into the active list
- No other pixel has yet to be expanded

- Set pointers for all neighbors to the seed point

1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

8

9

7

2

11

4

1

13

7

7

L = (4,8), (3,7), …

- Expand to least cost node
- Remove that node from active list

- Calculate cumulative cost of all neighbor pixels
- Excludes seed point

- Change pointers of neighbor pixels
- Only if new cost is smaller and pixel is not expanded

- Add or replace neighbor pixels into active list
- Do nothing if pointer was not updated

1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

8

9

7

2

9

5

4

1

6

13

7

6

14

L = (3,7), (2,8), (5,7) …

- Expand to least cost node
- Remove that node from active list

- Calculate cumulative cost of all neighbor pixels
- Excludes expanded pixels

- Change pointers of neighbor pixels
- Only if new cost is smaller and pixel is not expanded

- Add or replace neighbor pixels into active list
- Do nothing if pointer was not updated

1 2 3 4 5 6 7 8 9 10 11

1

2

3

4

5

6

7

8

9

6

6

12

14

23

7

2

9

5

9

20

16

4

1

6

13

18

13

7

6

14

L = (2,6), (3,6), (4,9), …

- Finished
- No more pixels to expand
- No more pixels on active list

- Mouse will constantly redraw optimal path
- A wire will ‘snap’ to objects with an image

- New seed points
- New seed points must be defined to surround an object
- Points will ‘snap’ to nearest edge

- Problem
- All seeds must be manually selected
- Complex objects may require many seed points

- All seeds must be manually selected
- Solution
- Apply automatic seed point
- As the user wraps the object, a common path is formed
- Make common path ‘cool’ into a new seed point

- Examples
- Manual seed points (bottom left)
- Auto seed points via cooling (bottom right)

- Problem
- Some objects have stronger edges then others
- If the desired edge is weaker than a nearby edge, then the path ‘jumps’ over to the stronger edge

- Some objects have stronger edges then others
- Solution
- Train the gradient magnitude to desire the weaker edge
- Use a sample of good path to train gradient magnitude
- Update sample as path moves along the desired edge

- Allow user to enable and disable training as needed

- Train the gradient magnitude to desire the weaker edge

- Examples
- Path segment jumps without training (top)
- Path segment follows trained edge (middle)

- Cost fG
- Normal response without training (lower left)
- Trained response from edge sample (lower right)

- Need
- Source objects need blend in with a new background
- Background may need to be in front of objects

- Solution
- Allow for 2-D transformations to occur on source objects
- Use low pass filters to blend the object into the destination’s scene
- Mask background objects to appear in front of source object

- Paper
- Describes a tool
- Selects image objects quickly and easily
- Provides the means manipulate and paste them into different images

- Describes a tool
- Abstract
- Brief mention of need
- List of abilities for a tool called ‘Intelligent Scissors’

- Introduction
- Defines need
- Claims current methods are not enough
- Claims this tool will help the problem
- Gives a small background on similar segmentation tools and their flaws

- Algorithms
- The paper does a good job on explaining how dynamic programming is used
- ‘Cooling’ was explained well, but no suggested times were given
- The section on ‘Dynamic Training’ could be explained more to better understand it
- Spatial Frequency and Contrast Matching needs more explanation

- Dynamic Programming
- Used as the main driving force of this tool
- The authors spend a lot of time on the dynamic programming section but not gratuitously
- Cost must be correctly attributed to the different edge features to take advantage of dynamic programming
- Optimal path is ‘Optimal’, not just a local answer