simultaneous structure and texture image inpainting n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Simultaneous Structure and Texture Image Inpainting PowerPoint Presentation
Download Presentation
Simultaneous Structure and Texture Image Inpainting

Loading in 2 Seconds...

play fullscreen
1 / 37

Simultaneous Structure and Texture Image Inpainting - PowerPoint PPT Presentation


  • 98 Views
  • Uploaded on

Simultaneous Structure and Texture Image Inpainting. by: Bertalmio, Sapiro, Vese, Osher Presented by: Shane Brennan June 7, 2007 EE 264 – Spring 2007. What Is Inpainting?. Inpainting can also be used for other purposes including:. Humor Entertainment Improving aesthetic quality of images

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Simultaneous Structure and Texture Image Inpainting' - moesha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
simultaneous structure and texture image inpainting

Simultaneous Structure and Texture Image Inpainting

by: Bertalmio, Sapiro, Vese, Osher

Presented by: Shane Brennan

June 7, 2007EE 264 – Spring 2007

inpainting can also be used for other purposes including
Inpainting can also be used for other purposes including:
  • Humor
  • Entertainment
  • Improving aesthetic quality of images
  • And for less playful purposes…
how to inpaint yes you can do it too
How To Inpaint(yes you can do it too!)
  • Identify the regions you want to fill/remove
  • Continue any lines arriving at those regions
  • Fill in the regions with texture/color from the surrounding areas
  • Enjoy!
image inpainting

Image Inpainting

by: Marcelo Bertalmio and Guillermo Sapiro

Proceedings of SIGGRAPH 2000

the very basic idea
The (very) Basic Idea
  • Form an iterative algorithm
  • Each update gets you closer to the desired result
the update image
The Update Image
  • Want to propagate “information” (aka lines) into the region being inpainted
  • Need information, and direction it is heading
  • Project information on that direction

Direction of Information

Rate of Change of Information

what is information
What Is Information?
  • Since want result to be smooth, use laplacian of the image to represent information.L = I * 2
  • The rate of change of the information entering a pixel is the derivative of the laplacian = [Li,j+1- Li,j-1, Li+1,j – Li-1,j]
propagation direction
Propagation Direction
  • Tangent to the isophate line (which is the information) arriving at the boundary
  • Tangent easily computed as:N =[-IY, Ix]note: I use the (x,y) coordinate notation
anisotropic diffusion
Anisotropic Diffusion
  • Areas where N = [0, 0] will never get updated! These are smooth regions
  • “Bleed” smooth regions into the region being inpainted
  • Also ensures lines stay smooth and curved
  • BUT! Very problematic!
  • My form of diffusion:
  • The authors form: ???
texture synthesis by non parametric sampling

Texture Synthesis by Non-Parametric Sampling

by: Alexei Efros and Thomas Leung

IEEE International Conference on Computer Vision, Corfu, Greece, September 1999

how it works
How It Works
  • Take a pixel to be synthesized. Find which pixels near it have already been synthesized (or pre-existed). Define this to be the mask
  • For every pixel in the image (the candidates), compare the WxW neighborhood to the neighborhood around the pixel to be synthesized using a distance metric, but only on pixels defined by the mask
  • Keep either the K most similar neighborhoods, or the neighborhoods whose distance is less than T (W, K, and T are user-defined values)
  • Of the remaining regions, select one at random
  • Assign the intensity of the center pixel of the selected region to the pixel being synthesized
the problem
The Problem
  • Image inpainting works well on regions with structure, but not on regions with texture
  • Texture synthesis works well on regions with texture, but not on regions with structure
  • Early papers: for each pixel decide if structure or texture, and perform the appropriate filling method
  • But there is a better way…
simultaneous structure and texture image inpainting1

Simultaneous Structure and Texture Image Inpainting

by: Bertalmio, Sapiro, Vese, Osher

IEEE Transactions On Image Processing, Vol. 12, No. 8, August 2003

how it works1
How It Works
  • Decompose image into two parts: structure image and texture image
  • Perform inpainting on the structure image
  • Perform texture synthesis on the texture image
  • Recombine the two images to get final result
  • But how to decompose? no, not like that
modeling textures with total variation minimization and oscillating patterns in image processing

Modeling Textures With Total Variation Minimization and Oscillating Patterns in Image Processing

by: Stanley Osher and Luminita Vese

Journal of Scientific Computing, Vol. 19, Nos. 1–3, December 2003

the initial version
The Initial Version
  • Considered an image to be some underlying “real image” and then noise added. Want to remove the noise (Rudin, Ohser, and Fatemi. 1992)
  • Find structure image, u, that minimizes:

Smoothness Term

Data Fidelity

incorporating the texture
Incorporating the Texture
  • No texture in there! But lets consider image to be f = u + v, where u is structure and v is texture. Note v = f – u
  • Solutions to this are known! (refer to report for some references on the solution)
cutting to the chase
Cutting to the Chase
  • Model texture as x and y components, call them g1 and g2
  • Some magic... (refer to the paper for more)

Smoothness on v (remove noise)

Smoothness on u

Reconstruction Error

an iterative solution
An Iterative Solution
  • Break the optimization down into an iterative solution. Although more iterations aren’t always better, in practice need to play with # of iterations
  • Also need to play with tuning parameters. Though there are default values which work well in a broad array of images
slide30

Original Image (f)

Structure Image (u)

Texture Image (v)

slide31

Original Image (f)

Structure Image (u)

Texture Image (v)

the algorithm
The Algorithm
  • Decompose Iin into sub-images u and v
  • Perform inpainting on u to obtain U
  • Perform texture synthesis on v to obtain V
  • Iout = U + V
  • And now for some results…
slide34

Original Image Structure Image Texture Image

Completed Image

Inpainted Structure Image

Synthesized Texture Image

slide35

Original Image Structure Image Texture Image

Completed Image

Inpainted Structure Image

Synthesized Texture Image

slide36

Original Image Structure Image Texture Image

Inpainted Structure Image

Completed Image

Synthesized Texture Image

references
References
  • M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester. Image inpainting. 2000.
  • A. A. Efros and T. K. Leung. Texture synthesis by non-parametric sampling. In ICCV (2), pages1033–1038, 1999.
  • M. Bertalmio, L. Vese, G. Sapiro, and S. Osher. Simultaneous structure and texture image inpainting, 2002.
  • L. Vese and S. Osher. Modeling textures with total variation minimization and oscillating patterns in image processing, 2002.