1 / 54

More details on presentations

More details on presentations. Aim to speak for ~50 min (after 15 min review, leaving 10 min for discussions) Try to plan discussion topics It’s fine to “steal” slides from the Web, but be sure to acknowledge sources Include SIGGRAPH videos, demos (if any)

Download Presentation

More details on presentations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. More details on presentations • Aim to speak for ~50 min (after 15 min review, leaving 10 min for discussions) • Try to plan discussion topics • It’s fine to “steal” slides from the Web, but be sure to acknowledge sources • Include SIGGRAPH videos, demos (if any) • Send me your slides afterwards for including on the class webpage • Feel free to include related papers and background material in the presentation, or make a presentation based on two (or more) closely related papers • Actually, this can make your life easier! • Feel free to propose papers not on the list • Email me four presentation preferences and any scheduling constraints by the end of Sunday • Come and talk to me about any doubts or questions

  2. Texture Synthesis Most slides from A. Efros

  3. Last Time: Beauty in Complexity

  4. Today: Texture • Texture is “stuff” (as opposed to “things”) • Characterized by spatially repeating patterns • Texture lacks the full range of complexity of photographic imagery, but makes a good starting point for study of image-based techniques radishes rocks yogurt

  5. Texture Synthesis • Goal of Texture Synthesis: create new samples of a given texture • Many applications: virtual environments, hole-filling, texturing surfaces

  6. The Challenge • Need to model the whole spectrum: from repeated to stochastic texture repeated stochastic Both?

  7. Statistical modeling of texture • Assume stochastic model of texture (Markov Random Field) • Stationarity: the stochastic model is the same regardless of position stationary texture non-stationary texture

  8. Statistical modeling of texture • Assume stochastic model of texture (Markov Random Field) • Stationarity: the stochastic model is the same regardless of position • Markov property: p(pixel | rest of image) = p(pixel | neighborhood) ?

  9. Motivation from Language • Shannon (1948) proposed a way to generate English-looking text using N-grams • Assume a Markov model • Use a large text to compute probability distributions of each letter given N–1 previous letters • Starting from a seed repeatedly sample the conditional probabilities to generate new letters • One can use whole words instead of letters too

  10. Mark V. Shaney (Bell Labs) • Results (using alt.singles corpus): • “As I've commented before, really relating to someone involves standing next to impossible.” • “One morning I shot an elephant in my arms and kissed him.” • “I spent an interesting evening recently with a grain of salt.” • Notice how well local structure is preserved! • Now let’s try this in 2D...

  11. Efros & Leung Algorithm Idea initially proposed in 1981 (Garber ’81), but dismissed as too computationally expensive!

  12. Efros & Leung Algorithm non-parametric sampling p Input image • Instead, we search the input image for all sufficiently similar neighborhoods and pick one match at random • Assume Markov property, sample from P(p|N(p)) • Building explicit probability tables infeasible Synthesizing a pixel

  13. Finding matches • Sum of squared differences (SSD) || – ||2

  14. Finding matches • Sum of squared differences (SSD) • Gaussian-weighted to make sure closer neighbors are in better agreement || *( – )||2

  15. Details • Random sampling from the set of candidates vs. picking the best candidate • Initialization • Start with a few rows of white noise and grow in scanline order • Start with a “seed” in the middle and grow outward in layers • Hole filling: growing is in “onion skin” order • Within each “layer”, pixels with most neighbors are synthesized first • Normalize error by the number of known pixels • If no close match can be found, the pixel is not synthesized until the end

  16. Varying Window Size input

  17. Varying Window Size Increasing window size

  18. Synthesis Results french canvas raffia weave

  19. More Results white bread brick wall

  20. Homage to Shannon

  21. Hole Filling

  22. Extrapolation

  23. Failure Cases Growing garbage Verbatim copying

  24. Summary • The Efros & Leung algorithm • Very simple • Surprisingly good results • …but very slow

  25. Accelerating texture synthesis • For textures with large-scale structures, use a Gaussian pyramid to reduce required neighborhood size Li-Yi Wei and Marc Levoy, "Fast Texture Synthesis using Tree-structured Vector Quantization," SIGGRAPH 2000

  26. Accelerating texture synthesis • For textures with large-scale structures, use a Gaussian pyramid to reduce required neighborhood size • Low-resolution image is synthesized first • For synthesis at a given pyramid level, the neighborhood consists of already generated pixels at this level plus all neighboring pixels at the lower level Li-Yi Wei and Marc Levoy, "Fast Texture Synthesis using Tree-structured Vector Quantization," SIGGRAPH 2000

  27. Accelerating texture synthesis • Perform fast approximate nearest neighbor search using tree-structured vector quantization Li-Yi Wei and Marc Levoy, "Fast Texture Synthesis using Tree-structured Vector Quantization," SIGGRAPH 2000

  28. Tree-structured Vector Quantization Slide credit: Li-Yi Wei

  29. Accelerating texture synthesis • Perform fast approximate nearest neighbor search using tree-structured vector quantization • Use all neighborhoods of the exemplar texture to build a tree-structured codebook • To find a match for a new neighborhood, follow the tree in best-first order (at each level, choose child codeword closest to the query) • Example running times from the paper: • Exhaustive search: 360 sec • Building codebook: 22 sec, synthesis: 7.5 sec • Shortcomings? Li-Yi Wei and Marc Levoy, "Fast Texture Synthesis using Tree-structured Vector Quantization," SIGGRAPH 2000

  30. Accelerating texture synthesis • Use original position of already synthesized neighborhood pixels to create a “short list” of candidates for the current pixel Ashikhmin, M. "Synthesizing Natural Textures,"2001 ACM Symposium on Interactive 3D Graphics

  31. Accelerating texture synthesis Wei & Levoy Original sample Boundaries Ashikhmin Ashikhmin, M. "Synthesizing Natural Textures,"2001 ACM Symposium on Interactive 3D Graphics

  32. Image Quilting [Efros & Freeman] B p Synthesizing a block • Idea: unit of synthesis = block • Exactly the same as Efros & Leung but now we want P(B|N(B)) • Much faster: synthesize all pixels in a block at once • Observation: neighbor pixels are highly correlated non-parametric sampling Input image

  33. B1 B1 B2 B2 Neighboring blocks constrained by overlap Minimal error boundary cut block Input texture B1 B2 Random placement of blocks

  34. Minimal error boundary 2 _ = overlap error min. error boundary overlapping blocks vertical boundary

  35. The Philosophy • The “Corrupt Professor’s Algorithm”: • Plagiarize as much of the source image as you can • Then try to cover up the evidence • Rationale: • Texture blocks are by definition correct samples of texture so problem only connecting them together

  36. Algorithm • Pick size of block and size of overlap • Synthesize blocks in raster order • Search input texture for block that satisfies overlap constraints (above and left) • Paste new block into resulting texture • use dynamic programming to compute minimal error boundary cut

  37. Failures (Chernobyl Harvest)

  38. input image Wei & Levoy Efros & Freeman

  39. input image Wei & Levoy Efros & Freeman

  40. input image Wei & Levoy Efros & Freeman

  41. Political Texture Synthesis!

  42. Texture Transfer • Take the texture from one object and “paint” it onto another object • This requires separating texture and shape • That’s HARD, but we can cheat • Assume we can capture shape by boundary and rough shading • Then, just add another constraint when sampling: similarity to underlying image at that spot

  43. Source correspondence image Target correspondence image Source texture Target image

  44. + =

More Related