Image deblocking using local segmentation
Download
1 / 33

Image deblocking using local segmentation - PowerPoint PPT Presentation


  • 146 Views
  • Updated On :

Image deblocking using local segmentation. By Mirsad Makalic Supervisor: Dr. Peter Tischer. Presentation Outline. An introduction to JPEG Lossy image compression Discrete Cosine Transform (DCT) and quantization Local segmentation and prior research Measuring image quality

Related searches for Image deblocking using local segmentation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Image deblocking using local segmentation' - ilar


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Image deblocking using local segmentation l.jpg

Image deblocking using local segmentation

By Mirsad Makalic

Supervisor: Dr. Peter Tischer


Presentation outline l.jpg
Presentation Outline

  • An introduction to JPEG

    • Lossy image compression

    • Discrete Cosine Transform (DCT) and quantization

  • Local segmentation and prior research

  • Measuring image quality

  • Deblocking Filter

  • Super-Resolution Filter

  • Conclusion


Lossy image compression l.jpg
Lossy image compression

  • JPEG is the most common lossy image compression format

    • Easy to implement

    • Good quality with high compression ratios

  • Block based transform approach

    • Divide image into 8x8 blocks

    • Perform discrete cosine transform (DCT) on each 8x8 block

    • Quantize the DCT coefficients


Discrete cosine transform l.jpg
Discrete Cosine Transform

DC coefficient

  • Transform 8x8 block of pixels into a set of weighted basis functions (DCT coefficients)

AC coefficients


Dct subbands l.jpg
DCT - Subbands

  • An image consisting of just one coefficient from each 8x8 block is called the sub-band image

  • An image which is divided into 8x8 blocks has 64 possible sub-band images

An 8x8 image split into 2x2 blocks has 4 subbands


Quantization l.jpg
Quantization

  • Exploit visual redundancy

  • Quantization is a many-to-one mapping

    • Divide each DCT coefficient by a value and round to the nearest integer

  • The decoder makes a guess from a range of values (pick midpoint by default)

5 6 7 8 9 10 11 12 13 14

Quantizer =10

= 1


Quantization7 l.jpg
Quantization

  • Coarse quantization introduces artifacts into reconstructed image

    • DCT coefficients are reconstructed inaccurately

  • Most visually distracting artifact is blockiness

JPEG compressed image at quality 10 PSNR: 30.41 dB


Deblocking techniques l.jpg
Deblocking Techniques

  • Three approaches in the literature:

    • Filter the reconstructed pixel values

    • Attempt to reconstruct DCT coefficients more accurately

    • A hybrid approach


Local segmentation l.jpg
Local Segmentation

  • Most deblocking filters introduce excessive blurring

    • Destroys the structure of the image

    • Edges lose their sharpness

  • Local segmentation takes into account the structure of the image


Local segmentation10 l.jpg
Local Segmentation

  • Divide a mask of pixels into N segments and filter each segment independently

Average of whole mask = 43.22, average of yellow segment = 20.8

  • Two questions:

    • How many segments do we use?

    • How do we segment a mask of pixels?


Prior research l.jpg
Prior Research

  • Lukasz Kizewski, BSE (hons) 2004

  • DC subband approach

    • Filter using a mask of DC subbands

  • How do we segment a mask of pixels?

    • Segment the pixel mask using thresholding

  • How many segments do we use?

    • Do-No-Harm heuristic


Prior research12 l.jpg
Prior Research

  • Do-No-Harm heuristic

    • Try a 1-segment model (average of the whole mask)

      • If filtered value is implausible reject and try a 2-segment model

      • A plausible value is one which falls inside the quantization range: midpoint +/- ½ Quantum

    • Try a 2-segment model

      • If still implausible then don’t filter


Prior research13 l.jpg
Prior Research

  • Results

(a) Unfiltered image

(a) Filtered image


Room for improvement l.jpg
Room for improvement

  • No objective measure used to test the effectiveness of the filter

    • Difficult to make comparisons

    • Difficult to rate changes in filter

  • Works only on DC subband

    • AC subbands contain edge and texture information

  • A very simple local segmentation method


Measuring image quality l.jpg
Measuring image quality

  • Peak-signal-to-noise-ratio

    • Most commonly used metric

    • Does not necessarily reflect the subjective visual quality

  • Generalized Block-Edge Impairment Metric (GBIM) – H.R. Wu, M. Yuen


Measuring image quality gbim l.jpg
Measuring image quality - GBIM

  • Measures the quality of DCT encoded images

    • Assume that what happens inside a block is the same as what happens across blocks

      • Take absolute mean difference of pixels inside a block (vertical/horizontal)

      • Take absolute mean difference of pixels across blocks (vertical/horizontal)

      • Compare them, if the two differ greatly than it is a sign of blockiness

2x2 block example with vertical blockiness


A new deblocking filter l.jpg
A new deblocking filter

  • Filter all coefficients

    • Treat each 8x8 block as a 64 element vector where each value in the vector is one of the DCT coefficients

  • Local segmentation no longer as simple (need to segment masks of vectors)

    • K-Means or K-Nearest Neighbours

  • DNH needs to work on vectors


A new deblocking filter18 l.jpg
A new deblocking filter

  • Basic structure of filter is same:

    • Create an NxN mask where each item is a 64 element DCT coefficient vector

    • Segment mask using K-Means or K-Nearest Neighbours segmentation

    • Check if segmentation produces valid result using DNH, if not, try different segment

    • If no segmentation produces valid result, leave alone


K means l.jpg
K-Means

  • Start with one segment (average of mask)

  • If segmentation is invalid, increase number of segments by one until a maximum number of segments is reached

  • Try largest change first


K nearest neighbours l.jpg
K-Nearest Neighbours

  • Set the number of nearest neighbours to find as the number of items in the entire mask

  • Keep decreasing by segment size by one until a valid segment is found


Vector dnh l.jpg
Vector DNH

  • Center DNH

    • Compare the filtered vector against only the center vector in the mask

  • Strict Segment DNH

    • Compare the filtered vector against all the vectors in the segment the center vector is in

  • Lesser Strict Segment DNH

    • Same as strict segment DNH with some error tolerance


Results l.jpg
Results

  • Best found parameters for the filter:

    • 3x3 vector mask (covers 24x24 pixels)

    • K-Nearest Neighbour segmentation

    • Lesser strict segment DNH with 1% error tolerance

  • Tested:

    • 5x5 mask, K-Means, Center DNH, Strict DNH etc.

    • Many variations of the filter parameters


Results23 l.jpg
Results

(a) JPEG compressed image

(b) Filtered image

  • An improvement of 0.07 dB in PSNR and a reduction of 0.62 in GBIM


Results24 l.jpg
Results

(a) JPEG compressed image

(b) Filtered image

  • An improvement of 0.07 dB in PSNR and a reduction of 0.55 in GBIM


Other uses for local segmentation l.jpg
Other uses for local segmentation

  • Super-resolution

    • Combine multiple slightly different images to form one higher quality image

    • Can we extract more information out of a single image?

      • Neighbouring pixels are similar and share information

      • Use local segmentation


Super resolution filter l.jpg
Super-resolution filter

  • Very similar to the deblocking filter

    • Instead of using a 64 element vector for each DCT coefficient, use one element vector containing each pixel value in the mask

  • How to test if it works?

    • Convert 8 bit image to 4 bits and attempt to reconstruct back an 8 bit image


Results27 l.jpg
Results

  • Best found parameters for filter:

    • 3x3 mask

    • K-Means segmentation with up to three segments

    • Center DNH

    • More than one iteration of the filter can further improve PSNR


Results28 l.jpg
Results

(a) An image rounded to 4 bits per pixel

(a) Filtered image (8 bits per pixel)

  • An improvement of 1.38 dB in PSNR


Results29 l.jpg
Results

(a) An image rounded to 4 bits per pixel

(a) Filtered image (8 bits per pixel)

  • An improvement of 1.32 dB in PSNR


Conclusion l.jpg
Conclusion

  • Filtering AC subbands is difficult because most have been quantized to zero or have very large quantization ranges

  • Most improvements in image quality are from the stricter DNH and better local segmentation techniques

  • The increase in computational complexity may not be worth the increase in image quality


Conclusion31 l.jpg
Conclusion

  • Super-resolution filter shows a lot of promise

    • Large increase in PSNR and image quality

  • A different DNH heuristic may work better with the super-resolution filter


Future research l.jpg
Future Research

  • Allow more variation in pixels for the strict DNH

    • Assume local mask is linear and not constant

  • Try different segmentation techniques

    • Region growing

  • Further investigate iterative filtering


The end l.jpg
The End

  • Questions?


ad