Loading in 5 sec....

Image deblocking using local segmentationPowerPoint Presentation

Image deblocking using local segmentation

- By
**ilar** - Follow User

- 146 Views
- Updated On :

Image deblocking using local segmentation. By Mirsad Makalic Supervisor: Dr. Peter Tischer. Presentation Outline. An introduction to JPEG Lossy image compression Discrete Cosine Transform (DCT) and quantization Local segmentation and prior research Measuring image quality

Related searches for Image deblocking using local segmentation

Download Presentation
## PowerPoint Slideshow about 'Image deblocking using local segmentation' - ilar

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Presentation Outline

- An introduction to JPEG
- Lossy image compression
- Discrete Cosine Transform (DCT) and quantization

- Local segmentation and prior research
- Measuring image quality
- Deblocking Filter
- Super-Resolution Filter
- Conclusion

Lossy image compression

- JPEG is the most common lossy image compression format
- Easy to implement
- Good quality with high compression ratios

- Block based transform approach
- Divide image into 8x8 blocks
- Perform discrete cosine transform (DCT) on each 8x8 block
- Quantize the DCT coefficients

Discrete Cosine Transform

DC coefficient

- Transform 8x8 block of pixels into a set of weighted basis functions (DCT coefficients)

AC coefficients

DCT - Subbands

- An image consisting of just one coefficient from each 8x8 block is called the sub-band image
- An image which is divided into 8x8 blocks has 64 possible sub-band images

An 8x8 image split into 2x2 blocks has 4 subbands

Quantization

- Exploit visual redundancy
- Quantization is a many-to-one mapping
- Divide each DCT coefficient by a value and round to the nearest integer

- The decoder makes a guess from a range of values (pick midpoint by default)

5 6 7 8 9 10 11 12 13 14

Quantizer =10

= 1

Quantization

- Coarse quantization introduces artifacts into reconstructed image
- DCT coefficients are reconstructed inaccurately

- Most visually distracting artifact is blockiness

JPEG compressed image at quality 10 PSNR: 30.41 dB

Deblocking Techniques

- Three approaches in the literature:
- Filter the reconstructed pixel values
- Attempt to reconstruct DCT coefficients more accurately
- A hybrid approach

Local Segmentation

- Most deblocking filters introduce excessive blurring
- Destroys the structure of the image
- Edges lose their sharpness

- Local segmentation takes into account the structure of the image

Local Segmentation

- Divide a mask of pixels into N segments and filter each segment independently

Average of whole mask = 43.22, average of yellow segment = 20.8

- Two questions:
- How many segments do we use?
- How do we segment a mask of pixels?

Prior Research

- Lukasz Kizewski, BSE (hons) 2004
- DC subband approach
- Filter using a mask of DC subbands

- How do we segment a mask of pixels?
- Segment the pixel mask using thresholding

- How many segments do we use?
- Do-No-Harm heuristic

Prior Research

- Do-No-Harm heuristic
- Try a 1-segment model (average of the whole mask)
- If filtered value is implausible reject and try a 2-segment model
- A plausible value is one which falls inside the quantization range: midpoint +/- ½ Quantum

- Try a 2-segment model
- If still implausible then don’t filter

- Try a 1-segment model (average of the whole mask)

Room for improvement

- No objective measure used to test the effectiveness of the filter
- Difficult to make comparisons
- Difficult to rate changes in filter

- Works only on DC subband
- AC subbands contain edge and texture information

- A very simple local segmentation method

Measuring image quality

- Peak-signal-to-noise-ratio
- Most commonly used metric
- Does not necessarily reflect the subjective visual quality

- Generalized Block-Edge Impairment Metric (GBIM) – H.R. Wu, M. Yuen

Measuring image quality - GBIM

- Measures the quality of DCT encoded images
- Assume that what happens inside a block is the same as what happens across blocks
- Take absolute mean difference of pixels inside a block (vertical/horizontal)
- Take absolute mean difference of pixels across blocks (vertical/horizontal)
- Compare them, if the two differ greatly than it is a sign of blockiness

- Assume that what happens inside a block is the same as what happens across blocks

2x2 block example with vertical blockiness

A new deblocking filter

- Filter all coefficients
- Treat each 8x8 block as a 64 element vector where each value in the vector is one of the DCT coefficients

- Local segmentation no longer as simple (need to segment masks of vectors)
- K-Means or K-Nearest Neighbours

- DNH needs to work on vectors

A new deblocking filter

- Basic structure of filter is same:
- Create an NxN mask where each item is a 64 element DCT coefficient vector
- Segment mask using K-Means or K-Nearest Neighbours segmentation
- Check if segmentation produces valid result using DNH, if not, try different segment
- If no segmentation produces valid result, leave alone

K-Means

- Start with one segment (average of mask)
- If segmentation is invalid, increase number of segments by one until a maximum number of segments is reached
- Try largest change first

K-Nearest Neighbours

- Set the number of nearest neighbours to find as the number of items in the entire mask
- Keep decreasing by segment size by one until a valid segment is found

Vector DNH

- Center DNH
- Compare the filtered vector against only the center vector in the mask

- Strict Segment DNH
- Compare the filtered vector against all the vectors in the segment the center vector is in

- Lesser Strict Segment DNH
- Same as strict segment DNH with some error tolerance

Results

- Best found parameters for the filter:
- 3x3 vector mask (covers 24x24 pixels)
- K-Nearest Neighbour segmentation
- Lesser strict segment DNH with 1% error tolerance

- Tested:
- 5x5 mask, K-Means, Center DNH, Strict DNH etc.
- Many variations of the filter parameters

Results

(a) JPEG compressed image

(b) Filtered image

- An improvement of 0.07 dB in PSNR and a reduction of 0.62 in GBIM

Results

(a) JPEG compressed image

(b) Filtered image

- An improvement of 0.07 dB in PSNR and a reduction of 0.55 in GBIM

Other uses for local segmentation

- Super-resolution
- Combine multiple slightly different images to form one higher quality image
- Can we extract more information out of a single image?
- Neighbouring pixels are similar and share information
- Use local segmentation

Super-resolution filter

- Very similar to the deblocking filter
- Instead of using a 64 element vector for each DCT coefficient, use one element vector containing each pixel value in the mask

- How to test if it works?
- Convert 8 bit image to 4 bits and attempt to reconstruct back an 8 bit image

Results

- Best found parameters for filter:
- 3x3 mask
- K-Means segmentation with up to three segments
- Center DNH
- More than one iteration of the filter can further improve PSNR

Results

(a) An image rounded to 4 bits per pixel

(a) Filtered image (8 bits per pixel)

- An improvement of 1.38 dB in PSNR

Results

(a) An image rounded to 4 bits per pixel

(a) Filtered image (8 bits per pixel)

- An improvement of 1.32 dB in PSNR

Conclusion

- Filtering AC subbands is difficult because most have been quantized to zero or have very large quantization ranges
- Most improvements in image quality are from the stricter DNH and better local segmentation techniques
- The increase in computational complexity may not be worth the increase in image quality

Conclusion

- Super-resolution filter shows a lot of promise
- Large increase in PSNR and image quality

- A different DNH heuristic may work better with the super-resolution filter

Future Research

- Allow more variation in pixels for the strict DNH
- Assume local mask is linear and not constant

- Try different segmentation techniques
- Region growing

- Further investigate iterative filtering

The End

- Questions?

Download Presentation

Connecting to Server..