Hiba Tariq School of Engineering

1. An FPGA-based implementation for Median Filter meeting the real-time requirements of Automated Visual Inspection SystemsMiguel A. Vega-Rodríguez, Juan M. Sánchez-Pérez, Juan A. Gómez-Pulido Hiba Tariq School of Engineering

2. Outline • Motivation/Background • What is image processing? • Classification • Parallelism • Reconfigurable computing • Experimental results • Advantages and Disadvantages • Contributions

3. Motivation • Remove high frequency or impulse noise from images while preserving quality • Producing real-time results for time-critical applications is desirable • Improving the efficiency of the algorithms is needed • Industrial automation for product classification

4. Real-time Applications • Health monitoring applications • Medical Image Processing • Factory automation • Facial recognition • Machine Vision

5. What is Image Processing? • Image processing is any form of signal processing for which the input is an image, such as a photograph or video frame and the output could be an image or a set of characteristics related to the image • The processing is usually done by computer • The data used in image processing could be in the form of pixels, features or objects

6. Types of Image Processing Downton and Crookes classification, 1998 • Low-level Image Processing (Pixels)contrast adjustment, edge detection, filtering • Intermediate-level Image Processing (Features)segmentation, feature extraction • High-level Image Processing (Objects)object classification, object recognition

7. Image Processing Pyramid

8. Low-level Image Processing • Focus on low-level image processing, median filtering in particular • Most low-level image processing operations are computationally intensive • Data is in the form of pixels

9. Median Filter • A median filter is a spatial non-linear filter • It is known to be very robust in terms of removing high-frequency or impulse noise • It preserves the sharpness of the edges • It produces better results compared to other linear filtering techniques • The median filter operation is not reversible.

10. Median Filter Example

11. How a median filter works? • It works on a set of pixels within an area also referred to as a ‘window’ or kernel • Typically 3X3 or 5X5 kernel size is used • Pixels are first sorted within a window • Each pixel value is replaced with its median

12. Numerical Example

13. Parallelism in Image Processing • Median filtering algorithm can be made to run in parallel because of neighborhood parallelism • It is very desirable to exploit this parallelism when implementing a VLSI architecture for the filter to achieve performance gains

14. Computational Complexity • Sorting is the biggest challenge specially when the image size gets big • A general purpose processor is not enough to produce real-time results • Traditionally ASIC’s are used but they are expensive and inflexible • Reconfigurable architectures are an attractive solution

15. Proposed Architecture • Proposed a reconfigurable architecture for image processing for a visual inspection system • Compared the performance for a SW design on a GPP vs. a pure HW Implementation • Used instruction level parallelism by calculating four pixels in the same clock cycle using a 2-stage pipeline • Perform sorting in hardware using a partial sorting network

16. Sorting Network • Originally proposed by John. L. Smith of Univision Technologies Inc., Billerica, MA • The minimum exchange network required to calculate the median for nine pixels consists of 19 comparison nodes • The idea is to calculate the median for each pixel based on nine (3X3 kernel) surrounding pixels

17. Finding Median of nine pixels…

18. Example: 8,7,5,2,3,1,6,4,9

19. Proposed Architecture

20. Median filter implementation

21. Pipeline Architecture

22. Pipeline Architecture contd… • They used a Harvard architecture with 32bit buses so they were able to clock in four pixels at the same time • A two-stage pipeline was used to compute write back four pixels in one clock cycle • Pixels P1, P2 and P3 are calculated in the first stage of the pipeline and P4 in the second stage

23. Equipment Used • The software application was tested on a 350MHzIntel Pentium II Processor with 64MB RAM • The Hardware implementation consisted of: • PCI interface • HOT2-XL containing XC4062XLA FPGA • 2304 CLBs • 16 MHz • Filtered 30 images of size 640X480 pixels producing a throughput of 30 images per second

24. Experimental Results

25. Experimental Results …

26. Advantages • Parallel processing is exploited using ILP • replicating filter functional units and increasing performance by four times • Speed up of 85% for performing the operation for the hardware implementation • Achieved performance by processing 30 images per second meeting real-time requirements

27. Disadvantages • Replicated four filter circuits which increased resource use by four on the FPGA • The pipelining of the architecture is not explained so well • There is no VHDL or pseudo-code provided in the paper so it will not be easy to reproduce • They didn’t mention what sorting algorithm they used in their software application • They also didn’t include any simulations of the design

28. Critiques • Two-pass median filter, weighted median filter and adaptive median filter were around in 2002. • Pixel replacement should only be performed for high probability impulse noise data to avoid image degradation

29. Contributions • Smart VLSI architecture for median filtering to produce four medians in once clock cycle • It is possible to meet real-time median filtering requirements using their approach

30. Questions?