Timothy Ragland IRIS ECE 573/574 December 5, 2008

1. Cameras and Color Calibration Timothy Ragland IRIS ECE 573/574 December 5, 2008

2. Introduction • At the beginning of the semester, I was new to the school and field, and did not know which specific area of Image Processing I wanted to pursue. • I was open to trying anything while I tried to learn the field and determine my interests • Dr. Abidi suggested I look at camera calibration and color correction. • I spent the semester exploring this topic

3. Start of Semester • I redesigned and cleaned FH 317 • I spent several hours with an ECE 400 student (Hsin-Yi) helping her get set up. • I spent the first few weeks obtaining and learning about the various cameras we had and how to use them

4. Cameras • I spent several weeks working with the Sony DXC-930 3CCD camcorder. This was my primary camera early in the semester. • I spent a limited amount of time working with the 21 MegaPixel Camera, as the required exposure time for the camera was too long to be useful for my purposes (at the time, my focus was on video). • I spent some time in November working with Zhiyu’s 3CCD line scan camera

5. Color Correction • My primary objective early in the semester was performing color correction on images by using MacBeth Charts. • The ultimate goal was calibration of the Sony camcorder. • The camcorder had adjustable zoom, focus, exposures, color temperature, and blue/red gains.

6. Sony Camcorder • Sony DXC-930 • 3 CCD Camcorder • Adjustible Exposure, R/B Gains, Color Temperature • Used NTSC Output (720x480) • Connected to a Sony VCL-716BXEA Lens (112 mm focal length) • Remotely Adjustable Zoom, Focus, and Iris settings

7. Color Calibration Software • I first wrote some simple software that would detect the MacBeth charts on an image and use a simple linear transformation to correct the image (taken from [1]) • I designed two presentations to show the results of this software • Software utilizes Open CV • Detection of the squares is taken from the squares.cpp example in the Open CV demo library [1] J. Marguier, N. Bhatti, H. Baker, M. Harville and S. Süsstrunk, “Color Correction of Uncalibrated Images for the Classification of Human Skin Color”, Proc. of the 15th Color Imaging Conference, pp. 331-335, 2007.

8. Color Correction Algorithm Where Rn, Gn, Bn are the reference RGB coordinates of the nth Macbeth square Where R’n, G’n, B’n are the average RGB coordinates of the detected nth Macbeth square. Where

9. Algorithm Once A is calculated, the following is performed on each pixel:

10. Setup for test images

11. Macbeth Software Example Example Uncorrected Chart

12. Macbeth Software Example Example Detected Squares:

13. Macbeth Software Example Original Corrected

14. Macbeth Software Example Captured Images Corrected Images

15. Light Booth • Due to the poor lighting and poor location in lighting of the images I used for the color correction software, even after correction, the images still had color issues. • I aimed to properly calibrate the camera by white-balancing under D65 light and setting the ideal camera settings • I used a MacBeth lighting booth to generate D65 light, but the ambient light around the booth caused some interference • I designed a cover for the camera and booth to block out ambient light

16. Light Booth Cover Successfully blocked light, but the camera quit working before I could begin significant calibration work. While it was under repair, I designed color analysis software to ultimately assist in calibrating this camera and potentially future video sources.

17. Color Software • I spent a large portion of time designing color analysis software for a video. • The software captures live incoming video, splits the signal into the red, green, and blue channels, and displays them both in grayscale and in pseudocolor. • The software does a running average of the color in the captured video. • The software allows the user to select a certain rectangular region to specifically analyze as well.

18. Color Software Screenshots

19. Color Software Screenshots

20. Color Software Notes • This software utilizes DirectShow and GDI+. • At the beginning of writing this software, I knew nothing about either of these, so a large portion of the development time was researching these two topics • The software also utilizes a basic GUI • I also had to research enough to develop the basic GUI • The software could ultimately be extended to allow settings for live color correction

21. Line-Scan Camera • I familiarized myself with the 3CCD Line-Scan camera and the software for it • I designed a lens plate for use with a short focal-length lens (105 mm) to be used by the camera • I obtained a few images of a small printed Macbeth charts to familiarize myself with use of the camera

22. Multi-Line CCD Camera Image acquisition electronics Computer Multi-Line-scan CCD Camera Slide taken from“Automated Correction and Optimized Contrast Enhancement of Multi-line CCD Images” by Zhiyu Chen, 2008, Slide 11 PerkinElmer YD5060NRS-011 – 6144 Tri-Linear Color Camera # of pixels: 6,144 Maximum line rate: 4.88 kHz Minimum line period: 204.8 ms Pixel size: 10 mm x 10 mm Separation between color lines: 40 mm (center-to-center) Peak light responses: 630 nm (red), 540 nm (green) and 460 nm (blue). RGB sensor Imaging data path [PerkinElmer] [online] Available: http://optoelectronics.perkinelmer.com/content/Datasheets/YD5000%20Datasheet%20web.pdf Sensor spectral sensitivity curve

23. Line-Scan Example Image

24. Proposal • I wrote much of a white paper for a MURI project. • I wrote a large portion of the proposal for the MURI Project. • The proposal was about image enhancement of long-range images that have been degraded by atmospheric disturbances. • The images would be enhanced in order to attempt to obtain the highest success rate from a pattern recognition engine. • The disturbances would be modeled by acquiring images of large calibration targets because a ground truth would be known, and thus a degradation model could be derived

25. Proposal • Involved in writing the proposal was • Performing a survey of existing research in • Atmospheric Modeling • Blind deconvolution • Geometric correction • Pattern Recognition • Designing a rough schedule for the project • Writing a summary of the proposed work • Describing the holes in the existing research and how our approach improves upon this • Explaining the relevance of the framework to the sponsor

26. Summary (573) • Acquainted myself with multiple cameras and captured images with them • Designed simple color correction software • Set up a covering for the lighting booth to block out the light • Designed color analysis software that separates and averages the color channels of the selected region of the incoming video signal • Acquired some images and familiarized myself with the Line-Scan camera

27. Summary (574) • Designed an introductory presentation about my incoming experience and presented it in an IRIS meeting • Spent time researching color correction • Wrote two basic presentations on my color correction software • Spent significant time researching DirectShow and GDI+ for my color analysis software • Wrote much of a White Paper for a MURI project • Wrote much of a Proposal for the same MURI project • Designed this concluding presentation

28. Overall Summary • When I got here, I was not overly familiar with the Image Processing field, and did not know what specific area I wanted to pursue, so I offered to try any area. • Dr. Abidi suggested I explore color correction and camera calibration. • To this effect, I wrote multiple pieces of software regarding color calibration and to do color analysis. • I also wrote a white paper and a proposal for the MURI project.

29. Future Work/Direction • Now that I have a greater depth of understanding of the field, through research and discussion and IRIS presentations, I have found that my main interests lie in High-Level Image Processing • Specifically, Tracking, Artificial Intelligence, Segmentation, Pattern/Object Recognition. • Ideally, I would like to explore one or more of these areas in the future semesters, as they would be the areas I would like to pursue for a Ph.D. topic.

30. Questions ?