Implementing Codesign in Xilinx Virtex II Pro

Implementing Codesign in Xilinx Virtex II Pro Betim Çiço, Hergys Rexha Department of Informatics Engineering Faculty of Information Technologies Polytechnic University of Tirana

Outline • Introduction • The main algorithm proposed • Spotlights • Static lights • Light pairs • System implementation • Results • Conclusions

Introduction • Future of automotive systems? => video based driver assistance. • Complex algorithm are required. (and they need to be executed faster and faster) • One solution would be the use of dedicated HW. • Drawback, is the lack of the flexibility. • Nowadays systems for driver assistance offer features such, lane departure or cruise control • cameras and radar sensors

Introduction • But we want more... • We want help in more complex situations, like urban traffic. • The proposed vision-based concept • Separation, repetitive operations and high level application code. • Repetitive operations are accelerated by coprocessors. • High level application code is implemented fully programmable on standard CPU cores. • Achivments • Fast execution • Flexibility

The main algorithm • An algorithm to detect cars by taillights. • Tunnel or night time driving on roads with separated lanes for each direction. • Properties of the algorithm: • searches for light pairs that stem from cars. • compare the properties of car lights with the lights of the tunnel. • searches for the illumination of a license plate. • We demonstrate that the best way for this algorithm to be implemented is the division into a hardware and a software part.

Cont` • The image is recorded from a video camera (25 grayscale frames per second). • The image then is processed by the subsequent engines and software parts. • At the end we have as a result, car identification by taillights.

Basic Schematic

Schematic Summary • Hardware implemented parts: • PlateSearchEngine (license of the car) • Spotlight-Engine • LabelingEngine • Software implemented parts • Programmable software that performs several actions like shown in the scheme.

Spotlight Engine • From Camera the frame goes: • To the Spotlight Engine • To the Platesearch Engine • Depending on a threshold, the spotlight engine will give a binary image.

Spotlights • Bright regions in the image that are ideally round and surrounded by dark pixels • Taillights of cars typically appear as spotlights • Proposed algorithm applies a simple shape filter to the image • We define the two pixel sets relative to the current pixel (CP). If all pixels in PF are darker than all pixels in PS, the CP is a spotlight pixel lum(PS) > lum(PF ) + threshold

Labeling Engine • Takes as input, the binary image given from Spotlight Engine • Searches the binary image for regions of connected white pixels • Creates a label for each region • Gives as an output a list of labels • Each label corresponds to one spotlight

Static lights • We have a list of spotlights that have been extracted • We have these properties for each spot: • Bounding rectangle • Position in the image (coordinates of the center of the bounding rectangle) • Total brightness • Number of pixels • Static lights, i.e. they do not move relative to the road • Direction of the light’s motion vector can be used to determine static lights. • For each light in the current frame, a close by light is searched in the previous frame

Finding light pairs • We have to consider • Distance of lights, y-component • Both taillights of a car are expected to be on the same height on a flat road • Distance of lights, x-component • A vehicle that is far away from the camera appears more in the top of the image than a closer vehicle • Existence of additional light in between the pair • In that case, the two outer lights of the four spotlights could be considered as one candidate pair • At the end, including the continuity vector, we come up with the best candidates for light pairs.

System Implementation • The system was implemented on a Xilinx Virtex-II Pro FPGA • That FPGA also features two embedded 300 MHz PowerPC CPU cores, one of which was used to run an embedded Linux operating system

Results • For the Spotlight Engine we have 7 times increase in speed, compared to Pentium 4 processor. • 173 time increase in speed compared to a Power PC processor.

Conclusions • As we have seen through this work in today’s video assistance systems for drivers need to be real time and flexible . • We used here a new type of design where hardware and software cooperate to give both real time performance and flexibility. • What is more important: • “By implementing hardware coprocessors we let repetitive tasks to be handled by hardware engines while the software part takes care of the interface between parts of the system and some computations on a small amount of data”. • With Codesign we release the processor load during the computations.

Thank you... bcico@icc-al.org, hrexha@gmail.com

Implementing Codesign in Xilinx Virtex II Pro