Real-time Face Tracking

1. Real-time Face Tracking Shane Kinsella 05491193 4th year Electronic Engineering 4BN1 NUI Galway Supervisor: Peter Corcoran March 2009

2. Concept • To investigate facial detection with reference to OpenCV and the Viola Jones technique • Explore parameters which effect the detection time • Optimise facial detection using knowledge about video streams and the movement of objects

3. Applications • Any detection system that is required to run in real-time or at very high frame rates • System does not have to detect faces, this system may be extended to work while detecting any object (given detect rules for that object) • Knowledge of the movement patterns of that object needs to be either known or learned on the fly • This system may be combined with other optimisation techniques to form a basis for real-time object tracking used in AI

4. Work Carried Out • Researched OpenCV (API) and Python (programming language) • Researched the Viola and Jones’ method of facial detection • Found example code of facial detection using OpenCV • Tested the detection time of facial detection when varying the parameters involved • Investigated movement patterns of faces on a webcam • Used this knowledge to define limits for optimised code

5. Optimisations which were applied • Scan area limited to a region around a previously detected face • Minimum face size is limited to a value which is proportional to the size of the previously detected face

6. Issues Encountered Project Specific Coding issues • Introduction of new faces • Elimination of duplicate faces • The velocity question • Setup of Python and OpenCV • Applying facial detection to sub regions of an image • Keyboard handling • Visibility of variables • Memory usage runaway

7. Reason for Discarding Velocity • A high frame rate is essential to the operation of a system which uses the previous position and the previous velocity • This is so that the system may pick up changes in velocity ex. A moving object suddenly stops • A higher frame rate means that objects will travel less from one frame to another • As the distance travelled by an object gets less and less, any advantage gained by introducing velocity also gets reduced

8. Reason for Discarding Velocity (Summary) • High frame rate is essential to proper operation • Becomes less effective (ineffective) the higher the frame rate

9. Results • Frame rates have been noted to improve by a factor of 5+ (typical 4 FPS to 20+ FPS) • Adapting the minimum face size accounts for the greatest improvement • FPS improvement without adaptable min face size approx 2 times faster

11. Features of Solution Code • Polled based keyboard input • Refreshes new faces when: • Spacebar pressed • No faces were found previously • Many display features for displaying data • Can vary the tracking parameters used • Can swap between un-optimised detection and optimised detection (key 6) • Can toggle the use of adaptable minimum face size (for comparison purposes)

12. Display Options • Key 1: toggle display faces detected (on by default) • Key 2: toggle display scan area used per face • Key 3: toggle display min face size used per face • Key 4: toggle print FPS to the console window

13. Configuration Options • Key 5: toggle un-optimised/optimised detection • Key 6: toggle use of adaptable min face size • Key w or s: select the variable you wish to vary (2 supported) • Scan area size • Minimum face size • Key a or d: vary the selected variable • Key 0: restore all of the default settings