My phd project
Download
1 / 22

my phd project - PowerPoint PPT Presentation


  • 354 Views
  • Updated On :

My PhD Project. Quantifying Visual Cues of Psychological States in Interview Situations (QVCPS). 18/05/2005 Enrica Dente Email: [email protected] Website: http://www.enricadente.com/imperial/. Contents. Problem Hypothesis Project Objectives Project Plan Achievements to Date

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'my phd project' - betty_james


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
My phd project l.jpg

MyPhD Project

Quantifying Visual Cues of Psychological States in Interview Situations (QVCPS)

18/05/2005

Enrica Dente

Email: [email protected]

Website: http://www.enricadente.com/imperial/


Contents l.jpg
Contents

  • Problem

  • Hypothesis

  • Project Objectives

  • Project Plan

  • Achievements to Date

  • Conclusion

Enrica Dente, QVCPS Project


Problem l.jpg
Problem

  • Most lie detection approaches can only detect “psychological states”, not “deception” (i.e. are less than 80% accuracy) [Vrij, 2004]

  • Manual observation of human behavior subjective and limited [Burgoon, 2005]

  • Polygraph, 100 years old lie detector problematic but still in use

  • Increased interest in alternative approaches:

    • Automated Face Analysis (i.e. changes in facial expressions)

    • Thermal imaging (i.e. changes in blood flow around eyes)

    • FMRI (i.e. BOLD)

    • Brain fingerprinting (i.e. brain waves)

    • Eye tracking (i.e. irregular eye movements in front of images )

  • Intrusiveness yet to be addressed.

Enrica Dente, QVCPS Project


Hypothesis l.jpg
Hypothesis

  • Non skilled liars more likely to experience cognitive load and controlled behavior[Vrij, 2004]

  • Decrease of hand/finger movements when subjects lie yet to be proved (or disproved)

  • Quantification of observations required

  • Focus on tracking of hand/finger movements in interview situations

  • Real life cases can make a difference in results but problematic [Vrij 2005]

  • Verbal behavior required to add context to the tracking

Enrica Dente, QVCPS Project


Project objectives l.jpg
Project Objectives

  • To automate the tracking of finger/hand movements in interview situations

  • Tocompute the frequency of hand and finger movements normalised to the baseline of each individual

  • To build a biomechanical model to aid hand gesture tracking

  • To provide a simple means of accessing and referring to the verbal behaviour, when required

  • Later, to compare the movements of the interviewee with the movements of the interviewer.

Enrica Dente, QVCPS Project


Project plan l.jpg
Project Plan

  • Year 1:

    • Skin Colour Modelling

    • Hand Tracking and Finger Position

    • Coding behaviour markup

    • Background Modelling

    • Graphical User Interface

    • Experimental Protocol

  • Year 2 and Year 3:

    • Kinematic models of upper body, limb and finger motion

    • Introduction of verbal behavior to add context to tracking

    • Train system to quantify different types of hand and finger movements

    • Writing up.

Enrica Dente, QVCPS Project


Achievements to date 1 segmentation l.jpg
Achievements to Date: 1) Segmentation

(a) Skin Colour Modelling

  • Color Predicate[Kieldsen, 1996]

  • Bayesian Posterior Map and Parzen colour space probability density estimates more accurate.

    (b) Hand Tracking and Finger Position

  • Connected Component and central moments

  • Centroid correspondence

  • Hands separation based on area and distance heuristics

  • Complex wavelet decomposition to detect changes in finger position.

  • Next Step: background modelling using parametric methods to address robustness.

  • Figure 1 -Hand Tracking

    Figure 2 -Finger Orientations

    Enrica Dente, QVCPS Project


    A skin colour modelling l.jpg
    (a) Skin Colour Modelling

    • Joint Conditional Density function in hue and sat for hand and non-hands (i.e. Parzen density estimation)

      where H and S represent hue and sat, Hn and Sn represent the observed values amongst the NH hands training set pixels, KH is a normalised constant and σ0 species the width of the kernel used in smoothing

    • Bayesian Posterior Map to identify most probable locations of skin in each frame

      where we assume that p(H,S | Hands) is approximated by the left hand side of Equation (1) and p(H,S | NonHands) is approximated by the left hand side of Equation (2).

    (1)

    (2)

    Enrica Dente, QVCPS Project


    A skin colour modelling cont l.jpg

    Figure 3 - Likelihood function for hand regions in a), for non-hand regions in b)

    (a) Skin Colour Modelling (cont.)

    • Parzen colour space probability density estimates for hand and for non-hand regions

    • There is overlap between the two classes

    Enrica Dente, QVCPS Project


    B hand tracking and finger position l.jpg
    (b) Hand Tracking and Finger Position non-hand regions in b)

    HAND TRACKING:

    • Centroid correspondence of each object across frames from binary image created in (a)

    • Connected component analysis to label blobs.

    • Switch from a one-hand to two-hand state by using distance and area heuristics.

      FINGER POSITION:

    • Complex wavelet decomposition

    • Orientation of local image structure relatively invariant to local phase

    • Filter bank of complex wavelets, each tuned to one of four directions

    • We compare output of four complex masks to make estimate of the presence of a line or an edge in each direction

    • Vector indicating the direction of local image structure estimated by a weighted vector summation operation

    • Multirate scheme (i.e. series of different scales).

    Enrica Dente, QVCPS Project


    Finger position estimate l.jpg
    Finger Position Estimate non-hand regions in b)

    • K/2 of the real and imaginary impulse responses of wavelets at scale 1 for K = 8

    • Orientation field O(l) (m, n) is defined by:

    Figure 4 -WaveletKernels

    Enrica Dente, QVCPS Project


    Finger position estimate cont l.jpg

    Figure 5 (a) Hands together (b) Hands moving non-hand regions in b)

    Finger Position Estimate (.cont)

    • Histogram of orientations for each hand in each frame

    • Weighting of the product of the orientation

    • Field magnitude and posterior skin tone map to appropriate histogram bin

    • Orientation density functions can be used to detect finger movement!

    Enrica Dente, QVCPS Project


    Achievements to date 2 xml markup for behavior coding l.jpg
    Achievements to Date: non-hand regions in b)2)XML Markup for Behavior Coding

    Check Verbal behavior:

    Check Non Verbal behavior:

    Enrica Dente, QVCPS Project


    Achievements to date 3 graphical user interface gui l.jpg
    Achievements to Date: non-hand regions in b)3) Graphical User Interface (GUI)

    Enrica Dente, QVCPS Project


    Achievements to date 4 experimental protocol l.jpg
    Achievements to Date: non-hand regions in b)4) Experimental Protocol

    • Stakes low in current lie detection experiments:

      • Interviewees are asked to lie on simulated actions

      • Interviewees less motivated to lie than in real-life situations, performance less effective

      • Vrij’s real-life police interview limited to visual observation of one liar

      • Problem: questions based on prior knowledge of what the interviewees will lie on and when

    • Our experimental protocol aims to raise the stakes of deception detection in interview situations:

      • Solution: design experiments where interviewees are not told in advance what they have to lie on.

    Enrica Dente, QVCPS Project


    Literature review 1 skin colour modelling l.jpg
    Literature Review non-hand regions in b)1) Skin Colour Modelling

    • Non Parametric Methods

      • Create an histogram or lookup table (LUT) common and simple

      • However, storage required, bins may miss pixel values, noise removal and occlusion still an issue

      • Bayesian posterior map computed from “hands” and non “hands” histograms more promising than Colour Predicate

    • Parametric Methods

      • Model skin colour as components of mixtures of Gaussians

      • Provide robustness, no need for storage space and ability to interpolate or generalise the training data

      • However, they require accurate initialization and assume number of components to be known in advance

    Enrica Dente, QVCPS Project


    Literature review 1 skin colour modelling17 l.jpg
    Literature Review non-hand regions in b)1) Skin Colour Modelling

    • Parametric Methods (cont.)

      • Foreground and background modelled by a joint probability density function

      • Model parameters (i.e. means and covariance) for the Gaussians estimated from the training data using maximum likelihood or Bayesian inference

      • Updated online based on colour, on position and/or on motion information

      • Skin probability computed from Gaussian Probability Density Functions (PDF's)

      • However, in interview situations update cannot be based on assumption that foreground is moving.

    Enrica Dente, QVCPS Project


    Literature review 2 kinematic 3 d models l.jpg
    Literature Review non-hand regions in b)2) Kinematic 3-D models

    • Need accurate tracking of hand/finger gestures

    • Choice of degrees of freedom crucial

    • Choice of type of model (i.e. stick figure or statistical)

    • Need markers for validation

    • Need simple and consistent movements

    • Does deception occur fast or slow in interviews?

    • How can we track several movements at the same time? Number of cameras?

    • How visible can hand's edges be in interviews? Camera resolution?

    Enrica Dente, QVCPS Project


    Conclusion l.jpg
    Conclusion non-hand regions in b)

    • Problem and Hypothesis:

      • Most lie detection approaches detect “psychological states”, not “deception”

      • Non skilled liars more likely to experience cognitive load and controlled behavior

      • Quantification of observations required

    • Achievements to Date:

      • Hand Tracking using Bayesian Posterior Map

      • Finger Position using Complex Wavelets

      • XML markup for correlating visual and verbal behavior

      • Experimental Protocol

    • Next Step:

      • Background modelling to address robustness.

    Enrica Dente, QVCPS Project


    Questions l.jpg
    Questions? non-hand regions in b)

    Enrica Dente, QVCPS Project


    References l.jpg
    References non-hand regions in b)

    • [Vrij, 2004] Vrij Aldert, Psychology, Crime and Law, Challenging Interviewees During Interviews, The Potential Effects on Lie Detection, 2004

    • [Vrij2, 2004] Vrij Aldert, Why Professionals fail to catch liars and how they can improve, Legal and Criminological Psychology, 9, 159-181, 2004

    • [Vrij, 2002] Aldert Vrj, Detecting Lies and Deceit : The Psychology of Lying and the Implications for Professional Practice, John Wiley & Sons, Inc

    • [Watson, 1081] Watson, K. W. Oral and written linguistic indices of deception during employment interviews. (Doctoral dissertation, Louisiana State University, 1981). Dissertations Abstracts International, 42, 06A 2367.

    • [Parker et al., 2000] Parker, A. & Brown, J., Detection of deception: Statement Validity Analysis as a means of determining truthfulness or falsity of rape allegations. Legal and Criminological Psychology, 5, 237-259, 2000

    Enrica Dente, QVCPS Project


    References cont l.jpg
    References (cont.) non-hand regions in b)

    • [Bradley, 1993] Bradley MT, Cullen MC, Polygraph lie detection on real events in a laboratory setting, 76(3 Pt 1):1051-8, Percept Mot Skills. June 1993

    • [Etcoff, 2000] Nancy L. Etcoff*, Paul Ekman, John J. Magee, Mark G. Frank, NATURE , VOL 405, 11 MAY 2000, Lie detection and language comprehension, 2000 Macmillan Magazines Ltd, Available Online: http://www.nature.com/cgi-taf/DynaPage.taf?file=/nature/journal/v405/n6783/full/405139a0_fs.html&content_filetype=pdf

    • [Kieldsen, 1996] R. Kjeldsen and J. Kender. Finding skin in color images. In Second International Conference on Automatic Face and Gesture Recognition, 1996.

    • [Burgoon, 2005] J. Burgoon et al. An approach for intent identi-

    • cation by building on deception detection. In Proc. of the 38th Annual Hawaii Int. Conf. on Detection of Deception: Collaboration Systems and Technology, 2005.

    • [Bharath, 2003] A. Bharath and J. Ng. A Steerable Complex Wavelet Construction and Its Application to Image Denoising. IEEE Transactions on Image Processing. June 2003

    Enrica Dente, QVCPS Project


    ad