1 / 21

KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY

IEEE International Conference on Image Processing 2013. KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY. Ruogu Fang 1 , Andrew C. Gallagher 1 Tsuhan Chen 1 , Alexander Loui 2 1 Cornell University 2 Eastman Kodak Company.

alesia
Download Presentation

KINSHIP CLASSIFICATION B Y MODELING FACIAL FEATURE HEREDITY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IEEE International Conference on Image Processing 2013 KINSHIP CLASSIFICATION BYMODELING FACIAL FEATURE HEREDITY Ruogu Fang1, Andrew C. Gallagher1 Tsuhan Chen1, Alexander Loui2 1 Cornell University 2 Eastman Kodak Company

  2. KINSHIP CLASSIFICATION BYMODELING FACIAL FEATURE HEREDITY Problem Definition: Recognizethe family that a query person belongs to from a set of families. Solution: Reconstruct the query face from a mixture of parts from a set of family members for the recognition. Motivation: Geneticmodelofreproductionusingthemathematicaltoolofsparsity.

  3. A genetic perspective • Why do we look like the way that we do? • DNA • How are our appearances affected by ancestors? • Inheritance and mutation • Facial features are part of the appearance. DNA Thefacialfeatureheredityalsofollowthemodelofgenetics.

  4. Mendel’s LawsI • Law of Random Segregation: For every particular trait, one randomly selected allele from each parent is passed down to the offspring. B: Brown eyes (dominant) b: blue eyes (recessive) Each facialfeature ofanindividualcan be represented by a sparse combination of the relatives with this feature. b b B b B b B b Bb Bb BB Bb Bb bb bb bb

  5. family sparsity Few families are selected. ?

  6. Energy function • For one family, given sufficient training samples ofafamily (m = feature length, n = number of training samples) • A new sample from the same family • Approximately lies in the linear span of the family member samples associated with this family • For all unrelated families, L2,1 norm: Family sparsity term N= # families L1 norm: Individual sparsity term (illumination, pose and expression)

  7. Mendel’s LawsII • Law of Independent Assortment: Genes of separate traits are passed down independently from parents to offspring. Thefacialfeaturesshouldbeanalyzedindependently. Credit:NortheastMedicalSchool

  8. Independence of facial parts For each part, a part-based dictionary is built.

  9. Reconstruction error for part p from family j classification Error Remove outliers due to recessive genes … • Choose three representative parts with smallest possible residues R. • Rank the normalized residues for all families on these three parts. • Sum the ranks and use the highest rank. # families Error … Byproduct: Find the three most distinguishable features # families … Error … # families

  10. Potential applications Family Image Retrieval Family Photo Album Distillation Tag Your Family Members From Sara Lee’s family? From Kelly Ng’s family? Social Websites: Auto Family Tagging Find Lost Relatives

  11. Family 101 database • 607 Individuals • 14,816 Images Download: http://tinyurl.com/kinshipclassification • 101 Different Families Kennedy 27 (410) # people # images

  12. family database Collection Kennedy Family 27 Individuals 48 Images of Caroline Kennedy

  13. Related databases • Facts about Family101 Database • Multiple generations • Every nuclear family has 6 family members on average • Every individual has 24 images on average

  14. Experiment setup • Feature: Dense SIFT 16x16 • Baseline • K nearest neighbors (KNN) • Support vector machine (SVM) • Sparse representation based recognition (SRC) • Unless specified in each scenario: • 3 family members for training, 2 for testing. • 20 families randomly selected for evaluation. • 30 images/person for both training and testing. • Evaluation metric: Mean per-family accuracy

  15. Exp 1: No. of families 3 family members for training 2 family members for testing 30 images/person for training/testing

  16. Exp 2: No. of people for training 20 families randomly selected 30 images/person for training/testing

  17. Facial Feature Matching • Task: Find the people with similar facial features to the query person. Martin Sheen High coefficients Martin Sheen Hair Eyes Nose Test Images Mouth Training Images Low

  18. conclusion • Motivation: biological process of inheritance • Mendel’s laws of random segregation and independent assortment • A new challenge: kinship classification • A new framework: reconstruct the query face from a mixture of parts from a set of families • A new dataset: Family 101

  19. Future work • Use family tree structure • Hallucination • Hallucinate what the appearance of the father might be, just by looking at the differences between a child and her mother.

  20. KINSHIP CLASSIFICATION BYMODELING FACIAL FEATURE HEREDITY Ruogu Fang Andrew C. Gallagher Q & A Thank you! ProjectPageandDatasetDownload: http://tinyurl.com/kinshipclassification Tsuhan Chen Alexander Loui

  21. Face detection & alignment Face Alignment: 6 Fiducial Points Active Shape Model: 82 Facial Points Face Detection

More Related