Applications of one class classification
Download
1 / 25

Applications of one-class classification - PowerPoint PPT Presentation


  • 72 Views
  • Uploaded on

Applications of one-class classification. -- searching for comparable applications for negative selection algorithms. background. Purpose: looking for real world applications that demonstrates the usage of V-detector (a negative selection algorithm) One-class classification problem:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Applications of one-class classification' - carmelita-aguilar


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Applications of one class classification

Applications of one-class classification

-- searching for comparable applications for negative selection algorithms


Background
background

  • Purpose: looking for real world applications that demonstrates the usage of V-detector (a negative selection algorithm)

  • One-class classification problem:

    • Different from conventional classification: only information of one of the classes (target class) is available

    • Original application: anomaly (outliner) detection


One class classification
One-class Classification

  • Basic concept of classification:

    • A classifier is a function which outputs a class label from each input object. It cannot be constructed from known rules.

    • In pattern recognition or machine learning: inferring a classifier (a function) from a set of training examples.

    • Usually, the type of function is chosen beforehand and parameters are to be determined.

      • Line classifier, mixture of Gaussians, neural networks, support vector classifiers


One class classification1
One-class Classification

  • Basic concept of classification:

    • Assumptions: continuity, enough information (amount of samples, limited noise), etc.

    • Multi-class classification can be decomposed into two-class classifications


One class classification2
One-class classification

  • Same problems as conventional classification

    • Definition of errors

    • Atypical training data

    • Measuring the complexity of a solution

    • The curse of dimensionality

    • The generalization of the method


Applications of one class classification

A conventional and a one-class classifier applied to an example dataset containing apples and pears, represented by 2 features per object. The solid line is the conventional classifier which distinguishes between the apples and pears, while the dashed line describes the dataset. This description can identify the outlier apple in the lower right corner, while the classifier will just classify it as an pear.


One class classification3
One-class classification example dataset containing apples and pears, represented by 2 features per object. The solid line is the conventional classifier which distinguishes between the apples and pears, while the dashed line describes the dataset. This description can identify the outlier apple in the lower right corner, while the classifier will just classify it as an pear.

  • Additional problems

    • Most conventional classifier’s assumption that more or less balanced data.

    • Hard to decide on the basis of on class how tightly the boundary should fit in each direction around the data.

    • Hard to find which features should be used to find the best separation.

    • Impossible to estimate false positives.

    • More prominent curse of dimension.

    • Extra constraints: closed boundary etc.


Various techniques
Various techniques example dataset containing apples and pears, represented by 2 features per object. The solid line is the conventional classifier which distinguishes between the apples and pears, while the dashed line describes the dataset. This description can identify the outlier apple in the lower right corner, while the classifier will just classify it as an pear.

  • Generate outliner detection

    • Some methods requires near-target objects;

  • Density method: directly estimating the density of target objects

    • some works requires density estimate in the complete feature space

    • Typical sample is assumed

  • Reconstruction methods: based on prior knowledge

  • Boundary methods

    • Well defined distance


Application 1 texture classification
Application 1: texture classification example dataset containing apples and pears, represented by 2 features per object. The solid line is the conventional classifier which distinguishes between the apples and pears, while the dashed line describes the dataset. This description can identify the outlier apple in the lower right corner, while the classifier will just classify it as an pear.

  • Problem: classification of texture images

    • polished granite (or ceramic) tiles that are widely used as construction elements

    • The polished granite tiles are usually inspected by a human expert using a chosen master tile as the reference.

    • Such inspection is subjective and qualitative

    • One-class classifier is suitable: Outliners cannot be used to train any methods

  • Recent development

    • quasi-statistical representation of binary images used as a feature space for texture image classification


Applications of one class classification

  • Based on CCR feature space (coordinated cluster representation)

  • Outline of the method:

    • Given a master texture image of a class, estimate statistics of CCR histogram

    • Use parameters of the statistics to define a closed decision boundary.


Applications of one class classification

Master images representation)


Ccr feature space
CCR feature space representation)

  • A binary image intensity: Sa={sa{l,m}}, where l=1, 2, …L and m=1, 2, …, M

  • A rectangular window W = I X J

  • Scan all over the image with one pixel steps using that window

  • The number of all possible state of the window is 2w

  • Coordinated clusters representation consists of a histogram Ha(I,J)(b)

    • a is the index of the image

    • (I,J) indicated the size of the window

    • b = 1, 2, …, 2w


Applications of one class classification

  • When a histogram is normalized, it is considered as a proability distribution function of occurrence

    • Fa(I,J)(b) = 1/A Ha(I,J)(b)

      • Where A = (L-I+1)X(M-J+1)

  • Histogram H contains all the information about n-point correlation moments of the image if and only if the separation vectors between n pixels fit between the scanning window

  • In general, when the order of statistics is higher, more structural information is available

  • There is a structural correspondence between a gray level image and its thresholded counterpart

  • Provided that the binary image keeps enough structural information about a primary gray level image to be classified, the CCR of a binary image is highly suitable for recognition and classification of gray level texture image


Applications of one class classification

  • Framework of classification proability distribution function of occurrence

    • Training phase

      • a set of gray level image from each texture class

      • Each threshold

      • Calculate CCR distribution function

    • Recognition phase

      • Input test image

      • Thresholded

      • CCR distribution

      • Compare with prototypes and assign to the class of best match

  • One-class classification

    • Define the limits of feature variations

    • Establish the criterion


Applications of one class classification

  • Thresholding (binarization) proability distribution function of occurrence

    • Because CCR is defined for binary image

    • Fuzzy C-Means clustering method


Applications of one class classification

  • Training phase proability distribution function of occurrence

    • assuming Q images of a class are available, a random set of P subimages is sampled

      • If only one image is available, Q independent random sets are sampled

    • Five measurements are calculated from distribution function Fa

      • F: mass center of subimages (not a value, still a function or histogram)

      • D: mean of distance (“distance” refers to the mean distance within a set)

      • s: mean of standard deviation (“standard deviation” refers to the standard deviation of a set)

      • D: mean of q-th sample center to the center of all samples

      • s2: variance of each sets with regard to center of samples


Applications of one class classification

  • Criterion proability distribution function of occurrence

    • d(Ftest, F) < D+Cs

    • D-2s<Dtest<D+2s

  • C is the emprical adjustment parameter

  • Ftest and Dtest are the mean of K ransom subimage of the texture image to be classified

  • L1 distance is used as the measures of distinction

    • d(Fa,Fb) = Sb|Fa(b)-Fb(b)|


Applications of one class classification

  • Results proability distribution function of occurrence

    • C should be in the range of 1, 2, …, 20

      • Based on observation that s is approximately ten times less than D

    • 8 master images (training data) plus 16 testing images are used (128X128).

    • For C=1 or 2, only the master images are recognized

    • More images are recognized for larger C

    • For C<19, no mis-classification

    • Proper C depends on the size of subimage (32, 24, 64 are discussed)


Application 2 authorship
Application 2: authorship proability distribution function of occurrence

  • Problem: authorship verification

    • Different from standard text categorization problem

    • No realistic to train with negative samples

  • Difference from other one-class classification

    • Negative samples are not lacked – hard to choose to represent the entire class

    • The object texts are long

      • We can chunk to multiple samples – a set instead of single instance


Applications of one class classification

  • New idea: proability distribution function of occurrence

    • Depth of difference between two sets

    • Test the rate of degradation of accuracy as the best features are iteratively dropped


Applications of one class classification

  • Standard method proability distribution function of occurrence

    • Choose a feature set: frequencies of function words, syntactic structures, parts-of-speech n-grams, complexity and richness measure, syntactic and orthographic idiosyncrasies

      • Note: very different from text categorization by topic

    • Having constructed feature vectors, use learning algorithm to construct distinguishing model

      • Similar to categorization by topic

      • Liner separators are believed to work well

      • Assessment: k-fold cross-validation or bootstrapping


Applications of one class classification

  • One-class scenario proability distribution function of occurrence

    • Naïve approach

      • Chunk two works to generate two sufficient large sets

      • Test if we can distinguish using cross-validation with high accuracy

      • Failed in experiment (different works are just different enough to tell)

    • New approach: unmasking

      • In the above approach, a small number of features are doing all the work. They are likely to be from thematic differences, difference in genre or purpose, chronological shift of style, deliberate attempt to mask identity

      • Unmasking: removing features that are most useful to distinguish

      • Hypothesis: if they are by the same author, difference will be refelected in only a relative small number of features

      • Sudden degradation shows the same author


Applications of one class classification

  • Results: proability distribution function of occurrence

    • Corpus: 21 19th century English iterature

    • Baseline: one-class SVM

    • Extension: using negative samples to eliminate false positive

    • Solution to a literary mystery: the case of the bashful rabbi


Bibliography
bibliography proability distribution function of occurrence

  • D. M. J. Tax, “One-class classification”, PhD thesis, 2001

  • D. M. J. Tax, “Data description toolbox: A Matlab toolbox for data description, outlier and novelty detection”. 2005

  • M. Koppel and J. Schler, Authorship verification as a one-class classification problem, in Proceedings of 21st International Conference on Machine Learning, 2004.

  • R.E.Sanchez-Yanez et al, One-class texture classifier in the CCRfeature space, Pattern Recognition Letter, 24, 2003.

  • R.E.Sanchez-Yanez et al, A framework for texture classification using the coordinated clusters representation, Pattern Recognition Letter, 24, 2003.