Create Presentation
Download Presentation

Download Presentation
## Pattern Recognition Systems

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Pattern Recognition Systems • One field is concerned with systems that recognize patterns with a classification scheme that has already been determined. • Example: Character recognition, the categories (26 letter of alphabet) are already known. • Referred to as pattern classification.**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Pattern Recognition Systems • Other field is concerned with obtaining a classifier for a set of patterns where the categories are not initially known. • Example: Clustering data concerning patients to see whether patients can be grouped in a way that helps indicate diagnoses. • Referred to as Pattern clustering**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Pattern Classification • Attempt to pick the correct category when presented with a pattern. • Solution is very problem specific • Character recognition will be different from recognizing the tank in the battle field**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Design a Fuzzy Pattern Classifier • Pattern Features • Create fuzzy sets that represent features in the data that are considered important. • Example: ratio of the character’s height to its width may be important while the size of a character is not!**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Design a Fuzzy Pattern Classifier • Defining the System • fuzzy sets are used to convert crisp inputs to grades that are then combined by a pattern classifier to obtain grades for each of the possible classification categories. • Highly problem dependent!**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Design a Fuzzy Pattern Classifier • Testing the System • use known and unknown test sets. • Typically, the system will not perform as well as desired at first. • One may have to test a fuzzy classifier • find why it gave a wrong result, • make changes to improve the classifier.**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Application:Character Recognition • Goal: design a system that will pick the correct character when it is given a bit map representation of a printed or written input character. • Example: classifying the ten digits o through 9.**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Pattern Features • INPUT • bit map • 25 horizontal X 37 vertical pixels (925 pixel field) • each pixel have a 0 or 1 associated with it.**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Pattern Features • Creating a rule set dealing with 925 inputs would be extremely difficult and time consuming. • Fuzzify the input, resulting in a compact summary of the overall state of the bit map.**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Fuzzy sets for the Horizontal Position • Define five triangular-shaped fuzzy sets for the horizontal position. • xS = 0:24; • xL = eqshape(3,xS,5); • xG = shapeval(xL, xS);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Fuzzy sets for the Vertical Position • Define seven triangular-shaped fuzzy sets for the vertical position. • yS = 0:36; • yL = eqshape(3,yS,7); • yG = shapeval(yL, yS);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Fuzzify the input pixel • Given any pixel (horizontal x and verticle y), the membership of that pixel can be calculated for each of 5 horizontal and 7 vertical regions. • G1 = fuzzyval (xS,xG,x); • G2 = fuzzyval (yS,yG,x);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Crate 5 x 7 regions of the 2-D bitmap • Find all combinations of intersections between the two groups of fuzzy sets • G3 = allcomb(‘min’, G1, g2); • G3 wil be a 35-element column vector of grades associated with the 35 regions of the 2-D bit map**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Multiple Pixels • If vectors X and Y represent the input pixels, then • G1 = fuzzyval(xS, xG, X); • G2 = fuzzyval(yS,yG, Y); and • G3 = allcomb(‘min’, G1, G2); • G3 represents a 35 -element column for every pixel crossed by all of the input pixels**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Single Grade for each 5 x 7 regions • Taking the union of all grades associated with each square region: • G = or (G3 ’ ) ’; • We now have 35 grades, each grade measuring to what extent the user’s drawn character passed through a different region. • 925-pixel bit map ----> 35 grades!**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • Once the good features have been found, one can define a system to transform the feature grades into the grades for each class.**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • Zero: • G0 = and([ or( G([2 7])); or( G([3 8])); or( G([4 9])); • or( G([11 12])); or( G([16 17])); or( G([21 22])); • or( G([14 15])); or( G([19 20])); or( G([24 25])); • or( G([27 32])); or( G([28 33])); or( G([29 34])); • not( G([13 18 23]) ) ]);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • One:**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • One: • G1 = or([ and([ G([5 9 18 27 31]); • not(G([1 2 3 6 7 11 25 29 30 33 34 35])) ]); • and([ G([1 7 18 29 35]); • not(G([3 4 5 9 10 15 21 26 27 31 32 33])) ]); • and([ G([1 6 11 16 21 26 31]); • not(G([3 4 5 8 10 13 15 18 20 23 25 ... • 28 30 33 34 35])) ]); • and([ G([2 3 4 5]); • not(G([6 7 10 11 12 15 16 17 20 25 30 35])); • or( G([8 9])); • or( G([13 14])); • or( G([18 19])); • or( G([23 24])); • or( G([28 29])); • or( G([33 34])); ]) ]);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • Two: • G2 = and([ somewhat(G(2)); • G([3 32 33 34]); • or( G([6 7])); or( G([30 25])); or( G([4 5])); • not([ G([22 28]); and(G([10 15])) ]) ]);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • Three: • G3 = and([ G([2 3 19 32 33 34]); • or( G([9 10]) ); • or( G([14 15]) ); • not( G([5 16 21]) ) ]);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • Four: • G4 = and([ or([ and(G([4 14 19 29])); and(G([5 15 20 30])) • and(G([3 13 18 28])) ]); • or([and(G([12 14 15])); and(G([17 19 20])); • and(G([22 24])) ]); • not([ G([1 2 6 7]); and(G([32 33 34])) ]) ]);**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Defining the System • Once the grades for each class have been found, they can be combined into a single grade vector, cG • cG = [G0 G1 G2 G3 G4 G5 G6 G7 G8 G9]; • THEN THE MAXIMUMGRADE CHARACTERS CAN BE FOUND • C = cS(find(cG >= max(cG)));**FuzzySystems ToolBox, Mark Beale and Howard Demuth**Testinging the System • Demo • Matlab : numrec**FuzzyLogic: Intellicence, Control, and Information, J Yen**and R Langari, Prentice Hall Pattern Clustering • Unsupervised Clustering • Find interesting patterns or groupings in a given set of data. • Example: perform the task of “segmenting” the images (I.e., partitioning pixels on an image into regions that correspond to different objects or different faces of objects in the images).**FuzzyLogic: Intellicence, Control, and Information, J Yen**and R Langari, Prentice Hall Pattern Clustering • Conventional clustering algorithms find a “Hard partition” of a given dataset based on certain criteria that evaluate the goodness of a partition. • “Hard Partition” - each datum belongs to exactly one cluster of the partition. • In many real-world clustering problems, some data points partially belong to multiple clusters (eg. A pixel in a MRI may correspond to a mixture of two different types of tissues.**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Soft Clustering Algorithms • Find a “soft partition” of a given data set based on certain criteria. • A datum can partially belong to multiple clusters • In practice, most soft clustering algorithms do generate a soft partition that also forms a fuzzy partition.**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means Algorithm • Best known fuzzy clustering algorithm • produces a constrained soft partition • I.e. • Smcj(xi) = 1 • Both fuzzy c-means and probabilistic clustering produce a partition of similar properties, the clustering criteria underlying these algorithms are very different!**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means Algorithm • Generalizes a hard clustering algorithm called the c-means algorithm • Hard C-Means algorithm (HCM) aims to identify compact well-separated clusters • compact cluster has a “ball like” shape • the center of the ball is called the center or the prototype of the cluster.**Example of two clusters that are not compact and well**separated FuzzyLogic: Intellicence, Control, and Information, J Yen and R Langari, Prentice Hall**Example of two clusters that are compact, but not well**separated FuzzyLogic: Intellicence, Control, and Information, J Yen and R Langari, Prentice Hall**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means Algorithm • Like HCM, fuzzy c-means also tries to find a good partition by searching for prototypes vi that minimize the objective function Jm • k=1,2,…c • m is a weight that affects how partial members of a cluster affect the clustering result • FCM also needs to search for membership function mci that minimize Jm**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means Theorem • A constrainted fuzzy partition {C1, C2, …, Ck} can be a local minimum of the objective function Jm only if the following conditions are satisfied: • Based on this theorem, FCM updates the prototypes and the membership function iteratively using Equation A and B until a convergence criterion is reached**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means Algorithm**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means - Example • Use FCM to partition the data set into two clusters (c=2), suppose we set the parameter m in FCM at 2, and initial prototypes to v1 = (5,5) v2 = (10,10)**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means - Example**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Fuzzy C-Means - Example • The initial membership functions of the two clusters are calculated using Equation A**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Example (Continued) • Similarly,**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Example (Continued) • Similarly,**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Example (Continued) • FCM algoruthm then updates the prototypes according to Equation B • For V1**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Example (Continued) • FCM algoruthm then updates the prototypes according to Equation B • For V2**FuzzyLogic: Intelligence, Control, and Information, J. Yen**and R. Langari, Prentice Hall Example (Continued) • The updated prototype v1 and v2 are move closer to the center of the cluster formed by x1,x2, x3 and x4,x5,x6 , respectively • Show Matlab example**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes • Used in PDAs • Interactive CAD systems**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes Multi-stroke shapes**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes Multi-stroke shapes**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes Multi-stroke shapes**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes Uni-stroke shapes**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes Uni-stroke shapes**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes Recognition algorithm: 1. Use global geometric properties of the shapes 2. Use a set of filters to either to identify shapes or to filter out unwanted outcomes 3. Use fuzzy logic to handle uncertainty**Manuel J.Fonseca, Joaquim A.Jorge, Using Fuzzy Logic to**Recognize Geometric Shapes Interactively Recognition of Geometric shapes Feature extraction Use statistical properties of shape’s features!