1 / 107

# Johanna GOLD - PowerPoint PPT Presentation

Rough Sets Theory Logical Analysis of Data. Monday , November 26, 2007. Johanna GOLD. Introduction. Comparison of two theories for rules induction. Different methodologies Same results?. Generalities. Set of objects described by attributes. Each object belongs to a class.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Johanna GOLD' - kaori

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Logical Analysis of Data.

Monday, November 26, 2007

Johanna GOLD

• Comparison of two theories for rules induction.

• Different methodologies

• Same results?

• Set of objects described by attributes.

• Each object belongs to a class.

• We want decision rules.

• There are two approaches:

• Rough Sets Theory (RST)

• Logical Analysis of Data (LAD)

• Goal : compare them

Rough Sets Theory

Logical Analysis Of data

Comparison

Inconsistencies

• Two examples having the exact same values in all attributes, but belonging to two different classes.

• Example: two sick people have the same symptomas but different disease.

• RST doesn’t correct or aggregate inconsistencies.

• For each class : determination of lower and upper approximations.

• Lower : objects we are sure they belong to the class.

• Upper : objects than can belong to the class.

• Lower approximation → certain rules

• Upper approximation → possible rules

• Rules induction on numerical data → poor rules → too many rules.

• Need of pretreatment.

• Goal : convert numerical data into discrete data.

• Principle : determination of cut points in order to divide domains into successive intervals.

• First algorithm: LEM2

• Improved algorithms:

• Include the pretreatment

• MLEM2, MODLEM, …

• Induction of certain rules from the lower approximation.

• Induction of possible rules from the upper approximation.

• Same procedure

• For an attribute x and its value v, a block [(x,v)] of attribute-value pair (x,v) is all the cases where the attribute x has the value v.

• Ex : [(Age,21)]=[Martha]

[(Age,22)]=[David ; Audrey]

Definitions (2)

• Let B be a non-empty lower or upper approximation of a concept represented by a decision-value pair (d,w).

• Ex : (level,middle)→B=[obj1 ; obj5 ; obj7]

• Let T be a set of pairs attribute-value (a,v).

• Set B depends on set T if and only if:

• A set T is minimal complex of B if and only if B depends on T and there is no subset T’ of T such as B depends on T’.

• Let T be a non-empty collection of non-empty set of attribute-value pairs.

• T is a set of T.

• T is a set of (a,v).

• T is a local cover of B if and only if:

• Each member T of T is a minimal complex of B.

• T is minimal

principle

• LEM2’s output is a local cover for each approximation of the decision table concept.

• It then convert them into decision rules.

Among the possible blocks, we choose the one:

• With the highest priority

• With the highest intersection

• With the smallest cardinal

• As long as it is not a minimal complex, pairs are added.

• As long as there is not a local cover, minimal complexes are added.

• Illustration through an example.

• We consider that the pretreatment has already been done.

• For the attribute Height, we have the values 160, 170 and 180.

• The pretreatment gives us two cut points: 165 and 175.

Blocks [(a,v)]

• [(Height, 160..165)]={1,3,5}

• [(Height, 165..180)]={2,4}

• [(Height, 160..175)]={1,2,3,5}

• [(Height, 175..180)]={4}

• [(Hair, Blond)]={1,2}

• [(Hair, Red)]={3}

• [(Hair, Black)]={4,5,6}

• G = B = [(Attraction,-)] = {1,4,5,6}

• Here there is no inconsistencies. If there were some, it’s at this point that we have to chose between the lower and the upper approximation.

• Pair (a,v) such as [(a,v)]∩[(Attraction,-)]≠Ø

• (Height,160..165)

• (Height,165..180)

• (Height,160..175)

• (Height,175..180)

• (Hair,Blond)

• (Hair,Black)

• We chose the most appropriate, which is to say (a,v) for which

| [(a,v)] ∩ [(Attraction,-)] |

is the highest.

• Here : (Hair, Black)

• The pair (Hair, Black) is a minimal complex because:

• B = [(Attraction,-)] – [(Hair,Black)]

= {1,4,5,6} - {4,5,6}

= {1}

• Through the pairs (Height,160..165), (Height,160..175) and (Hair, Blond).

• Intersections having the same cardinality, we chose the pair having the smallest cardinal:

(Hair, Blond)

• Problem :

• (Hair, Blond) is non a minimal complex.

• We chose the following pair:

(Height,160..165).

• {(Hair, Blond),(Height,160..165)} is a second minimal complex.

• {{(Hair, Black)}, {(Hair, Blond), (Height, 160..165)}}

is a local cover of [(Attraction,-)].

• (Hair, Red) → (Attraction,+)

• (Hair, Blond) & (Height,165..180 ) → (Attraction,+)

• (Hair, Black) → (Attraction,-)

• (Hair, Blond) & (Height,160..165 ) → (Attraction,-)

Rough Sets Theory

Logical Analysis Of data

Comparison

Inconsistencies

• Work on binary data.

• Extension of boolean approach on non-binary case.

• Let S be the set of all observations.

• Each observation is described by n attributes.

• Each observation belongs to a class.

• The classification can be considered as a partition into two sets

• An archive is represented by a boolean function Φ :

• A literal is a boolean variable or its negation:

• A term is a conjunction of literals :

• The degree of a term is the number of literals.

• A term Tcovers a point

if T(p)=1.

• A characteristic term of a point p is the unique term of degree n covering p.

• Ex :

• A term T is an implicant of a boolean function f if T(p) ≤ f(p) for all

• An implicant is called prime if it is minimal (its degree).

• A positive prime patternis a term covering at least one positive example and no negative example.

• A negative prime patternis a term covering at least one negative example and no positive example.

• is a positive pattern :

• There is no negative example such as

• There is one positive example : the 3rd line.

• It's a positive prime pattern :

• covers one negative example : 4th line.

• covers one negative example : 5th line.

• symmetry between positive and negative patterns.

• Two approaches :

• Top-down

• Bottom-up

• we associate each positive example to its characteristic term→ it’s a pattern.

• we take out the literals one by one until having a prime pattern.

• we begin with terms of degree one:

• if it does not cover a negative example, it is a pattern

• If not, we add literals until having a pattern.

• We prefer short pattern → simplicity principle.

• we also want to cover the maximum of examples with only one model → globality principle.

• hybrid approach bottom-up – top-down.

• We fix a degree D.

• We start by a bottom-up approach to generate the models of degree lower or equal to D.

• For all the points which are not covered by the 1st phase, we proceed to the top-down approach.

• Extension from binary case : binerization.

• Two types of data :

• quantitative : age, height, …

• qualitative : color, shape, …

• For each value v that a qualitative attribute x can be, we associate a boolean variable b(x,v) :

• b(x,v) = 1 if x = v

• b(x,v) = 0 otherwise

• there are two types of associated variables:

• Level variables

• Interval variables

• For each attribute x and each cut point t, we introduce a boolean variable b(x,t) :

• b(x,t) = 1 if x ≥ t

• b(x,t) = 0 if x < t

• For each attribute x and each pair of cut points t’, t’’ (t’<t’’), we introduce a boolean variable b(x,t’,t’’) :

• b(x,t’,t’’) = 1 if t’ ≤ x < t’’

• b(x,t’,t’’) = 0 otherwise

• A set of binary attributes is called supporting set if the archive obtained by the elimination of all the other attributes will remained "contradiction-free".

• A supporting set is irredundant if there is no subset of it which is a supporting set.

• We associate to the attribute a variable

such as if the attribute belongs to the supporting set.

• Application : elements a and e are different on attributes 1, 2, 4, 6, 9, 11, 12 and 13 :

• We do the same for all pairs of true and false observations :

• Exponential number of solutions : we choose the smallest set :

our example

• Positive patterns :

• Negative patterns :

Rough Sets Theory

Logical Analysis Of data

Comparison

Inconsistencies

• LAD more flexible than RST

• Linear program -> modification of parameters

Comparisonblocks / variables

• RST : couples (attribute, value)

• Correspondence?

• For an attribute a taking the values :

• Discretization : convert numerical data into discrete data.

• Principle : determination of cut points in order to divide domains into successive intervals :

• RST : for each cut point, we have two blocks :

• LAD : for each cut point, we have a level variable :

• ...

• LAD : for each pair of cut points, we have a interval variable :

• ...

• Correspondence :

• Level variable :

• Correspondence :

• Interval variable :

• Three parameters can change :

• Right hand side of constraints:

• coefficients of the objective function:

• coefficients of the left hand side of the constraints:

• We try to adapt the three heuristics :

• The highest priority

• The highest intersection with the concept

• The smallest cardinality

• Priority on blocks -> priority on attributes

• Introduction as weights in the objective function

• Minimization : choice of pairs with first priorities

• Pb : in LAD, no notion of concept ; everything is done symmetrically, the same time.

• Modification of the heuristic : difference between the intersection with a concept and the intersection with the other.

• The highest, the better.

• Goal of RST : find minimal complexes:

• Find blocks covering the most examples of the concept : highest possible intersection with the concept

• Find blocks covering the less examples of the other concept : difference of intersections

• For LAD : difference between the number of times a variable takes the value 1 in

and in .

• Introduction as weights in the constraints : we choose first the variable with the highest difference.

• Simple : number of times a variable takes the value 1.

• Introduction as weight in the constraints.

• Two calculations to be introduced :

• The highest difference

• The smallest cardinality

• Difference of the two calculations

• Before : everything is 1.

• Pb : modification of the weights of the left hand side has no signification.

• Average of compared to the number of attributes.

• Average of in each constraint

• Inconvenient : not a real signification

• Not touch the weight in the constraints: introduce everything in the coefficients of the objective function:

Rough Sets Theory

Logical Analysis Of data

Comparison

Inconsistencies

• Use of two approximations : lower and upper.

• Rules generation: sure and possible.

• Classification mistakes: positive point classified as negative or the other way.

• Two different cases.

classified as neg.

• All other points are well classify : our point will not be covered.

• If the number of non covered points is high: generation of longer patterns.

• If this number is small : erroneous classification and we forgot the points for the following.

classified as pos.

• Terms covering a lot of positive points : also some negative points.

• Probably wrongly classified : not taken into account for the evaluation of candidates terms.

• We introduce a ratio.

• A term is still candidate if the ratio between negative and positive points is smallest than:

Inconsistenciesand mistakes

• An inconsistence can be considered as a mistake of classification

• Inconsistence : two « identical » objects differently classified.

• One of them is wrongly classified (approximations)

• Let consider an inconsistence in LAD :

• two points :

• two classes :

• There are two possibilities :

• is not covered by small degree patterns

• is covered by patterns of

1st case

• We have only one inconsistence.

• The covered point is isolated ; it’s not taken into account.

• Patterns of will be generated without the inconsistence point

-> lower approximation

2nd case

• A point covered by the other concept patterns is wrongly classified.

• It’s not taken into account for the candidate terms.

• It’s not taken into account for the pattern generation of

-> lower approximation

2nd case

• Not taken into account for but not a problem for

• For : upper approximation

• According to a ratio, LAD decide if a point is well classified or not.

• For an inconsistence, it’s the same as consider:

• The upper approximation of a class

• The lower approximation of the other

• On more than 1 inconsistence : we re-classify the points.

• Complete data : we can try to match LAD and RST.

• Inconsistencies : classification mistakes of LAD can correspond to approximations.

• Missing data : different management

• Jerzy W. Grzymala-Busse, MLEM2 - Discretization During Rule Induction, Proceedings of the IIPWM'2003, International Conference on Intelligent Information Processing and WEB Mining Systems, Zakopane, Poland, June 2-5, 2003, 499-508. Springer-Verlag.

• Jerzy W. Grzymala-Busse, Jerzy Stefanowski, Three Discretization Methods for Rule Induction, International Journal of Intelligent Systems, 2001.

• Endre Boros, Peter L. Hammer, Toshihide Ibaraki, Alexander Kogan, Eddy Mayoraz, Ilya Muchnik, An Implementation of Logical Analysis of Data, Rutcor Research Raport 22-96, 1996.

• Endre Boros, Peter L. Hammer, Toshihide Ibaraki, Alexander Kogan, Logical Analysis of Numerical Data, Rutcor Research Raport 04-97, 1997.

• Jerzy W. Grzymala-Busse, Rough Set Strategies to Data with Missing Attribute Values,Proceedings of theWorkshop on Foundation and New Directions in Data Mining, Melbourne, FL, USA. 2003.

• Jerzy W. Grzymala-Busse, Sachin Siddhaye, Rough Set Approaches to Rule Induction from Incomplete Data, Proceedings of the IPMU'2004, the 10th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based System[C],Perugia,Italy, July 4, 2004 2 : 923- 930.

• Jerzy Stefanowski, Daniel Vanderpooten, Induction of Decision Rules in Classi_cation and Discovery-Oriented Perspectives, International Journal of Intelligent Systems, 16 (1), 2001, 13-28.

• Jerzy Stefanowski, The Rough Set based Rule Induction Technique for Classification Problems, Proceedings of 6th European Conference on Intelligent Techniques and Soft Computing EUFIT 98, Aachen 7-10 Sept., (1998) 109.113.

• Roman Slowinski, Jerzy Stefanowski, Salvatore Greco, Benedetto Matarazzo, Rough Sets Processing of Inconsistent Information in Decision Analysis, Control and Cybernetics 29, 379±404, 2000.