fuzzy interpretation of discretized intervals dr xindong wu n.
Skip this Video
Loading SlideShow in 5 Seconds..
Fuzzy Interpretation of Discretized Intervals Dr. Xindong Wu PowerPoint Presentation
Download Presentation
Fuzzy Interpretation of Discretized Intervals Dr. Xindong Wu

Loading in 2 Seconds...

play fullscreen
1 / 33

Fuzzy Interpretation of Discretized Intervals Dr. Xindong Wu - PowerPoint PPT Presentation

  • Uploaded on

Fuzzy Interpretation of Discretized Intervals Dr. Xindong Wu. Andrea Porter April 11, 2002. Plan For Presentation. Introduction to Problem, HCV Discretization Techniques/Fuzzy Borders A Hybrid Solution for HCV Experiments and Results Conclusion. Introduction.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Fuzzy Interpretation of Discretized Intervals Dr. Xindong Wu' - jaden

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
plan for presentation
Plan For Presentation
  • Introduction to Problem, HCV
  • Discretization Techniques/Fuzzy Borders
  • A Hybrid Solution for HCV
  • Experiments and Results
  • Conclusion
  • Real-world data contains both numerical and nominal data, must be able to deal with different types of data.
  • Existing systems discretize numerical domains into intervals and treat intervals as nominal values during induction.
  • Problems occur if test examples are not covered in training data (no-match, multiple match)
  • The solution is a hybrid approach using fuzzy intervals for no-match problem.
  • Attribute based rule induction algorithm, extension matrix approach
    • Divide positive examples into intersecting groups
    • Find a heuristic conjunctive rule in each group that covers all PE and no NE
  • HCV can find a rule in the form of variable-valued logic
  • More compact than the decision trees/rules of ID3 and C4.5
variable valued logic and selectors
Variable Valued Logic and Selectors
  • Represents decisions where variables can take a range
  • Selector:

[ X # R ]

X = attribute

# = relational operator ( = , <, >, . . . )

R = Reference, list of 1 or more values

e.g [ Windy = true] [Temp > 90]

hcv software
HCV Software
  • C++ implementation
  • Can work with noisy and real-valued domains as well as nominal and noise-free databases
  • Provides a set of deduction facilities for the user to test the accuracy of the produced rules on test examples
c4 5 results vs hcv
C4.5:The T class

X2 = b

X1 = 0 & X3 = a

X1 = 0 & X3 = b

X1 = 0 & X2 = a

C4.5 Results vs. HCV
  • HCV:The T class
  • X2 = b
  • X1 = 0 & X2 = a
  • X1 = 0 & X4 = 0
  • C4.5:The F class

X1 = 1 & X2 = a

    • X1 = 1 & X2 = c
    • X2 = c & X3 = c
deduction of induction results
Deduction of Induction Results
  • Induction generates knowledge from existing data
  • Deduction applies induction results to interpret new data.
  • With real-world data, induction can not be assumed to be perfect
  • Three cases:

1) no-match (measure of fit)

2) single-match

3) multiple-match (estimate of probability)

  • Occurs during rule induction
  • Discretize numerical domains into intervals and treat similar to nominal values.
  • The challenge is to find the right borders for the intervals
  • Possible Methods:

1) Simplest Class-Separating Method

2) Information Gain Heuristic (implemented in HCV)

simplest class separating method
Simplest Class- Separating Method:
  • Interval Borders are places between each adjacent pair of examples which have different classes.
  • If attribute is very informative - method is efficient and useful.
  • If attribute is not informative - method produces too many intervals
information gain heuristic
Information Gain Heuristic

Use IGH to find more informative border.

  • x = (xi + xi+1)/2 for (i = 1, …, n-1)
  • x is a possible cut point if xi and xi+1 are of different classes.
  • Use IGH to find best x
  • Recursively split on left and right
  • To stop recursive splitting:

1) stop if IGH is same on all possible cut points.

2) stop if # of examples to split is less than a predefined number

3) limit the number of intervals

fuzzy borders
Fuzzy Borders
  • Discretization of continuous domains does not always fit accurate interpretation.
  • Instead of using sharp borders, use a membership function, measures the degree of membership.
  • A value can be classified into a few different intervals at the same time (e.g. single to multiple match)
fuzzy borders 2
Fuzzy Borders (2)
  • Fuzzy matching - deduction with fuzzy borders of discretized intervals.
  • Take the interval with the greatest degree as the value’s discrete value.
  • 3 functions to fuzzify borders:

1) linear

2) polynomial

3) arctan

  • Definitions

s = spread parameter l = length of original

xleft, xright = left/right sharp borders


xleft xright

linear membership function



xleft xright

Linear Membership Function

a = -kxleft + 1/2 b = kxright + 1/2

linleft(x) = kx + a

lin right(x) = -kx + b

lin(x) = MAX(0, MIN(1,linleft(x),linright(x)))

k = 1/2sl

polynomial membership function
*Polynomial Membership Function

polyside(x) = asidex3 + bsidex2 + csidex + dside

aside = 1/(4(ls)3)

bside = -3asidexside side {left,right}

cside = 3aside(xside2 - (ls)2)

dside = -a(xside3 -3xside(ls)2 + 2(ls)3)

polyleft(x), if xleft -ls  x  xleft + ls

poly(x) = polyright(x), if xright -ls  x  xright +ls

1, if xleft +ls  x  xright -ls

0, otherwise

match degree
Match Degree
  • Selector method - take the max membership degree of the value in all the intervals involved. If 2 adjacent intervals have the same class, values close to the border will have low membership.
  • Conjunction method - adds with fuzzy plus

ab=a + b - ab

no match resolution
No-Match Resolution

Largest Class

  • Assign all no match examples to the largest class, the default class.
  • Works well, if the number of classes in a training set is small and one class is clearly larger.
  • Deteriorates if there is a larger number of classes and the examples are evenly distributed
no match resolution1
No-Match Resolution

Measure of Fit

Calculate the measure of fit for each class:

1) calculate MF for each selector (sel)

MF(sel, e) = 1, if sel is satisfied by e

n/|x|, otherwise

2) calculate MF for each conjunctive rule(conj)

MF(conj, e) =  MF(sel, e) * n(conj)/N

no match resolution2
No-Match Resolution

Measure of Fit (2)

3) calculate MF for each class c

MF(c, e) = MF(conj1, e) + MF(conj2, e) - MF(conj1,e)MF(conj2,e)

* For more than two rules, apply formula recursively.

* Find maximum MF - determines which class is closest to the example

multiple match
  • Caused by over-generalization of the training examples at induction time
  • Example
    • (X1 = a, X2 = 1)
      • All PE cover X1 = a
      • All NE cover X2 = 1
      • Multiple Match
multiple match resolution
Multiple-Match Resolution

First Hit

  • Use first rule which classifies the example
  • Produces reasonable results if the rules from induction have been ordered according to a measure of reliability
  • Advantages - straightforward, efficient
  • Disadvantages - have to sort rules at induction time
multiple match resolution1
Multiple-Match Resolution

Largest Rule

  • Similar to largest class method from no-match resolution
  • Choose conjunctive rule that covers the most examples in the training set.
multiple match resolution2
Multiple-Match Resolution

Estimation of Probability

  • Assign EP value to each class based on the size of the satisfied conjunctive rules.

1) Find EP for each conjunctive rule (conj):

EP(conj, e)= { n(conj)/N, if conj is satisfied by e

0, otherwise

n(conj) = number of examples covered by conj

N = number of total examples

multiple match resolution3
Multiple-Match Resolution

Estimation of Probability (2)

2) Find EP value for each class:

EP(c, e) = EP(conj1, e) + EP(conj2, e) - EP(conj1,e)EP(conj2,e).

* For more rules, apply formula recursively

* Choose class with highest EP value

hybrid interpretation
Hybrid Interpretation
  • Used because fuzzy borders only add conflicts because they don’t reduce the number rules that are applicable
  • HCV - use sharp borders during induction and use fuzzy borders only during deduction
  • Algorithm:

* Single match - use class indicated by rules

* Multiple match - use estimation probability (EP) with sharp borders

* No match - use fuzzy borders with polynomial membership function to find closest rule

the data
The Data
  • Used 17 databases from the Machine Learning Database Repository, U. of California, Irvine.
  • Databases selected because:

1) All include numerical data

2) All lead to situations where no rules clearly apply.

results cont
Results (cont.)
  • The results shown for C4.5 and NewID are the pruned ones
    • These were usually better than the unpruned ones in this experiment
  • HCV did not fine tune different parameters because this would be loss of generality and applicability of the conclusions
accuracy results
Accuracy Results
  • HCV(hybrid) - 9 databases
  • C4.5 (R 8) - 7 databases
  • C4.5 (R 5) - 6 databases
  • HVC (large) - 3 databases
  • HCV (fuzzy) - 2 databases
hcv comparison
HCV Comparison
  • HCV (fuzzy) generally performs better than the simple largest class method
  • HCV’s performance improves significantly when the fuzzy borders (for no match) are combined with probability estimation (for multiple match) in HCV (hybrid)
  • Fuzzy borders are constructed and used at deduction time only when a no match case occurs.
  • This hybrid method performs more accurately than several other current deduction programs.
  • Fuzziness is strongly domain dependent, HCV allows the user to specify their own intervals and fuzzy functions.