a dynamic programming algorithm for hierarchical discretization of continuous attributes l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
A dynamic-programming algorithm for hierarchical discretization of continuous attributes PowerPoint Presentation
Download Presentation
A dynamic-programming algorithm for hierarchical discretization of continuous attributes

Loading in 2 Seconds...

play fullscreen
1 / 31

A dynamic-programming algorithm for hierarchical discretization of continuous attributes - PowerPoint PPT Presentation


  • 199 Views
  • Uploaded on

A dynamic-programming algorithm for hierarchical discretization of continuous attributes. Amit Goyal (15 st April 2008) Department of Computer Science The University of British Columbia. Reference.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'A dynamic-programming algorithm for hierarchical discretization of continuous attributes' - Ava


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
a dynamic programming algorithm for hierarchical discretization of continuous attributes

A dynamic-programming algorithm for hierarchical discretization of continuous attributes

Amit Goyal (15st April 2008)

Department of Computer Science

The University of British Columbia

reference
Reference
  • Ching-Cheng Shen and Yen-Liang Chen. A dynamic-programming algorithm for hierarchical discretization of continuous attributes. In European Journal of Operational Research 184 (2008) 636-651 (ElseVier).

Amit Goyal (UBC Computer Science)

overview
Overview
  • Motivation
  • Background
  • Why need Discretization?
  • Related Work
  • DP Solution
  • Analysis
  • Conclusion

Amit Goyal (UBC Computer Science)

motivation
Motivation
  • Situation: Attrition rate for mobile phone customer is around 25-30% per year
  • Task:
    • Given customer information for past N months, predict who is likely to attrite next month
    • Also estimate customer value & what is the cost effective order to be made to this customer
  • Customer Attributes:
    • Age
    • Gender
    • Location
    • Phone bills
    • Income
    • Occupation

Amit Goyal (UBC Computer Science)

pattern discovery
Pattern Discovery

t1: Beef, Chicken, Milk

t2: Beef, Cheese

t3: Cheese, Boots

t4: Beef, Chicken, Cheese

t5: Beef, Chicken, Clothes, Cheese, Milk

t6: Chicken, Clothes, Milk

t7: Chicken, Milk, Clothes

  • Transaction data
  • Assume:
    • min_support = 30%
    • min_confidence = 80%
  • An example frequent itemset:
    • {Chicken, Clothes, Milk} [sup = 3/7]
  • Association rules from the itemset:
    • Clothes  Milk, Chicken [sup = 3/7, conf = 3/3]
    • … …
    • Clothes, Chicken  Milk, [sup = 3/7, conf = 3/3]

Amit Goyal (UBC Computer Science)

issues with numeric attributes
Issues with Numeric Attributes
  • Size of the discretized intervals affect support & confidence

{Occupation = SE, (Income = $70,000)}  {Attrition = Yes}

{Occupation = SE, (60K  Income  80K)}  {Attrition = Yes}

{Occupation = SE, (0K  Income  1B)}  {Attrition = Yes}

    • If intervals too small
      • may not have enough support
    • If intervals too large
      • may not have enough confidence
  • Loss of Information (How to minimize?)
  • Potential solution: use all possible intervals
  • Too many rules!!!

Amit Goyal (UBC Computer Science)

background
Background
  • Discretization
    • reduce the number of values for a given continuous attribute by dividing the range of the attribute into intervals.
  • Concept hierarchies
    • reduce the data by collecting and replacing low level concepts (such as numeric values for the attribute age) by higher level concepts (such as young, middle-aged, or senior).

Amit Goyal (UBC Computer Science)

why need discretization
Why need discretization?
  • Data Warehousing and Mining
    • Data reduction
    • Association Rule Mining
    • Sequential Patterns Mining
  • In some machine learning algorithms like Bayesian approaches and Decision Trees.
  • Granular Computing

Amit Goyal (UBC Computer Science)

related work
Related Work
  • Manual
  • Equal-Width Partition
  • Equal-Depth Partition
  • Chi-Square Partition
  • Entropy Based Partition
  • Clustering

Amit Goyal (UBC Computer Science)

simple discretization methods binning
Simple Discretization Methods: Binning
  • Equal-width (distance) partitioning:
    • It divides the range into N intervals of equal size: uniform grid
    • if A and B are the lowest and highest values of the attribute, the width of intervals will be: W = (B-A)/N.
    • The most straightforward
  • Equal-depth (frequency) partitioning:
    • It divides the range into N intervals, each containing approximately same number of samples

Amit Goyal (UBC Computer Science)

chi square based partitioning
Chi-Square Based Partitioning
  • Χ2 (chi-square) test
  • The larger the Χ2 value, the more likely the variables are related
  • Merge: Find the best neighboring intervals and merge them to form larger intervals recursively

Amit Goyal (UBC Computer Science)

entropy based partition
Entropy Based Partition
  • Given a set of samples S, if S is partitioned into two intervals S1 and S2 using boundary T, the entropy after partitioning is
  • The boundary that minimizes the entropy function over all possible boundaries is selected as a binary discretization.
  • The process is recursively applied to partitions obtained until some stopping criterion is met

Amit Goyal (UBC Computer Science)

clustering
Clustering
  • Partition data set into clusters based on similarity, and store cluster representation (e.g., centroid and diameter) only
  • Can be very effective if data is clustered but not if data is “smeared”
  • Can have hierarchical clustering and be stored in multi-dimensional index tree structures
  • There are many choices of clustering definitions and clustering algorithms

Amit Goyal (UBC Computer Science)

weaknesses
Weaknesses
  • Seeks a local optimal solution instead of a global optimal
  • Subject to constraint that each interval can only be partitioned into a fixed number of sub-intervals
  • Constructed tree may be unbalanced

Amit Goyal (UBC Computer Science)

notations
Notations
  • val(i): value of ith data
  • num(i): number of occurrences of value val(i)
  • R: depth of the output tree
  • ub: upper boundary on the number of subintervals spawned from an interval
  • lb: lower boundary

Amit Goyal (UBC Computer Science)

example
Example

R = 2, lb = 2, ub = 3

Amit Goyal (UBC Computer Science)

problem definition
Problem Definition

Given parameters R, ub, and lb and input data val(1), val(2), …, val(n) and num(1), num(2), … num(n), our goal is to build a minimum volume tree subject to the constraints that all leaf nodes must be in level R and that the branch degree must be between ub and lb

Amit Goyal (UBC Computer Science)

distances and volume
Distances and Volume
  • Intra-distance of a node containing data from data i to data j
  • Inter-distance b/w two adjacent siblings; first node containing data from i to u, second node containing data from u+1 to j
  • Volume of a tree is the total intra-distance minus total inter-distance in the tree

Amit Goyal (UBC Computer Science)

theorem
Theorem
  • The volume of a tree = the intra-distance of the root node + the volumes of all its sub-trees - the inter-distances among its children

Amit Goyal (UBC Computer Science)

notations20
Notations
  • T*(i,j,r): the minimum volume tree that contains data from data i to data j and has depth r
  • T(i,j,r,k): the minimum volume tree that contains data from data i to data j, has depth r, and whose root has k branches
  • D*(i,j,r): the volume of T*(i,j,r)
  • D(i,j,r,k): the volume of T(i,j,r,k)

Amit Goyal (UBC Computer Science)

notations cont
Notations Cont.

Amit Goyal (UBC Computer Science)

notations cont22
Notations Cont.

Amit Goyal (UBC Computer Science)

algorithm
Algorithm

Amit Goyal (UBC Computer Science)

algorithm cont
Algorithm Cont.

Amit Goyal (UBC Computer Science)

algorithm cont25
Algorithm Cont.

Amit Goyal (UBC Computer Science)

the complete dp algorithm
The complete DP algorithm

Amit Goyal (UBC Computer Science)

slide27

Run times of different algorithms

Volume of trees constructed

Gain Ratios of different algorithms

(Monthly Household Income)

Gain Ratios of different algorithms

(Money Spent Monthly)

Amit Goyal (UBC Computer Science)

conclusion
Conclusion
  • Global optima instead of local optima
  • Each interval is partitioned into the most appropriate number of subintervals
  • Trees are balanced
  • Time complexity is cubic, thus slightly slower

Amit Goyal (UBC Computer Science)

http www cs ubc ca goyal goyal@cs ubc ca
http://www.cs.ubc.ca/~goyal(goyal@cs.ubc.ca)

Thank you !!!

Amit Goyal (UBC Computer Science)

gain ratio
Gain Ratio

The information gain due to particular split of S into Si, i = 1, 2, …., r

Gain (S, {S1, S2, …., Sr) = purity(S ) – purity (S1, S2, … Sr)

Amit Goyal (UBC Computer Science)

chi square test example
Chi-Square Test Example

In order to see whether this result is statistically significant, the P-value (the probability of this result not being due to chance) must be calculated or looked up in a chart. The P-value is found to be Prob(X21 ≥ 0.36) = 0.5485. There is thus a probability of about 55% of seeing data that deviates at least this much from the expected results if indeed the coin is fair. Hence, fair coin.

Amit Goyal (UBC Computer Science)