1 / 50

Data Warehousing and Data Mining

Data Warehousing and Data Mining. Prepared By: Amit Patel http://ajpatelit.hpage.com. Decision Tree. A DECISION TREE is a flowchart-like tree structure, W here each internal node ( nonleaf node) denotes a test on an attribute , E ach branch represents an outcome of the test ,

kevlyn
Download Presentation

Data Warehousing and Data Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Warehousing and Data Mining Prepared By: Amit Patel http://ajpatelit.hpage.com

  2. Decision Tree • A DECISION TREE is a flowchart-like tree structure, • Where each internal node (nonleafnode) denotes a test on an attribute, • Each branch represents an outcome of the test, • And each leaf node (or terminal node) holds a class label. • The topmost node in a tree is the root node. http://ajpatelit.hpage.com

  3. Decision Tree http://ajpatelit.hpage.com

  4. Decision Tree • Algorithm: Generate_decision_tree. Generate a decision tree from the training tuples of data partition D. • Input: • Data partition, D, which is a set of training tuples and their associated class labels; • Attribute_list, the set of candidate attributes; • Attribute_selection_method, a procedure to determine the splitting criterion that “best” partitions the data tuples into individual classes. This criterion consists of a splitting attribute and, possibly, either a split point or splitting subset. • Output: A decision tree. http://ajpatelit.hpage.com

  5. Decision Tree http://ajpatelit.hpage.com

  6. Let A be the splitting attribute. (a) If A is discrete-valued, then one branch is grown for each known value of A.(b) If A is continuous-valued, then two branches are grown, corresponding to A < split point and A > split point. (c) If A is discrete-valued and a binary tree must be produced, then the test is of the form A 2 SA, where SA is the splitting subset for A.

  7. Let A be the splitting attribute. (a) If A is discrete-valued, then one branch is grown for each known value of A. (b) If A is continuous-valued, then two branches are grown, corresponding to A split point and A > split point. (c) If A is discrete-valued and a binary tree must be produced, then the test is of the form A 2 SA, where SA is the splitting subset for A. where http://ajpatelit.hpage.com

  8. Naïve Bayesian Classification • Procedure: • Assume that there are a set of m samples S={s1, s2,…,sm} (the training data set). • Where every sample si is represented as an n-dimensional vector (x1,x2,…,xn) • Values xi correspond to attributes A1, A2,…, An respectively. • Also, there are k classes C1, C2,…, Ck, and every sample belongs to one of these classes. • These probabilities are computed using Bayes’ Theorem: • P(Ci|X)=[P(X|Ci).P(Ci)]/P(X) • As P(X) is constant for all classes, only the product P(X|Ci).P(Ci) needs to be maximized. http://ajpatelit.hpage.com

  9. Naïve Bayesian Classification http://ajpatelit.hpage.com

  10. Naïve Bayesian Classification http://ajpatelit.hpage.com

  11. Naïve Bayesian Classification • The data tuples are described by the attributesage, income, student, and credit rating. The class label attribute, buys_computer, has two distinct values namely, {yes, no}. • Let Class: • C1: buys_computer= “yes” • C2: buys_computer= “no” • The data sample we want to classify is: • X=(age=youth, income=medium, student=yes, credit rating=fair) http://ajpatelit.hpage.com

  12. Naïve Bayesian Classification • We need to maximize P(X|Ci).P(Ci), for i = 1, 2. P(Ci), the prior probability of each class, can be computed based on the training tuples: • P(buys computer = yes) • P(buys computer = no) • To compute P(X|Ci), for i = 1, 2, we compute the following conditional probabilities: = 9/14 = 0.643 = 5/14 = 0.357 http://ajpatelit.hpage.com

  13. Naïve Bayesian Classification = 2/9 = 0.222 • P(age = youth |buys_computer= yes) • P(age = youth |buys_computer= no) • P(income = medium |buys_computer= yes) • P(income = medium |buys_computer= no) • P(student = yes | buys_computer= yes) • P(student = yes |buys_computer= no) • P(credit_rating= fair |buys_computer= yes) • P(credit_rating= fair |buys_computer= no) • X=(age=youth, income=medium, student=yes, credit_rating=fair) = 3/5 = 0.600 = 4/9 = 0.444 = 2/5 = 0.400 = 6/9 = 0.667 = 1/5 = 0.200 = 6/9 = 0.667 = 2/5 = 0.400 http://ajpatelit.hpage.com

  14. Naïve Bayesian Classification • Using the above probabilities, we obtain • P(X|buys_computer= yes) = • P(age = youth |buys_computer= yes) * • P(income = medium |buys_computer= yes) * • P(student = yes |buys_computer= yes) * • P(credit_rating= fair |buys_computer= yes) • = 0.2220 * 0.4440 * 0.6670 * 0.667 = 0.044 http://ajpatelit.hpage.com

  15. Naïve Bayesian Classification Similarly, P(X|buys_computer= no) = 0.6000 * 0.4000 * 0.2000 * 0.4000 = 0.019 To find the class, Ci, that maximizes P(X|Ci).P(Ci), we compute P(X|buys_computer= yes) * P(buys_computer= yes) = 0:0440 * 0.643 = 0.028 P(X|buys_computer= no) * P(buys_computer= no) = 0.0190 * 0.357 = 0.007 Therefore, the naïve Bayesian classifier predicts buys_computer= yes for tuple X. http://ajpatelit.hpage.com

  16. Rule-Based Classification • Using IF-THEN Rules for Classification • Rules are a good way of representing information or bits of knowledge. A rule-based classifier uses a set of IF-THEN rules for classification. An IF-THEN rule is an expression of the form • IF condition THEN conclusion. • An example is rule R1, • R1: IF age = youth AND student = yes THEN • buys computer = yes. http://ajpatelit.hpage.com

  17. Rule-Based Classification • The “IF”-part (or left-hand side)of a rule is known as the rule antecedent or precondition. • The “THEN”-part (or right-hand side) is the rule consequent. • In the rule antecedent, the condition consists of one or more attribute tests (such as age = youth, and student = yes) that are logically ANDed. • The rule’s consequent contains a class prediction (in this case, we are predicting whether a customer will buy a computer). • R1 can also be written as • R1: (age = youth)^(student = yes)  (buys computer = yes). http://ajpatelit.hpage.com

  18. Rule-Based Classification • If the condition (that is, all of the attribute tests) in a rule antecedent holds true for a given tuple, we say that the rule antecedent is satisfied (or simply, that the rule is satisfied) and that the rule covers the tuple. • A rule R can be assessed by its coverage and accuracy. • Given a tuple, X, from a class labeled data set, D, • let ncoversbe the number of tuples covered by R; • ncorrectbe the number of tuples correctly classified by R; and • |D|be the number of tuples in D. http://ajpatelit.hpage.com

  19. Rule-Based Classification We can define the coverage and accuracy of R as: Find the Coverage and Accuracy for the Rule R1: IF age = youth AND student = yes THEN buys_computer= yes ??? http://ajpatelit.hpage.com

  20. Rule-Based Classification http://ajpatelit.hpage.com

  21. Rule-Based Classification • These are class-labeled tuples from the AllElectronicscustomer database. • Our task is to predict whether a customer will buy a computer. • Consider rule R1 above, which covers 2 of the 14 tuples. It can correctly classify both tuples. • Therefore, • coverage(R1) • accuracy (R1) = 2/14 = 14.28% = 2/2 = 100% http://ajpatelit.hpage.com

  22. Rule-Based Classification R2: IF age = youth AND student = no THEN buys_computer= no coverage(R2) = accuracy (R2) = R3: IF age = senior AND credit_rating= excellent THEN buys_computer= yes coverage(R3) = accuracy (R3) = R4: IF age = middle_agedTHEN buys_computer= yes coverage(R4) = accuracy (R4) = 3/14 = 21.43% 3/3 = 100% 2/14 = 14.28% 0/2 = 0% 4/14 = 28.57% 4/4 = 100% http://ajpatelit.hpage.com

  23. Basic Sequential Covering algorithm IF-THEN rules can be extracted directly from the training data (i.e., without having to generate a decision tree first) using a sequential covering algorithm. • Input: • D, a data set class-labeled tuples; • Att_vals, the set of all attributes and their possible values. http://ajpatelit.hpage.com

  24. Rule-Based Classification http://ajpatelit.hpage.com

  25. CART Classification and Regression Trees • University of California- a study into patients after admission for a heart attack. • 19 variables collected during the first 24 hours for 215 patients (for those who survived the 24 hours) • Question: Can the high risk (will not survive 30 days) patients be identified? http://ajpatelit.hpage.com

  26. CART Is the minimum systolic blood pressure over the 1st24 hours>91? Is age>62.5? H Is sinus present? L H L http://ajpatelit.hpage.com

  27. CART Plan for construction of a CART • Selection of the Splits • Decisions when to decide that a node is a terminal node (i.e. not to split it any further) • Assigning a class to each terminal node http://ajpatelit.hpage.com

  28. X CART Where splits need to be evaluated…??? Sorted by Blood Pressure Sorted byAge X http://ajpatelit.hpage.com

  29. CART Features of CART • Binary Splits • Splits based only on one variable http://ajpatelit.hpage.com

  30. CART Advantages of CART • Can cope with any data structure or type • Classification has a simple form • Uses conditional information effectively • Invariant under transformations of the variables • Is robust with respect to outliers • Gives an estimate of the misclassification rate http://ajpatelit.hpage.com

  31. CART Disadvantages of CART • CART does not use combinations of variables • Tree can be deceptive – if variable not included it could be as it was “masked” by another • Tree structures may be unstable – a change in the sample may give different trees • Tree is optimal at each split – it may not be globally optimal. http://ajpatelit.hpage.com

  32. Neural Networks • Neural networks are useful for data mining and decision-support applications. • People are good at generalizing from experience. • Computers excel at following explicit instructions over and over. • Neural networks bridge this gap by modeling, on a computer, the neural behavior of human brains. http://ajpatelit.hpage.com

  33. Neural Networks • Neural Networks map a set of input-nodes to a set of output-nodes • Number of inputs/outputs is variable • The Network itself is composed of an arbitrary number of nodes with an arbitrary topology http://ajpatelit.hpage.com

  34. Neural Networks • A neuron: many-inputs / one-output unit • Output can be excited or not excited • Incoming signals from other neurons determine if the neuron shall excite ("fire") • Output subject to attenuation in the synapses, which are junction parts of the neuron http://ajpatelit.hpage.com

  35. Neural Networks Neural Networks A multilayer feed-forward neural network. http://ajpatelit.hpage.com

  36. Prediction • (Numerical) prediction is similar to classification • construct a model • use model to predict continuous or ordered value for a given input • Prediction is different from classification • Classification refers to predict categorical class label • Prediction models continuous-valued functions http://ajpatelit.hpage.com

  37. Prediction • Major method for prediction: regression • Model the relationship between one or more independent or predictor variables and a dependent or response variable • Regression analysis • Linear and multiple regression • Non-linear regression • Other regression methods: generalized linear model, Poisson regression, log-linear models, regression trees http://ajpatelit.hpage.com

  38. Linear Regression • Linear regression: involves a response variable y and a single predictor variable x y = w0 + w1 x where w0 (y-intercept) and w1 (slope) are regression coefficients • Method of least squares: estimates the best-fitting straight line http://ajpatelit.hpage.com

  39. Linear Regression • Multiple linear regression: involves more than one predictor variable • Training data is of the form (X1, y1), (X2, y2),…, (X|D|, y|D|) • Ex. For 2-D data, we may have: y = w0 + w1 x1+ w2 x2 • Many nonlinear functions can be transformed into the above http://ajpatelit.hpage.com

  40. Nonlinear Regression • Some nonlinear models can be modeled by a polynomial function • A polynomial regression model can be transformed into linear regression model. For example, y = w0 + w1 x + w2 x2 + w3 x3 convertible to linear with new variables: x2 = x2, x3= x3 y = w0 + w1 x + w2 x2 + w3 x3 • Other functions, such as power function, can also be transformed to linear model • Some models are intractable nonlinear (e.g., sum of exponential terms) • possible to obtain least square estimates through extensive calculation on more complex formulae http://ajpatelit.hpage.com

  41. Information Gain • ID3 uses information gain as its attribute selection measure. • Let node N represent or hold the tuples of partition D. The attribute with the highest information gain is chosen as the splitting attribute for node N. • This attribute minimizes the information needed to classify the tuples in the resulting partitions and reflects the least randomness or “impurity” in these partitions. http://ajpatelit.hpage.com

  42. Information Gain (cont…) • InfoA(D) is the expected information required to classify a tuple from D based on the partitioning by A. • Information gain is defined as the difference between the original information requirement (i.e., based on just the proportion of classes) and the new requirement (i.e., obtained after partitioning on A). http://ajpatelit.hpage.com

  43. Information Gain (cont…) Examples Over all Information for a database http://ajpatelit.hpage.com

  44. Information Gain (cont…) Hence, the gain in information from such a partitioning would be http://ajpatelit.hpage.com

  45. Gain Ratio • Consider an attribute that acts as a unique identifier, such as product_ID. A split on product_IDwould result in a large number of partitions (as many as there are values), each one containing just one tuple. • Because each partition is pure, the information required to classify data set D based on this partitioning would be Infoproduct_ID(D) = 0. • Therefore, the information gained by partitioning on this attribute is maximal. Clearly, such a partitioning is useless for classification. http://ajpatelit.hpage.com

  46. Gain Ratio (cont…) This value represents the potential information generated by splitting the training data set, D, into v partitions, corresponding to the v outcomes of a test on attribute A. Note that, for each outcome, it considers the number of tuples having that outcome with respect to the total number of tuples in D. It differs from information gain, which measures the information with respect to classification that is acquired based on the same partitioning. The gain ratio is defined as http://ajpatelit.hpage.com

  47. Cluster Analysis • Cluster: a collection of data objects • Similar to one another within the same cluster • Dissimilar to the objects in other clusters • Cluster analysis • Finding similarities between data according to the characteristics found in the data and grouping similar data objects into clusters • Unsupervised learning: no predefined classes • Typical applications • As a stand-alone tool to get insight into data distribution • As a preprocessing step for other algorithms http://ajpatelit.hpage.com

  48. Clustering: Rich Application • Pattern Recognition • Spatial Data Analysis • Create thematic maps in GIS by clustering feature spaces • Detect spatial clusters or for other spatial mining tasks • Image Processing • Economic Science (especially market research) • WWW • Document classification • Cluster Weblog data to discover groups of similar access patterns http://ajpatelit.hpage.com

  49. Requirements of Clustering in DM • Scalability • Ability to deal with different types of attributes • Ability to handle dynamic data • Discovery of clusters with arbitrary shape • Minimal requirements for domain knowledge to determine input parameters • Able to deal with noise and outliers • Insensitive to order of input records • High dimensionality • Incorporation of user-specified constraints • Interpretability and usability http://ajpatelit.hpage.com

  50. End of the Unit-5 http://ajpatelit.hpage.com

More Related