1 / 22

Part II

Part II. Ch 5. Knowledge Discovery in Databases Ch 6. The Data Warehouse Ch 7. Formal Evaluation Technique. Tools for Knowledge Discovery. Knowledge Discovery in Databases. Chapter 5. 5.1 A KDD Process Model. 1. Goal identification 2. Creating an initial target data set

zanna
Download Presentation

Part II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Part II Ch 5. Knowledge Discovery in Databases Ch 6. The Data Warehouse Ch 7. Formal Evaluation Technique Tools for Knowledge Discovery

  2. Knowledge Discovery in Databases Chapter 5

  3. 5.1 A KDD Process Model • 1. Goal identification • 2. Creating an initial target data set • Data preprocessing • Data transformation • Data mining • Interpretation and evaluation • Taking action

  4. Figure 5.1 A seven-step KDD process model

  5. Figure 5.2 Applyiing the scientific method to data mining

  6. Step 1: Goal Identification Define the Problem. Choose a Data Mining Tool. Estimate Project Cost. Estimate Project Completion Time. Address Legal Issues. Develop a Maintenance Plan.

  7. Step 2: Creating a Target Dataset (1) Flat file or Spread Sheet format (2) Relational Database Collection of tables (rows and columns) RDB  Reduce data redundancy (Decomposition) DM  Uncover the inherent redundancy in data (Join is required) (3) (1) + (2)  Data transformation is required. (4) Data Warehouse : Historical database designed specifically for decision support.

  8. Figure 5.3 The Acme credit card database

  9. Noisy Data Step 3: Data Preprocessing Missing Data Locate Duplicate Records. Locate Incorrect Attribute Values. Outliers • Discard records with missing values. • Replace missing real-valued items with the class mean. • Replace missing values with values found within highly • similar instances.

  10. Step 4: Data Transformation Data Normalization (2) Data Type Conversion (3) Attribute and Instance Selection

  11. (1) Data Normalization Decimal Scaling Min-Max Normalization Normalization using Z-scores Logarithmic Normalization (2) Data Conversion Categorical  Numeric equivalent

  12. (3) Attribute and Instance Selection a. Eliminating Attributes 1. Highly correlated 2. High domain predictability 3. Low attribute significance score

  13. Creating Attributes • - Combining attributes e.g. P/E ratio • - Differences between the attributes • - Percent increase or decrease • Instance Selection • - Use Instance Typicality • Supervised : Use Highly and moderately typical • training instance • Unsupervised : Eliminate most atypical instance •  well defined cluster

  14. Step 5: Data Mining Choose training and test data. Designate a set of input attributes. If learning is supervised, choose one or more output attributes. Select learning parameter values. Invoke the data mining tool.

  15. Step 6: Interpretation and Evaluation Statistical analysis. Heuristic analysis. Experimental analysis. Human analysis.

  16. Step 7: Taking Action Create a report. Relocate retail items. Mail promotional information. Detect fraud. Fund new research.

  17. 5.9 The Crisp-DM Process Model(Cross Industry Standard Process for Data Mining) Business understanding Data understanding Data preparation Modeling Evaluation Deployment

  18. 5.10 Experimenting with ESX A Four-Step Model for Knowledge Discovery • Identify the goal. • Prepare the data. • Apply data mining. • Interpret and evaluate the results.

  19. Experiment 1: Attribute Evaluation Use unsupervised clustering to see how well the set of input attributes are able to define the classes - Domain Summary: eight, eleven/ nine, ten, twelve - Class Summary: /nine, ten Class 1 : Accept 84% Class 2 : Reject 90% Repeat DM process to create a best data model *Applying the Four-Step Process Model to the Credit Screening Dataset*

  20. Experiment 2: Parameter Evaluation *Applying the Four-Step Process Model to the Satellite Image Dataset*

  21. Figure 5.4 Satellite image data

More Related