Practices of business intelligence
Download
1 / 139

商業智慧實務 Practices of Business Intelligence - PowerPoint PPT Presentation


  • 167 Views
  • Uploaded on

Tamkang University. 商業智慧的資料探勘 (Data Mining for Business Intelligence). 商業智慧實務 Practices of Business Intelligence. 1022BI05 MI4 Wed, 9,10 (16:10-18:00) (B113). Min-Yuh Day 戴敏育 Assistant Professor 專任助理教授 Dept. of Information Management , Tamkang University 淡江大學 資訊管理學系

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' 商業智慧實務 Practices of Business Intelligence' - noah-jordan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Practices of business intelligence

Tamkang University

商業智慧的資料探勘 (Data Mining for Business Intelligence)

商業智慧實務Practices of Business Intelligence

1022BI05

MI4

Wed, 9,10 (16:10-18:00) (B113)

Min-Yuh Day

戴敏育

Assistant Professor

專任助理教授

Dept. of Information Management, Tamkang University

淡江大學資訊管理學系

http://mail. tku.edu.tw/myday/

2014-03-19


課程大綱 (Syllabus)

週次 (Week) 日期 (Date) 內容 (Subject/Topics)

1 103/02/19 商業智慧導論 (Introduction to Business Intelligence)

2 103/02/26 管理決策支援系統與商業智慧 (Management Decision Support System and Business Intelligence)

3 103/03/05 企業績效管理 (Business Performance Management)

4 103/03/12 資料倉儲 (Data Warehousing)

5 103/03/19 商業智慧的資料探勘 (Data Mining for Business Intelligence)

6 103/03/26 商業智慧的資料探勘 (Data Mining for Business Intelligence)

7 103/04/02 教學行政觀摩日 (Off-campus study)

8 103/04/09 資料科學與巨量資料分析 (Data Science and Big Data Analytics)


課程大綱 (Syllabus)

週次 日期 內容(Subject/Topics)

9 103/04/16 期中報告 (Midterm Project Presentation)

10 103/04/23 期中考試週 (Midterm Exam)

11 103/04/30 文字探勘與網路探勘 (Text and Web Mining)

12 103/05/07 意見探勘與情感分析 (Opinion Mining and Sentiment Analysis)

13 103/05/14 社會網路分析 (Social Network Analysis)

14 103/05/21 期末報告 (Final Project Presentation)

15 103/05/28 畢業考試週 (Final Exam)


Data mining at the intersection of many disciplines
Data Mining at the Intersection of Many Disciplines

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


A taxonomy for data mining tasks
A Taxonomy for Data Mining Tasks

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Data mining software
Data Mining Software

Source: KDNuggets.com, May 2009

  • Commercial

    • SPSS - PASW (formerly Clementine)

    • SAS - Enterprise Miner

    • IBM - Intelligent Miner

    • StatSoft – Statistical Data Miner

    • … many more

  • Free and/or Open Source

    • Weka

    • RapidMiner…

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Data mining process
Data Mining Process

  • A manifestation of best practices

  • A systematic way to conduct DM projects

  • Different groups has different versions

  • Most common standard processes:

    • CRISP-DM (Cross-Industry Standard Process for Data Mining)

    • SEMMA (Sample, Explore, Modify, Model, and Assess)

    • KDD (Knowledge Discovery in Databases)

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Data mining process crisp dm
Data Mining Process: CRISP-DM

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Data mining process crisp dm1
Data Mining Process: CRISP-DM

Accounts for ~85% of total project time

Step 1: Business Understanding

Step 2: Data Understanding

Step 3: Data Preparation (!)

Step 4: Model Building

Step 5: Testing and Evaluation

Step 6: Deployment

The process is highly repetitive and experimental (DM: art versus science?)

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Data preparation a critical dm task
Data Preparation – A Critical DM Task

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Data mining process semma
Data Mining Process: SEMMA

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Cluster analysis
Cluster Analysis

Used for automatic identification of natural groupings of things

Part of the machine-learning family

Employ unsupervised learning

Learns the clusters of things from past data, then assigns new instances

There is not an output variable

Also known as segmentation

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Cluster analysis1
Cluster Analysis

Clustering of a set of objects based on the k-means method. (The mean of each cluster is marked by a “+”.)

Source: Han & Kamber (2006)


Cluster analysis2
Cluster Analysis

  • Clustering results may be used to

    • Identify natural groupings of customers

    • Identify rules for assigning new cases to classes for targeting/diagnostic purposes

    • Provide characterization, definition, labeling of populations

    • Decrease the size and complexity of problems for other data mining methods

    • Identify outliers in a specific domain (e.g., rare-event detection)

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems



Cluster analysis for data mining
Cluster Analysis for Data Mining

  • How many clusters?

    • There is not a “truly optimal” way to calculate it

    • Heuristics are often used

      • Look at the sparseness of clusters

      • Number of clusters = (n/2)1/2(n: no of data points)

      • Use Akaike information criterion (AIC)

      • Use Bayesian information criterion (BIC)

  • Most cluster analysis methods involve the use of a distance measure to calculate the closeness between pairs of items

    • Euclidian versus Manhattan (rectilinear) distance

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


K means clustering algorithm
k-Means Clustering Algorithm

k : pre-determined number of clusters

Algorithm (Step 0: determine value of k)

Step 1: Randomly generate k random points as initial cluster centers

Step 2: Assign each point to the nearest cluster center

Step 3: Re-compute the new cluster centers

Repetition step: Repeat steps 2 and 3 until some convergence criterion is met (usually that the assignment of points to clusters becomes stable)

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Cluster analysis for data mining k means clustering algorithm
Cluster Analysis for Data Mining - k-Means Clustering Algorithm

Source: Turban et al. (2011), Decision Support and Business Intelligence Systems


Similarity and dissimilarity between objects
Similarity and Dissimilarity Between Objects

  • Distances are normally used to measure the similarity or dissimilarity between two data objects

  • Some popular ones include: Minkowski distance:

    where i = (xi1, xi2, …, xip) and j = (xj1, xj2, …, xjp) are two p-dimensional data objects, and q is a positive integer

  • If q = 1, d is Manhattan distance

Source: Han & Kamber (2006)


Similarity and dissimilarity between objects cont
Similarity and Dissimilarity Between Objects (Cont.)

  • If q = 2, d is Euclidean distance:

    • Properties

      • d(i,j) 0

      • d(i,i)= 0

      • d(i,j)= d(j,i)

      • d(i,j) d(i,k)+ d(k,j)

  • Also, one can use weighted distance, parametric Pearson product moment correlation, or other disimilarity measures

Source: Han & Kamber (2006)


Euclidean distance vs manhattan distance
Euclidean distance vs Manhattan distance

Euclidean distance:

= ((3-1)2 + (5-2)2 )1/2= (22 + 32)1/2

= (4+ 9)1/2

= (13)1/2

= 3.61

x2 (3, 5)

5

4

3.61

3

3

2

2

x1 = (1, 2)

Manhattan distance:

= (3-1) + (5-2)

= 2 + 3

= 5

1

1

2

3

Distance of two point x1 = (1, 2) and x2 (3, 5)


The k means clustering method

10

9

8

7

6

5

4

3

2

1

0

0

1

2

3

4

5

6

7

8

9

10

The K-Means Clustering Method

  • Example

10

9

8

7

6

5

Update the cluster means

Assign each objects to most similar center

4

3

2

1

0

0

1

2

3

4

5

6

7

8

9

10

reassign

reassign

K=2

Arbitrarily choose K object as initial cluster center

Update the cluster means

Source: Han & Kamber (2006)


K-Means ClusteringStep by Step


K-Means Clustering

Step 1: K=2, Arbitrarily choose K object as initial cluster center

M2 = (8, 5)

m1 = (3, 4)


Step 2: Compute seed points as the centroids of the clusters of the current partition

Step 3: Assign each objects to most similar center

M2 = (8, 5)

m1 = (3, 4)

K-Means Clustering


Step 2: Compute seed points as the centroids of the clusters of the current partition

Step 3: Assign each objects to most similar center

M2 = (8, 5)

Euclidean distance b(3,6) m2(8,5)

= ((8-3)2 + (5-6)2 )1/2= (52 + (-1)2)1/2

= (25+ 1)1/2

= (26)1/2

= 5.10

m1 = (3, 4)

Euclidean distance b(3,6) m1(3,4)

= ((3-3)2 + (4-6)2 )1/2= (02 + (-2)2)1/2

= (0+ 4)1/2

= (4)1/2

= 2.00

K-Means Clustering


Step 4: Update the cluster means, of the current partition Repeat Step 2, 3, stop when no more new assignment

m1 = (3.86, 5.14)

M2 = (7.33, 4.33)

K-Means Clustering


Step 4: Update the cluster means, of the current partition Repeat Step 2, 3,stop when no more new assignment

m1 = (3.67, 5.83)

M2 = (6.75., 3.50)

K-Means Clustering


stop when no more new assignment of the current partition

K-Means Clustering


K means clustering k 2 two clusters
K-Means of the current partition Clustering (K=2, two clusters)

stop when no more new assignment

K-Means Clustering


Sas em case study 1 cluster analysis k means using sas em banking segmentation
個案分析與實作一 of the current partition(SAS EM 分群分析):Case Study 1 (Cluster Analysis – K-Means using SAS EM)Banking Segmentation


行銷客戶分群 of the current partition


案例情境 of the current partition

  • ABC銀行的行銷部門想要針對該銀行客戶的使用行為,進行分群分析,以了解現行客戶對本行的往來方式,並進一步提供適宜的行銷接觸模式。

  • 該銀行從有效戶(近三個月有交易者),取出10萬筆樣本資料。依下列四種交易管道計算交易次數:

    • 傳統臨櫃交易(TBM)

    • 自動櫃員機交易(ATM)

    • 銀行專員服務(POS)

    • 電話客服(CSC)

Source: SAS Enterprise Miner Course Notes, 2014, SAS


資料欄位說明 of the current partition

資料集名稱: profile.sas7bdat

Source: SAS Enterprise Miner Course Notes, 2014, SAS


行銷客戶分群實機演練 of the current partition

演練重點:

• 極端值資料處理

• 分群變數選擇

• 衍生變數產出

• 分群參數調整與分群結果解釋

分析目的

依據各往來交易管道TBM、ATM、POS、CSC進行客戶分群分析。

Source: SAS Enterprise Miner Course Notes, 2014, SAS


Sas enterprise miner sas em case study
SAS Enterprise Miner (SAS EM) Case Study of the current partition

  • SAS EM 資料匯入4步驟

    • Step 1. 新增專案 (New Project)

    • Step 2. 新增資料館 (New / Library)

    • Step 3. 建立資料來源 (Create Data Source)

    • Step 4. 建立流程圖 (Create Diagram)

  • SAS EM SEMMA 建模流程


Download EM_Data.zip (SAS EM Datasets) of the current partition

http://mail.tku.edu.tw/myday/teaching/1022/DM/Data/EM_Data.zip


Upzip em data zip to c data em data
Upzip EM_Data.zip to C:\DATA\EM_Data of the current partition


Upzip em data zip to c data em data1
Upzip EM_Data.zip to C:\DATA\EM_Data of the current partition


Vmware horizon view client softcloud tku edu tw sas enterprise miner
VMware Horizon View Client of the current partitionsoftcloud.tku.edu.twSAS Enterprise Miner


Sas locale setup manager english ui
SAS Locale Setup Manager of the current partition English UI


Sas enterprise guide 5 1 sas eg
SAS Enterprise Guide 5.1 (SAS EG) of the current partition


Sas enterprise miner 12 1 sas em
SAS Enterprise Miner 12.1 (SAS EM) of the current partition


Sas locale setup manager 3 1
SAS Locale Setup Manager 3.1 of the current partition


Source: http:// of the current partitionwww.sasresource.com/faq11.html


C:\Program Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Sas enterprise miner 12 1 sas em1
SAS Enterprise Miner 12.1 (SAS EM) Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Sas enterprise miner sas em
SAS Enterprise Miner (SAS EM) Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe

SAS Enterprise Miner (SAS EM)

English UI


SAS Enterprise Guide 5.1 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe(SAS EG)Open SAS .sas7bdat FileExport to Excel .xlsx FileImport Excel .xlsx FileExport to SAS .sas7bdat File


Sas enterprise guide 5 1 sas eg1
SAS Enterprise Guide 5.1 (SAS EG) Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


New project
New Project Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Open sas data file profile sas7bdat
Open SAS Data File: Profile.sas7bdat Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Export sas sas7bdat to to excel xlsx file
Export SAS Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe.sas7bdat to to Excel .xlsx File


Import excel file to sas eg
Import Excel File to SAS EG Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Import excel file to sas eg1
Import Excel File to SAS EG Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Export excel xlsx file to sas sas7bdat file
Export Excel Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe.xlsx File to SAS .sas7bdat File


Profile excel xlsx
Profile_Excel.xlsx Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Profile sas sas7bdat
Profile_SAS.sas7bdat Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Sas enterprise miner 12 1 sas em2
SAS Enterprise Miner 12.1 (SAS EM) Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Sas em 4
SAS EM Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe資料匯入4步驟

Step 1. 新增專案 (New Project)

Step 2. 新增資料館 (New / Library)

Step 3. 建立資料來源 (Create Data Source)

Step 4. 建立流程圖 (Create Diagram)


Step 1 new project
Step 1. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增專案 (New Project)


Step 1 new project1
Step 1. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增專案 (New Project)


Step 1 new project2
Step 1. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增專案 (New Project)


Sas enterprise miner em project1
SAS Enterprise Miner (EM_Project1) Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Step 2 new library
Step 2. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增資料館(New / Library)


Step 2 new library1
Step 2. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增資料館(New / Library)


Step 2 new library2
Step 2. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增資料館(New / Library)


Step 2 new library3
Step 2. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增資料館(New / Library)


Step 2 new library4
Step 2. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe新增資料館(New / Library)


Step 3 create data source
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source1
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source2
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source3
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source4
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source5
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)

EM_LIB.PROFILE


Step 3 create data source6
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source7
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source8
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source9
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source10
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source11
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source12
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source13
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 3 create data source14
Step 3. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立資料來源(Create Data Source)


Step 4 create diagram
Step 4. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立流程圖(Create Diagram)


Step 4 create diagram1
Step 4. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立流程圖(Create Diagram)


Step 4 create diagram2
Step 4. Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe建立流程圖(Create Diagram)


Sas enterprise miner sas em case study1
SAS Enterprise Miner (SAS EM) Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exeCase Study

  • SAS EM 資料匯入4步驟

    • Step 1. 新增專案 (New Project)

    • Step 2. 新增資料館 (New / Library)

    • Step 3. 建立資料來源 (Create Data Source)

    • Step 4. 建立流程圖 (Create Diagram)

  • SAS EM SEMMA 建模流程


案例情境模型流程 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Em lib profile
EM_Lib.Profile Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


篩選-間隔變數 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe …


121 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


篩選-匯入的資料 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe … [勘查]


篩選-匯入的資料 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


篩選-匯出的資料 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe … [勘查]


篩選-匯出的資料 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe


Explore cluster
勘查 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Explore)-群集 (Cluster)


Cluster
群集 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Cluster) 結果


Assess segment profile
評估 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Assess)-區段特徵描繪(Segment Profile)

區段特徵描繪 (Segment Profile)


Assess segment profile1
評估 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Assess)-區段特徵描繪(Segment Profile)


Assess segment profile2
評估 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Assess)-區段特徵描繪(Segment Profile)


Assess segment profile3
評估 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Assess)-區段特徵描繪(Segment Profile)


Assess segment profile4
評估 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Assess)-區段特徵描繪(Segment Profile)


Segment profile
區段特徵描繪 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Segment Profile)


Segment profile1
區段特徵描繪 Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe (Segment Profile)


References
References Files\SASHome\SASEnterpriseMinerWorkstationConfiguration\12.1\em.exe

  • Efraim Turban, Ramesh Sharda, Dursun Delen, Decision Support and Business Intelligence Systems, Ninth Edition, 2011, Pearson.

  • Jiawei Han and Micheline Kamber, Data Mining: Concepts and Techniques, Second Edition, 2006, Elsevier

    • Jim Georges, Jeff Thompson and Chip Wells, Applied Analytics Using SAS Enterprise Miner, SAS, 2010

  • SAS Enterprise Miner Course Notes, 2014, SAS

  • SAS Enterprise Miner Training Course, 2014, SAS

  • SAS Enterprise Guide Training Course, 2014, SAS


ad