1 / 24

Case Study –Data Warehouse 遠傳電信 CDR DW POC

Case Study –Data Warehouse 遠傳電信 CDR DW POC. Situation overview Customer currently use Teradata with high maintain cost fee Auto process complex procedures Competing against Oracle Strategy & solution

evelyn
Download Presentation

Case Study –Data Warehouse 遠傳電信 CDR DW POC

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Case Study–Data Warehouse遠傳電信 CDR DW POC • Situation overview • Customer currently use Teradata with high maintain cost fee • Auto process complex procedures • Competing against Oracle • Strategy & solution • Provide a stable, scalable and reliable Operational Data Store(ODS) solution that can share CDR(Call Detail Records) space occupation and heavy processes in Teradata EDW system, in order to enhance the performance of FET EDW

  2. Solution Design Consideration– Import/Export Strategy Fact Table Data Mart - MS SQL 2008 Bulk Copy Cubes Voice CDR SMS CDR GPRS CDR WAP CDR SSAS Cubes PSTAGE CDR Summary Bulk Insert Bulk Export Fast Load Loading PDATA Trans-form Bulk Insert SSIS PMART Subscriber Profile CDR ODS - MS SQL 2008 Teradata Fast Export

  3. Our Solution Architecture 750GB 874 Text files SQL 2008 New Features in this POC Teradata 12 nodes Capacity :11TB Page Compression Improved Query Performance Multi-thread Bulk Insert CDRs ADO .net Provider Data Partitioning SQL Query Analyzer Current Data Volume: ~9TB(FET+KGT) HP DL580 + EMC CX3-20 SQL 2008

  4. ODS POC Final Result

  5. SAN Storage • 儲存設備部分,微軟建議在遠傳現有的EMC CX3-80 SAN的擴充,新增磁碟增加所需儲存空間;IT人員不須學習新的維運管理技能就可有效管理本系統。 • 以下為所建議之SAN組成之方式,以資料使用狀況分開存放於不同的RAID群組中。

  6. Solution Design Consideration- High Availability • 兩台資料庫伺服器分別處理不同工作,預估除了在處理EOD/EOM等尖峰時間外,平時的CPU使用率並不會超過40% • 規劃兩台SQL 資料庫伺服器將組成Active-Active mode Fail-over Cluster ,互為備援;當一台伺服器故障後,另一台伺服器將可接手所有的工作負載

  7. Case Study –高效能警政署150億筆通聯資料 4~5seconds search • Situation overview • CIB was pure Oracle database environment • Customer needs to deal with 15 billoins records • Oracle ran insert & index building 15 hours • Strategy & solution • Partner came to MTC for a POC support • POC result in preparing time for 6 hours and search for 5 seconds in 15 billions record to exceed CIB’s expectation. • Also provide best backup and synchronized VLDB solution in different options for CIB • Microsoft provides enterprise-class, mission-critical systems and support

  8. 警政署通聯分析平台架構 Distribution Link Architecture SSIS(Bulk Insert ) + Create 5 Indexes SSIS(Bulk Insert ) + Create 5 Indexes Linked servers Nod1 Node2 QM

  9. Testing Result- Size / Records

  10. Testing Result- Demo

  11. 資農業產銷資訊整合平台 品項範圍選定原則: (依本年度時程考量) • 廣度 • 規劃以農糧署為資料提供單位,以農糧署、農委會統計室與企劃處 經濟研究科為資料使用單位。 • 本年度訪談計畫將安排統計室及農糧署各三場、經濟研究科二場, • 訪談內容包括決定重要農產品項目、監控基準、資料來源、資料清理原則、 • 資料採礦模式、展現方式等。 • 深度 • 以時間、地區、產量及價格為主要分析項目 • 作業功能: • 1.最新新聞閱覽 • 2.文件管理 • 3.量化指標 11

  12. 資農業產銷資訊整合平台 IBM HS22 12

  13. 農委會BI專案範圍 13

  14. 個人化首頁 14

  15. 農情 BI 中心 15

  16. 農產品價格分析 16

  17. 時間地區分解樹分析 17

  18. 地區時間分解樹分析 18

  19. 價格預測分析平台 19

  20. 青蔥產量價格分析 20

  21. Too large dimensions Big Distinct Count Measure Groups ... With bad partitioning Many-to-Many dimensions ...with large intermediate fact tables Parent/Child dimension ... With too many members Near Real Time ... With a constant, high throughput flow of data Partitioning ... With too many partitions The Common, non-AMO pitfalls See a pattern?

  22. Excel工具整合 時間序列預測分析 22

  23. Excel工具整合 狀況分析 (What-if假設分析) 23

  24. 異動資料擷取(CDC) • 提供資料表的歷程記錄變更資訊 • 有效監控資料狀態 • 提供具效率的資料整理方式

More Related