1 / 31

The IBM Netezza Appliance: Revolutionizing Analytics

DataWarhouse rješenje za banke - IBM Netezza Leader in Data Warehouse Appliances Robert Božič robert.bozic@si.ibm.com. The IBM Netezza Appliance: Revolutionizing Analytics. What is Netezza?. completely transforming the user experience. Appliances make it simple,. Dedicated device

cachez
Download Presentation

The IBM Netezza Appliance: Revolutionizing Analytics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DataWarhouse rješenje za banke - IBM NetezzaLeader in Data Warehouse AppliancesRobert Božičrobert.bozic@si.ibm.com

  2. The IBM Netezza Appliance: Revolutionizing Analytics What is Netezza?

  3. completely transforming the user experience. Appliances make it simple, • Dedicated device • Optimized for purpose • Complete solution • Fast installation • Very easy operation • Standard interfaces • Low cost 3

  4. The IBM Netezza Appliance: Revolutionizing Analytics • Purpose-built analytics engine • Integrated database, server & storage • Standard interfaces • Low total cost of ownership • Speed: 10-100x faster than traditional systems • Simplicity: Minimal administration • Scalability: Peta-scale user data capacity • Smart: High-performance advanced analytics

  5. Some of the customers • Zavarovalnica Maribor • Zavarovalnca Triglav • Telekom Slovenije • Tuš Mobil • Petrol • Informatika • NLB • MKZ • Fabrika Duvana Sarajevo

  6. Digital Media Financial Services Government Health & Life Sciences Retail / Consumer Products Telecom Other Page 6

  7. Business case – financial institutionEvolution of DWH environment • Year 2000 – first “real” data warehouse • 1 power user, 20 common users, 1 IT developer • Years 2001 – 2005 • Number of users increased – 3 IT professionals, 100 common users, 10 power users • Enlargment of the data warehouse by factor 3 • DWH/BI becomes invloved in all crucial processes • Years 2005 – 2007 • Enlargment of the data warehouse by 100 • Increasing the number of common users to 250 • By end of 2007 DWH/BI declared as a key business process at ZM

  8. Users expectations • High demand for information from the DWH system – the users expect that the system contains everything • “Immediate” responsiveness • Daily or even shorter frequencies for data loading • Real time BI • Sophisticated iterative analysis: marketing analysis, product or customer profitability analysis etc. • Quick ad-hoc reporting for various purposes

  9. Main issues with existing DWH • More and more administration: database servers, application servers, BI tools, BI applications... • Higher and higher ownership costs (HW; SW licences...) • Smaller and smaller time window for ETL • More and more end users • Continously increased complexity of the reports and analysis - killer queries which can only be run in the evening hours , “freeze” the server, even on DWH server.

  10. Main challenges with building fully mature DWH/BI infrastructure • Cost boost • Entering in the upper cost-range of DWH servers • Unfavorable licensing policy for database SW (number of users, CPU licensees) • Complex administration of database servers • Required 1 - 4 employees to manage DWH

  11. Benefits • Considered by the management as one of the best IT investments in the last 10 years • The same DWH/BI team is capable to manage even larger BI infrastructure. • Vast simplification of many complex queries and batch jobs • Development of new BIsolutions • Users completely changed the way of thinking what they can get out of DWH

  12. Smart Predicts what shoppers are likely to buy in future visits Coupon redemption rates as high as 25% “Because of (Netezza’s) in-database technology, we believe we'll be able to do 600 predictive models per year (10X as many as before) with the same staff." Eric Williams, CIO and executive VP

  13. Appliance Simplicity

  14. Managing The Netezza Appliance No software installation No storage administration No database tuning Less DBA drudgery, More applications

  15. The Netezza Appliance – Loading Data Integration • Ab Initio • Business Objects/SAP • Composite Software • Expressor Software • GoldenGate Software (Oracle) • Informatica • IBM Information Server • Sunopsis (Oracle) • WisdomForce Data In SQL ODBC JDBC OLE-DB

  16. The Netezza Appliance – Querying Reporting & Analysis • Actuate • Business Objects/SAP • Cognos (IBM) • Information Builders • Kalido • KXEN • MicroStrategy • Oracle OBIEE • QlikTech • Quest Software • SAS • SPSS (IBM) • Unica (IBM) Data Out SQL ODBC JDBC OLE-DB

  17. Simple to Deploy and Operate • Operations • Simply load and go .… it’s an appliance • Installation to Business Value in ~2 days • Ease of Evaluation and Perform As Advertised • BI Developers • Data model agnostic • No configuration or physical modeling • No indexes or tuning – out of the box performance • Focus on business value, not physical design • ETL Developers • Faster load and transformation times • No aggregate tables needed – simpler ETL logic • In-database transformation – ‘ELT’ • Business Analysts • On-Stream processing by 100’s of nodes • Train of thought analysis – 10 to 100x faster • True ad hoc queries • Lower latency – load & query simultaneously 17

  18. Appliance Architecture

  19. IBM Netezza True Appliance Architecture Client Database Server Storage DATA SQL ETL Server DBA CLI Source Systems I/O I/O SOLARIS AIX CACHE CACHE CACHE 3rd PartyApps HP-UX TRU64 LINUX WINDOWS High Performance Loader SQL Data

  20. IBM Netezza True Appliance Architecture Client Server Database Storage ODBC 3.X JDBC Type 4 SQL-92 SQL-99 Analytics ETL Server DBA CLI Source Systems I/O I/O SOLARIS AIX CACHE CACHE CACHE 3rd PartyApps HP-UX TRU64 LINUX WINDOWS High Performance Loader Database, Server, Storage - in one

  21. IBM Netezza True Appliance Architecture Optimized Hardware+Software Purpose-built for high performance analytics; requires no tuning Streaming Data Hardware-based query acceleration for blistering fast results True MPP All processors fully utilized for maximum speed and efficiency Deep Analytics Complex analytics executed in-database for deeper insights 21

  22. IBM Netezza True Appliance Massively Parallel Processing ODBC 3.X JDBC Type 4 OLE-DB SQL/92 Client ETL Server DBA CLI Source Systems SOLARIS AIX 3rd PartyApps HP-UX TRU64 LINUX WINDOWS High Performance Loader S-Blade 1 Processor & streaming DB logic SQL Compiler Query Plan Optimize Admin 2 S-Blade Execution Engine Processor & streaming DB logic S-Blade 3 Processor & streaming DB logic Ÿ Ÿ Ÿ High-PerformanceDatabase Engine Streaming joins, aggregations, sorts High-Speed Loader/Unloader 960 S-Blade DBOS Front End Processor & streaming DB logic Massively Parallel Intelligent Storage Network Fabric SMP Host

  23. IBM Netezza True Appliance Massively Parallel Processing™ Client ETL Server DBA CLI Source Systems SOLARIS AIX 3rd PartyApps HP-UX TRU64 LINUX WINDOWS High Performance Loader S-Blade 1 Snippets Processor & streaming DB logic SQL Compiler Query Plan Optimize Admin 3 3 2 2 2 3 2 3 2 3 1 SQL SQL 2 S-Blade Execution Engine Processor & streaming DB logic S-Blade 3 Processor & streaming DB logic Ÿ Ÿ Ÿ High-PerformanceDatabase Engine Streaming joins, aggregations, sorts High-Speed Loader/Unloader 960 S-Blade DBOS Front End Processor & streaming DB logic Massively Parallel Intelligent Storage Network Fabric SMP Host

  24. IBM Netezza True Appliance Massively Parallel Processing™ Client ETL Server DBA CLI Source Systems SOLARIS AIX 3rd PartyApps HP-UX TRU64 LINUX WINDOWS High Performance Loader S-Blade 1 Consolidate Processor & streaming DB logic SQL Compiler Query Plan Optimize Admin 3 3 2 2 2 3 2 3 2 S-Blade Execution Engine Processor & streaming DB logic S-Blade 3 Processor & streaming DB logic Ÿ Ÿ Ÿ High-PerformanceDatabase Engine Streaming joins, aggregations, sorts High-Speed Loader/Unloader 960 S-Blade DBOS Front End Processor & streaming DB logic Massively Parallel Intelligent Storage Network Fabric SMP Host

  25. Our Secret Sauce select DISTRICT, PRODUCTGRP, sum(NRX) from MTHLY_RX_TERR_DATA where MONTH = '20091201' and MARKET = 509123 and SPECIALTY = 'GASTRO' FPGA Core CPU Core Restrict, Visibility Complex ∑ Joins, Aggs, etc. Project Uncompress Slice of table MTHLY_RX_TERR_DATA (compressed) sum(NRX) select DISTRICT, PRODUCTGRP, sum(NRX) where MONTH = '20091201' and MARKET = 509123 and SPECIALTY = 'GASTRO' 

  26. Appliance family for data life-cycle management

  27. Advanced Analytics  

  28. Advanced Analytics the Netezza Way SPSS • complex analytics • SAS, SPSS, R, Java, etc • implicit parallelism • petabyte scalability • appliance simplicity Demand Forecasting SQL Fraud Detection R, S+ SQL

  29. Pre-Built In-Database Analytics Statistics Transformations Time Series Mathematical • Basic Math* • Permutation and Combination* • Greatest Common Divisor and Least Common Multiple* • Conversion of Values* • Exponential and Logarithm* • Gamma and Beta Functions • Matrix Algebra+ • Area Under Curve* • Interpolation Methods* • Descriptive Statistics+ • Distance Measures* • Hypothesis Testing* • Chi-Square & Contingency Tables* • Univariate & Multivariate Distributions+ • Monte Carlo Simulation* • Data Profiling / Descriptive Statistics+ • General Diagnostics • Statistics+ • Sampling • Data prep • Autoregressive+ • Forecasting* Data Mining Predictive Geospatial * Fuzzy Logix DB Lytix capabilities + Netezza Analytics and Fuzzy Logix DB Lytix capabilities • Association Rules+ • Clustering+ • Feature Extraction+ • Discriminant Analysis* • Linear Regression+ • Logistic Regression+ • Classification • Bayesian • Sampling • Model Testing • Geospatial Data Type • Geometric Functions • Geometric Analysis Go to 'View > Header and Footer' to change this footer text to the event title Go to 'View > Header and Footer' to change this footer text to the event title

  30. IBM Netezza: bold claims, backed up

  31. Bold Claims, but...We Prove Them! • We prove we are simpler • We prove we deliver performance • We prove we work within your environment • We prove we integrate with your 3rd party tools • We prove we are “easy to do business with” • We prove we have the lowest TCO • We prove business value

More Related