CDC Transformation and Delivery

1. CDC Transformation and Delivery Data at the speed of business

2. What is CDC • Change Data Capture • Capture data events in source database and move only the changes to the target • Many different ways of doing CDC • Timestamps • Triggers • API • Log-based

3. What fuels the IBM CDC Roadmap? • The widest breadth of functionality: • Batch/pull and real-time push processing • Guaranteed delivery/transactional integrity • Multiple topologies (peer to peer, 1 to many, many to 1, uni-directional, bi-directional) • Homogeneous & heterogeneous data synchronization • Broadest range of sources and targets • Log-based capture agents for DB2 (on all platforms), Oracle, SQL Server, Sybase, IMS, VSAM, IDMS, ADABAS • Native/parallel applies for all RDBMS and JMS • Multiple data delivery protocols (TCP/IP, JMS) • Industry leading performance and scalability • End to end throughput and low latency • Parallel Apply to target system • Low impact on source database systems

4. What fuels the IBM CDC Roadmap …. • 3000+ customers using the existing CDC products for; • HA/DR (DB back-up, fault tolerance) • Real-time reporting/off-load querying • Application Co-existence (migrations, upgrades, modernization) • eCommerce (web apps, portals, data distribution) • Dynamic Data Warehousing, Master Data Management • 700+ people in engineering focused on Information Integration including 170+ focused on CDC technologies • The most comprehensive suite of data integration products • BoB transform / cleanse / discovery, metadata management, scalable performance, services enabled for SOA architectures • 5000+ customers using Information Server components

5. The IBM Solution: IBM Information ServerDelivering information you can trust IBM Information Server Unified Deployment Transform Deliver Understand Cleanse Discover, model, and govern information structure and content Standardize, merge, and correct information Combine and restructure information for new uses Capture, virtualize and move information for in-line delivery Unified Metadata Management Parallel Processing Rich Connectivity to Applications, Data, and Content

6. Provides real-time change data capture and delivery for Dynamic warehousing and real-time reporting Synchronization and replication Event detection Minimal impact on production systems High scalability and end-to-end performance Guaranteed data integrity Proven Heterogeneous support Architects Developers InfoSphere CDC Solution Deliver DataMirror Delivers real time changed data to Information Server, applications and targets or message queues Without impacting performance of production systems

7. Low Low Impact Latency LATENCY 1. Near zero latency for pervasive integration projects. 2. ETL can also deliver low latency but at what impact to Continuous Consistent Data Delivery CONSISTENT DATA DELIVERY 1. Data pushed from source, delivered in continuous stream, continuous with business operations. 2. Transaction consistency maintained to preserve units of work, referential integrity. 3. Full transaction granularity, before and after image of all transactional changes. 4. Data event aware, can be used to trigger specific business processes. 5. Fault tolerance, recover to last committed transaction. Key Value Proposition product systems and mission-critical applications. IMPACT 1. Reduces risk to operational systems. 2. Non intrusive to applications and databases. 3. Use of native DB logs, documented overhead of 2-5%. 4. No use of disk based staging or triggers. 5. Management easily integrated into existing IT operations. 6. Help reduce/manage operational windows.

8. Architecture Java-based GUI for admin & monitoring Subscriber Publisher Database ODS TCP/IP Audit JMS Journal Log Redo/Archive Logs Source Engine And Metadata Target Engine And Metadata Business Process Flat files Databases Oracle, DB2, DB2 UDB, SQL Server, Sybase, Teradata, Netezza, PointBase IMS, VSAM, IDMS, Adabas, DataCom - Classic Platforms z/OS, System i5, Red Hat and SUSE Linux, AIX, HP/UX (PA-RISC and Itanium), Solaris SPARC, Tru64 UNIX, Windows Messaging Middleware MQSeries, Sun Open Message Queue (JMS), TIBCO, BEA AquaLogic, Oracle Fusion Middleware Direct to existing ETL

9. Use Cases Customer examples

10. Native Native “CDC” “CDC” DB DB Continuous (to “Dynamic ODS”) Continuous (to “Dynamic ODS”) Log Log Each OLTP insert, update and delete operation can be stored as an insert, update and delete to maintain synchronized copy of data. All OLTP insert, update and delete operations can be stored as inserts to maintain complete transaction history. Add relevant information such as timestamp, transaction type, source system id, and id of user who changed the transaction. 1. Building A Low Latency ODS for Operational Reporting and Auditing “Solution deployed to improve visibility into lines of business for organizations with Operational BI and Data Auditing requirements” Production Server ERP Operational Data Store OLTP Manufacturing ODS Production Server Finance OLTP Manufacturing

11. Stage can be: Complementary ETL Technologies: 1. Relational Table 1. Informatica “Power Center” 2. Flat File 2. Business Objects “Data Integrator” 3. Message Queue 3. Ab Initio 4. Direct to ETL 4. IBM “DataStage” (has native integration) 2. Complementing An Existing ETL Technology “Solution deployed to improve visibility into lines of business (i.e. Dynamic Warehousing) and help manage impact concerns caused by ETL on mission critical systems” Production Server ETL Server Data Warehouse Point Of Sale Native “CDC” Continuous ETL DB Scheduled Batch OLTP Stage EDW Log Retail

12. Appliance Load API “CDC” Continuous (to Appliance) CDC Stage Flat File Supported Appliances Flat file containing transaction changes viewed as an external file to the appliance. 1. Teradata 2. Netezza Load threshold based on # of Transactions or time interval. 3. GreenPlum 4. Paraccel Once threshold reached, call appliance “load API” to bulk load transactions into appliance. 5. IBM Balanced Warehouse 3. Continuous Feed Of A Business Intelligence Appliance “Solution deployed to improve visibility into lines of business by combining the cost/performance benefits of a BI Appliance with real-time data feeds”. Production Server Appliance Nodes/Cluster Staging Server ERP Native OLTP DB Appliance Log Manufacturing

13. E S B “CDC” Continuous Native DB Log Data Event Synchronization via an Enterprise Service Bus “Solution deployed to provide real time data feeds for SOA and application integration business requirements”. “Solution deployed to provide real time data feeds for SOA and application integration business requirements”. Production Server Production Server Billing CRM “CDC” OLTP Continuous OLTP ETL Telco Telco Complimentary ESB Technologies: CDC/Replication Process 1. IBM “MQ Series” Other Technology 2. TIBCO “Business Works” 3. BEA “Aqualogic” CDC/Replication License 4. WebMethods “Fabric” A license would reside on the server that hosts the message oriented middleware.

14. Continuous (bi-directional) “CDC” “CDC” Continuous (bi-directional) Provides continuous bi-directional synchronization between web based applications and mission critical business applications. Helps organizations improve customer online shopping experience with improved visibility into inventory and customer shopping activities. 5. e-Commerce Application Synchronization “Solution deployed to provide continuous customer, sales and inventory visibility in web base e-commerce applications”. Website Orders Native OLTP DB Log Production Server Inventory Corporate Native DB OLTP Log Point Of Sale Retail Native OLTP DB Log Downtown Store

15. Keep data synchronized between current production server and a server deployed to test a new application upgrade/version, or a hardware/OS upgrade. Workload balancing capability (i.e. master to master support) allows database instances to remain synchronized where dual or double data entry is a requirement (i.e. data entry occurring on both systems at the same time). 6. Data Synchronization for Upgrades, Migrations and Workload Balancing “Solution deployed to help IT support application, database and platform migrations”. Production Server Testing Server ERP ERP Native Native Upgrades, Migrations OLTP OLTP DB DB Workload Balancing Log Log Manufacturing Manufacturing

16. Replication frequency generally varies from continuous (near real-time) to periodic. Table level refresh or copy can be used in addition to log based change data capture. Reporting server can also be used for consolidation requirements i.e. consolidating financials from multiple branches into a single corporate instance. 7. Offloading Production Query & Reporting Cycles “Solution deployed to allow organizations to offload the impact of query and reporting to a non mission critical system”. Production Server(s) Finance 1 Native OLTP DB Log “CDC” Periodic Services Reporting Server Finance 2 “Table Copy” Report OLTP Query Services “CDC” Continuous Finance 3 Native DB OLTP Log Services

17. Availability of data only, does not support DDL replication. Backup OLTP Exact image replication to produce a backup copy on a separate server or in a different partition on the same server. A separate license is not required for each partition used on the production server. 8. Data Backup And Availability “Solution deployed to allow organizations to backup copies of critical data for recovery where a full disaster solution is not a requirement”. Production Server AvailabilityServer Finance “CDC” Native Backup Continuous (to backup instance) DB Log Partition 1 Partition 2

18. Thank You