Change Data Capture & Change Tracking Deep Dive

Change Data Capture& Change TrackingDeep Dive W. Kevin Hazzard LinchpinPeople.com Group Principal

Change Data Capture analysis transactions • Turn it onEXECUTE sys.sp_cdc_enable_db; • Which generates for the whole database • A capture job • A cleanup job • Then include at least one table withEXECUTE sys.sp_cdc_enable_table …; • Which creates a capture instance containing • A change table • An all-changes function • A net-changes function (optional) sources data mart ETL capture job warehouse ETL archive sector changes

Capture Instance • Contains • The capture table • cdc.fn_cdc_get_all_changes_<instance name> function • cdc.fn_cdc_get_net_changes_<instance name> function (optional) • Maximum two per source table

Sys.sp_CDC_ENABLE_TABLE • @source_schema • @source_name • @supports_net_changes • @role_name • @filegroup_name • @index_name • @captured_column_list • @allow_partition_switch • @capture_instance

DEMONSTRATION Enabling Change Data Capture (CDC) Creating a CDC Capture Instance Querying CDC Tables as Things Change

SQL Server Agent JOBS • cdc.Capture_capture • Starts with SQL Agent • RAISERROR(22801, 10, -1); • EXEC sys.sp_MScdc_capture_job; • Just a wrapper for sys.sp_cdc_scan • cdc.Capture_cleanup • Runs at 02:00 daily • EXEC sys.sp_MScdc_cleanup_job;

CDC Data Retention • Based on LSN Validity Intervals • Database • Capture Instance • The cleanup job deletes CDC data acording to retention policySELECT * FROM msdb.dbo.cdc_jobs WHERE job_type = N'cleanup'; • You can start the cleanup job manuallyEXEC sys.sp_cdc_start_job @job_type = N'cleanup';

Handling Schema Changes • Deleted and new columns are handled well • Modified columns require specific steps: • Stop the capture and cleanup jobs • Change the schema as necessary • Generate new capture instances for all modified tables • Process all data in the old and new capture instances • Manually run the cleanup job • Delete the old capture instance • Turn the capture and cleanup jobs back on

Change Tracking • Turn it on for one databaseALTER DATABASE <DBNAME> SET CHANGE_TRACKING = ON; • Then enable a tableALTER TABLE <TBLNAME> ENABLE CHANGE_TRACKING; sync transactions warehouse ETL archive sector sources

Change Tracking TABLES & FUNCTIONS • sys.change_tracking_databases • sys.change_tracking_tables • CHANGETABLE(CHANGES) • CHANGETABLE(VERSION) • CHANGE_TRACKING_CURRENT_VERSION() • CHANGE_TRACKING_MIN_VALID_VERSION() • CHANGE_TRACKING_IS_COLUMN_IN_MASK() • WITH CHANGE_TRACKING_CONTEXT

DEMONSTRATION Enabling Change Tracking (CT) for a Database Enabling CT on a Table Querying CT Tables and Functions as Things Change

Handling Schema Changes • No modifications to the primary key are allowed including related indexes • Dropping columns is OK but they may still appear in the change data • When adding columns, changes are tracked but the metadata change is not reported • Switching partitions will fail on change tracked changes • Data type changes are not tracked

Which one is RIGHT FOR ME? Change Data Capture Change Tracking Works in all versions of SQL Server Returns differences from current Storage light Good for device synchronization No job agent required Operates best with snapshot isolation • Works in Enterprise Edition only • Stores every discrete change • Storage intensive • Good for auditing • Requires SQL Server agent • No special serialization required

Change Data Capture & Change Tracking Deep Dive