Comments on the Utility vs. Burden of Audit Trails

1. Comments on the Utility vs.Burden of Audit Trails “Audit trails are the single largest cost component of 21 CFR 11 compliance.” John Doe, presenting at CHPA / FDA 1999 FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

2. A Word From Our Sponsor Subpart B—Electronic Records § 11.10 Controls for closed systems. …Such procedures and controls shall include the following: (e) Use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes shall not obscure previously recorded information. Such audit trail documentation shall be retained for a period at least as long as that required for the subject electronic records and shall be available for agency review and copying. FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

3. Part 11’s Literal Meaning • The only transactions that need audit trails are ones performed by “operators” • The only data that is required to be in the audit trail itself is the date and time • This means we do not have to replicate data from the transaction in the audit trail • Technically, we do not we do not even need to record the operator’s ID • There are some very good reasons to take a minimalist approach to audit trails FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

4. Audit Trails - Current Pharmaceutical Model • Audit trails are usually replications of a subset of a transaction record • “Source record” >>> “Audit record” • Audit records are usually stored in a similar (if not the same) data structure • Ubiquitously, audit records have the same or lower security level as source records • Hollis refers to this scheme as“Data-level Audit Records” FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

5. Record #1234 // UID tquinn // RtID // 34441 // text // NSTRCT // DENTRY // Field // PATWGT // DVALUE // 237.4 // DMUNITS // LBTNTH // APPRVL // BLM // Data-level Audit Records(Creating a New Record) SOURCE DATABASE AUDIT DATABASE Record #1234 // UID tquinn // RtID // NSTRCT // DCREAT // Field // PATWGT // OLDVAL // 000.0 // NEWVAL // 237.4 // CHGRSN // CREVAL // APPRVL // BLM // CHDATE // 20040611144723 // EOR FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

6. Data-Level Audit Records(Correcting a Typographic Error) SOURCE DATABASE AUDIT DATABASE Record #1234 // UID tquinn // RtID // Record #1234 // UID tquinn // RtID // 34441 // text // NSTRCT // DMODFY // NSTRCT // DCREAT // Field // PATWGT // Field // PATWGT // DVALUE // 237.4 // OLDVAL // 237.4 // NEWVAL // 137.4 // DMUNITS // LBTNTH // APPRVL // BLM // CHGRSN // TYPGER // APPRVL // BLM // CHDATE // 20040611145341 // EOR FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

7. Audit Trails – CurrentFinancial Model • The term “audit trails” is misleading; these are actually “audited transactions” • System A proposes transaction • System B proposes agreement • System X (the security system) examines • The data labelling • A’s and B’s privileges • The structure of the transaction • System X grants permission for the transaction • And keeps a log • All in real-time FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

8. System-Level Audit Records(Any Type of Transaction) SOURCE DATABASE JOURNAL FILE Read:Cust_Rec:tquinn2270; Cust_ID::tquinn // Xactn::Withdrwl // *.*|| Acct_ID 1234abcd567 // Amt // 60.00 // Writ>:Xact_prop:tquinn2270; Cur_Bal;310.65|| Term_ID // Pa431 // DatTim // 2004062013025433 Read:ACF_2_Rcpt:Auth_cod: <result>|| Writ:tquinn2270:Cur_Bal; 310.65:Auth_cod;<result>|| FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

9. Comparing the Two • Data-level audit trails: • Are much easier to program and run • Tend to produce larger record sets • Keep the audit and source data in the format • A MUCH easier to compromise • System-level audit trails: • Are much more difficult to include in designs • Tend to produce smaller record sets • Keep the audit and source records separate • Are MUCH more difficult to compromise FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

10. Risk Analysis • Data-level audit records and source data are (about) equally vulnerable to insider threats • Insiders are the most common threat • Replicating data-level audit records provides outsider adversaries with two attack vectors • It’s more effective to invest in other defenses • System-level audit records are only useful in prevention if they are used in real-time • In order to assist with detection, they must be periodically and meticulously reviewed FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

11. Recommendations • Do NOT change the audit trail wording of 21 CFR § 11.10 (e) to require more information in the audit trail • Perform a Regulatory Flexibility Analysis to justify the requirement for audit trails, and include details of: • Financial burden of audit trails, particularly upon small and disadvantaged businesses • Raw and normalized statistics of when audit trails have been useful in protecting public health FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn

12. Questions? Thomas Quinn, President The Hollis Group, Inc37 North Valley Rd. #105Station Square IIPaoli, PA 19301 tquinn@hollisgroup.comwww.hollisgroup.com v: 610.889.7350 f: 610.296.2339 FDA -21 CFR 11 Public Meeting, 2004JUN11, T.Quinn