1 / 32

A Computer Scientist Looks at the Year 2000 Problem

A Computer Scientist Looks at the Year 2000 Problem. The Keys to High-Quality Year 2000 System Renovation Johnson (John) M. Hart jmhart@world.std.com JMH Associates. “Give us the tools, and we will finish the job.” Winston Churchill to President Roosevelt, 1941. Pretend.

hedwig
Download Presentation

A Computer Scientist Looks at the Year 2000 Problem

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Computer Scientist Looks at the Year 2000 Problem The Keys to High-Quality Year 2000 System Renovation Johnson (John) M. Hart jmhart@world.std.com JMH Associates “Give us the tools, and we will finish the job.”Winston Churchill to President Roosevelt, 1941

  2. Pretend • You are the CIO at a large corporation • You are responsible for the Year 2000 readiness of • 100s of millions of lines of COBOL code • Terabytes of data • The future of the company is in your hands! • And - Pretend it’s 1997 or earlier • Can computer science and software engineering help you?

  3. Y2K Renovation – Getting Started • You are Y2K aware • Have a budget • Performed inventory and impact analysis • The project team is organized • You’ve decided on windowing or expansion • Of course you’ll test the renovated systems • Now what? • How will you renovate your code and data?

  4. Renovation: The Neglected Step • How will you renovate your systems? • Better renovation: fewer errors to find in test • Shift resources from testing to renovation • Large systems require automation benefits • Two key steps • Date-Sensitive Item Identification • System Correction • Caveat: Not entirely automated • Some manual intervention

  5. Common Statements Demonstrate Renovation Neglect • “It will be good enough” • “There’s no ROI in better renovation” • “We’ll fix the mistakes in testing” • “Not enough testing? Decrease the renovation budget!” • “We’ll test all the paths” • “We only need to test what has changed”

  6. Caution! • Testing misses serious & common errors • “Change-based testing” doesn’t test what should have been changed • Good ID & correction: fewer errors to detect • You can’t test quality in • Do it right the first time • You’ll save time and money! • More on this later

  7. Date-Sensitive Item Identification • Key requirement for quality Y2K renovation • What are Date-Sensitive Items? • Variables, constants • File & screen fields • Anything containing year/century data • Required for ALL renovation strategies • You can’t fix what you can’t find • Faulty ID leads to defective systems

  8. DSI ID Investment Pay-Offs • Decreased testing time and expense • Decreased risk to operation • Decreased system repair time • System documentation with long-term value

  9. Faulty DSI ID Example: Original Code DATA DIVISION. WORKING-STORAGE-SECTION 01 TODAYS-DATE PIC 9(6).Correctly identified - yymmdd 01 PAYMENT-YEAR-TO-DATE PIC 9(6). False positive - yymmdd 01 PAYMENT-DUE PIC 9(6).Correctly identified - yymmdd 01 NEW-PAYMENT PIC 9(6).False negative - is yymmdd 01 MAX-YEAR-TO-DATE PIC 9(6) VALUE 100000.False positive - yymmdd . . . PROCEDURE DIVISION. IF TODAYS-DATE > PAYMENT-DUE AND PAYMENT-YEAR-TO-DATE > MAX-YEAR-TO-DATE THEN MOVE NEW-PAYMENT TO PAYMENT-DUE.

  10. Renovated (Expanded) Code DATA DIVISION. WORKING-STORAGE SECTION 01 TODAYS-DATE PIC 9(8).Correctly identified - yymmdd 01 PAYMENT-YEAR-TO-DATE PIC 9(8).False positive - yymmdd 01 PAYMENT-DUE PIC 9(8).Correctly identified - yymmdd 01 NEW-PAYMENT PIC 9(6).False negative - is yymmdd 01 MAX-YEAR-TO-DATE PIC 9(8) VALUE 19100000. False positive - yymmdd . . . PROCEDURE DIVISION. IF TODAYS-DATE > PAYMENT-DUE AND PAYMENT-YEAR-TO-DATE > MAX-YEAR-TO-DATE THEN MOVE NEW-PAYMENT TO PAYMENT-DUE.

  11. Defective DSI ID Example: Results • Defective code and corrupted data • PAYMENT-YEAR-TO-DATE must exceed $19M rather than just $100K • PAYMENT-DUE appears to be a firstcentury date • PAYMENT-YEAR-TO-DATE a file field? Corrupted data! • $500,000 becomes $19,500,000 or $20,500,000 • Windowing logic correction is just as defective

  12. Identification: Method • “Knowledge” propagation • Start with some “truth” about the system • Initial knowledge seeds the propagation • Propagate this knowledge through the code • Get seed information from easy places • Files, reports, screens, pattern matching suggestions • Provide attribute information • Format (YY, YYMMDD, YYDDD, etc.) • There are some hard cases

  13. Knowledge Propagation • Propagate knowledge by applying identification rules at each program statement SEED • IF TODAYS-DATE > PAYMENT-DUE AND • PAYMENT-YEAR-TO-DATE > MAX-YEAR-TO-DATE • THEN • MOVE NEW-PAYMENT TO PAYMENT-DUE.

  14. Propagating Knowledge File A File B Linkage Module A Module B Module C Year 2000-sensitive I/O fields Year 2000-sensitive variables and constants Variables that are not Year 2000-sensitive

  15. DSI ID – Key Points • DSI ID must be automated in large systems • Pattern matching: inadequate • Can suggest seeds • Cannot make decisions • The complete system must be examined • DSI ID must also determine format (YYMMDD, DDMMYY, YYDDD, ...)

  16. Total Automation Not Possible • There are always some human decisions • “No silver bullet” • Pattern matching has some value • Can provide seeding & consistency checks • Assessment • Inventory • Other preparatory phases • But, no matter how refined, pattern (and usage) matching can not decide the dates

  17. These Techniques Work! • Tools from several vendors in use today • Tools can be fast & are getting faster • Best techniques: based on compiler theory • Hard work turns the theory into practice • Cost-effective, especially in large systems • Why do anything else? • Human knowledge & decisions still necessary • Some situations are inherently ambiguous

  18. Quality DSI ID Can Prevent Defects • Before they get into the system! • Faulty DSI ID causes costly defects • Corrupted data • Incorrect business decisions • Defect detection through testing is expensive, risky, and will miss many defects • We’ll see why later • Identification drives code and data correction

  19. Automated Year/2000Correction is Necessary • Correction must be accurate, repeatable • Systems are large and complex • Simplified testing and verification • Intricate changes must be consistent, reliable • Date identification drives correction

  20. Example: Code Renovation Windowing Logic Correction • (a/k/a Procedural Workaround) • Procedure division logic changed • Interpret 2-digit dates in a century window • Window range [ 1900+b and 2000+b–1 ] • Example: [1950, 2049 ] • Existing files retain 2-digit year fields • No changes to existing data files • Variations: Date Expansion, in-line, custom

  21. Logic Correction Example –Windowing Logic Correction • Code fragment before correction IF DUE-DATE GREATER THAN START-DATE AND DUE-DATE LESS THAN END-DATE OR START-DATE EQUAL FIRST-DATE PERFORM SOMETHING.

  22. Logic Correction Example –Corrected Code CALL 'CVRT2TO4' USING DUE-DATE YYYYMMDD-1 CALL 'CVRT2TO4' USING START-DATE YYYYMMDD-2 CALL 'CVRT2TO4' USING DUE-DATE YYYYMMDD-3 CALL 'CVRT2TO4' USING END-DATE YYYYMMDD-4 IF YYYYMMDD-1 GREATER THAN YYYYMMDD-2 AND YYYYMMDD-3 LESS THAN YYYYMMDD-4 OR START-DATE EQUAL FIRST-DATE PERFORM SOMETHING.

  23. ID & Correction: The Keys to Quality • Don’t rely on testing, no matter how thorough • It’s expensive • It will miss many defects • It finds symptoms, not causes • Do it right the first time • Sound software engineering • Essential for “due diligence”

  24. Testing Will MissMost False Negatives • Despite extensive, full-coverage test cases • Unknown dates will not be aged • “Test what has changed” • Totally ignores false negatives • The original code would pass the test! • Also misses many false positives, bad logic • Testing finds symptoms, not root causes

  25. The Reality • Many renovations are poor quality • Recent competitive comparison • 6 vendors, 100 KLOC, 159 required changes • Accuracy ratings: 89%, 75%, 75%, 70%, 60%, 56% • 90% means 16 errors on 100 KLOC 1.6 per 10 KLOC

  26. Answer: Independent Pre-Test Audit • A high-quality tool reports: • All detected false positives and negatives • All detected bad fix insertions • All missing fixes, faulty logic, mixes bases • Fix it now! Before test! • You are fixing the root cause • Testing only finds a symptom • Due diligence, IV&V

  27. It Works! It’s Cheap! • Examples: • 110 KLOC, tested. Found 33 errors • 350 KLOC, tested. Found 200 errors • 400 KLOC, tested. Found 70 errors • 250 KLOC, carefully renovated.Found one error • High-quality renovation: high-quality result

  28. A More Dramatic Case Study • Testing found 31 faulty DSI IDs • 55 staff-days; 1.77 SD per defect found • Independent audit found 22 more faulty IDs • 4 staff-days to find. 0.075 SD per defect • Nearly 25 : 1 cost ratio per defect detected • Plus, additional defects were detected • Detected root cause, not symptoms • Cost-to-fix ratio is even better • No collateral damage from residual defects

  29. Savings from Independent Renovation Inspection • Example: Original Budget 50% TESTING & FIX 10% RENOVATION 40% PLANNING & ANALYSIS BUDGET ASSUMPTIONS: 1,000,000 LINES OF CODE $1,000,000 DOLLAR BUDGET $1.00/LOC

  30. Reduced Budget with IRI 5% IRI 25% TESTING & FIX 10% RENOVATION 20% PROJECT SAVINGS 40% PLANNING & ANALYSIS ASSUME: IRI FOR 1/2 THE COST OF RENOVATION AN AVERAGE OF 1 ERROR PER 10,000 LINES OF CODE ASSUME TESTING UNCOVERS ANOTHER 100 ERRORS

  31. Summary • Renovation is an exacting, repetitive task • Accurate Date ID is essential • Changes procedure, data divisions and files • Automated tools are fast and accurate • Testing is necessary but not sufficient • Independent audits are cost-effective • Experience to date is excellent

  32. More Information • Date Sensitive Item Identification Year/2000 Journal, March/April, 1998 http://www.y2kjournal.com (see “past issues”) • Automation Assisted Correction • Year/2000 Journal, May/June, 1998 • Testing Limitations and Auditing http://www.itpolicy.gsa.gov/mks/yr2000/y2kconf/paper59fp.htm http://www.itpolicy.gsa.gov/mks/yr2000/y2kconf/paper60fp.htm Year/2000 Journal, Sept/Oct, 1998

More Related