Managing Complex Testing Environments by John Butler SASQAG 3/20/2003

1. Managing Complex Testing Environmentsby John ButlerSASQAG 3/20/2003 Disclaimer The information in this presentation are my opinions and not necessary those of my employer.

2. John Butler, Manager Integrated Validation The Boeing Company In his current position in the Information Systems Organization John has the responsibility for Integrated Unit Test, Integrated Functional Test and validation of the Business Processes. John has been supporting the implementation of a large-scale system and business process re-engineering plan, for over 8 years in areas dealing with the design, planning, & testing.

3. Work Package Validation Flow Application A Application F Application G WP 5.01 Application C WP 5.12 Application E Application H Application B WP 5.03 Application D Application J = Functional Test = Integrated Unit Test

4. IUT Next Blkpt IUT Sust IUT Next Blkpt IUT Sust Test Planning ICE 1 NextBlkpt ICE 1 Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust Sustaining ICE Vision 13 week cycle Production Environment IUT Next Blkpt IUT Sust Test Planning ICE 1 NextBlkpt ICE 1 Sust IUT Next Blkpt IUT Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust IUT Next Blkpt IUT Sust Test Planning ICE 1 NextBlkpt ICE 1 Sust IUT Next Blkpt IUT Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust

5. Sustaining ICE Vision IUT Next Blkpt IUT Sust IUT Next Blkpt IUT Sust Test Planning ICE 1 NextBlkpt ICE 1 Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust 13 week cycle Production Environment IUT Next Blkpt IUT Sust Test Planning ICE 1 NextBlkpt ICE 1 Sust IUT Next Blkpt IUT Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust IUT Next Blkpt IUT Sust Test Planning ICE 1 NextBlkpt ICE 1 Sust IUT Next Blkpt IUT Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust

6. Sustaining ICE Vision Scheduled Block point Scheduled Block point Production Environment E-rolls ICE 1 Block point A ICE 1 Sust • Integrated • IFT • Block point Testing • Performance • Interoperability • Scalability • User Acceptance testing • Production Support • Ticket Evaluation • E, S, C –Roll test • Production Performance test • Production Blackout Support • Problem Management • Detailed Analysis E-rolls Sync Build ICE 2 Block point B ICE 2 Sust

8. Sustaining ICE Vision 13 week cycle Production Environment IUT Next Blkpt IUT Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust • Sustaining • Production Support • Ticket Evaluation • E, S, C –Roll test • Production Performance test • Production Blackout Support • Problem Management • Detailed Analysis IUT Next Blkpt IUT Sust Test Planning ICE 1 NextBlkpt ICE 1 Sust • Next Block point • Block point Testing • Performance • Interoperability • Scalability • IFT • User Acceptance testing IUT Next Blkpt IUT Sust Test Planning ICE 2 NextBlkpt ICE 2 Sust • Test Planning • Review Content • Determine Test Plans • Determine Data Requirement • Determine VPS test • Determine Batch jobs • Develop Test execution schedule

9. Entrance Criteria

10. Build Readiness Review QUESTIONS 1.Your project's integrated test plans (735) are updated, accurate, and complete. 2a. Your project’s build scripts have been reviewed and validated. 2b. Your project's code is complete. 2c. Your project’s functional test is complete. 3.Your project is ready to build on the build dates. 4.Your project's Change Request are in the appropriate state (i.e. Work, Verify CR Complete, Perform System Test). 5.Your project’s configuration files have been reviewed and validated. Yes = Project must perform this activity. It has been completed. No = Project must perform this activity. It has not be completed. N/A = Project does not perform this activity for this build cycle.