Popkin Software: Software Quality Assurance

1. Popkin Software:Software Quality Assurance November 2004 Mariya Shapran

2. Content Table • Popkin Software • who we are • what we do • Our Development Life Cycle • what, when, why, who and where.. • Quality Assurance vs. Quality Control • Quality assurance as part of the product development lifecycle • Standards • Test Life Cycle • SA Metamodel • IDEF0/Black Box/White Box Test Approaches • Best practices • Test Approaches • Merging industry standards into our test methodology • Defects Life Cycle • Critical path • Non-critical path • Questions and Answers

3. Popkin Software The Enterprise Architecture Company

4. Popkin Software Overview • The Enterprise Architecture Company • Founded in 1986 • Leading Provider of Enterprise Architecture Tools • Flagship Product is System Architect® • Vision: • We develop, globally market, sell, and support a diverse but integrated set of Enterprise Architecture, Business Analysis, and Systems Design tools. • Our tools enable clients to visualize, analyze, implement, and communicate their Enterprise Architecture. This enables clients to maximize the efficiency of their organization and improve their strategic decision making capabilities. • World Wide Presence: • Corporate Headquarters in New York City • European Headquarters in United Kingdom • Other Major Offices in Washington DC and The Netherlands • Distributors throughout the World

5. Company Overview • Products: • System Architect®– Flagship Product • System Architect DoDAF • iRMA™ • SA Information Web Publisher™ • SA Compare™ • SA Simulator™ • Training • Consulting • Users: • More than 80,000 users and 40,000 licenses at 7,000 sites worldwide • User groups in the US, UK, and Australia

6. Partial List of Clients • IBM • Lockheed Martin • Publix • Paychex • McGraw Hill • Scholastic • CSC • JPMorganChase • Virgin Mobile • DuPont Corporation • Borders • Vodafone • JB Hunt • Siemens • British Airways • Samsung • Verizon • AT&T • Glaxo-Smithkline • MCI Worldcom • Bell Canada • Time Warner Telecom • BT • Ericsson • Nokia • T-Mobile • Telia Sonera • Department of Defense • US Air Force • Department of Treasury • Department of Agriculture • Veterans Administration • GSA • Department of Homeland Security • Intelligence Agencies

7. System Architect

8. System Architect • Leading Enterprise Architecture Tool on Market • Integrated Support for Modeling Disciplines: • Business Modeling • Latest Standards: BPMN, BPEL4WS, etc • Simulation of Process Models • UML for Systems Modeling • Synchronization with code such as Java, VB.NET, C++, etc • Relational Data Modeling • Implement/Reverse Engineer DBMS’s such as Oracle, DB2, and SQL Server • XML Design • Network Modeling • Communicate Models by Publishing to Websites • Repository based with Networking support

9. System Architect V10 Framework Manager SA Guidebooks Shared Definitions New • Analyze Business: • Enterprise Direction • Processes (BPMN) • Organization • Functions • Technology Infra • IDEF0/IDEF3 • Simulation (IDEF3, BPMN, Process Charts) • UML • Design Applications: • Use Cases • Object Interactions • Classes • Components • State Machines • Java • VB .NET • Design Databases: • ERD Model • Physical Model • IDEF1X • DB Synchronize™ • SQL Server • Oracle 8, 9i • Analyze Legacy Systems: • Gane/Sarson • Ward/Mellor • Yourdon/DeMarco • SSADM • Design XML Schemas: • DTDs • BizTalk • Instance Docs • Test Data • UML Integration • Data Modeling Integration Data Modeling Structured Methods XML Design Business Modeling New Explorer Diagram New New New Repository New Customizable Repository Metamodel SA Compare Microsoft SQL Server New Reporting/Publishing Browsing Interfaces Matrices MS Office HTML XML SVG HTML OLE Automation CSV XML HTML MS Office XML SA InfoWeb Pub Improved XML New

10. Repository Repository Repository Customizable Repository Metamodel Customizable Repository Metamodel Customizable Repository Metamodel System Architect V10 FunctionsUsers Encyclopedias NT Authorization Role-BasedAccess Control New Catalog Configuration Management SA Compare Backup Reporting Merge/Extract

11. Popkin’s View of Standards IDEF BPMN Catalyst Vendor- driven Entity Relationships UML BPEL4WS FEA Frameworks

12. Enterprise Architecture XML HTML/Web Automated Reports on Information in Repository Information Publisher XSLT Enterprise view of data SA Repository 16+ years of enterprise architecture and modeling ICAMS System Architect Multiple Method View of Information Industry Standards BPMN XML Schema Entity / Relation UML Frameworks C4ISR DoDAF Methods E/R Data UML BPM Frameworks

13. Evolution of System Architect Business Process Modeling (IDEF) First Windows-Based Modeling Tool Introduced C4ISR Framework Support Structured Analysis and Design TOGAF Support UML Support ‘88 ‘96 ‘97 ‘98 ‘99 ‘00 ‘01 ’02 ‘03 Object- oriented Data Modeling Simulation, XML Support SQL Server As Underlying Repository Zachman Framework Support Multiple Methods

14. Product Development Life Cycle Management Team Commercial Release Project Proposal Business Team Business Requirements Acceptance Definition Test Development Team Test Cycle Technical Program Code Unit Test Integration Fix Bugs Design & Audit Specification Test System Deployment Quality Control Team Test Regression Test Build Test Review Test Execute Test Quality Review Cases Cases Cases Performance & Multi-user Test

15. Quality Assurance vs. Quality Control • Quality Assurance • Quality Control What’s the difference?

16. Quality Assurance: • System Development Methodologies • Estimation Processes • System Maintenance Processes • Requirements Definition Processes • Testing Processes and Standards • Measurement Programs to evaluate processes • Processes gap analysis • Concerned with all of the products that will ever will be produced by a process

17. Quality Control • Relates and concern with a specific product or service; • Verifies whether specific attributes are in, or are not in a specific product or service; • Identifies defects for the primary purpose of correcting defects;

18. Test Life Cycle QA Management Team Commercial Release Project Proposal Test Plan Business Team Business Requirements Acceptance Definition Test Development Team Test Cycle Technical Program Code Unit Test Integration Fix Bugs Design & Audit Specification Test System Deployment Quality Control Team Test Regression Test Build Test Review Test Execute Test Quality Review Cases Cases Cases Performance & Multi-user Test

19. Quality Factors

20. The System Architect Metamodel

21. IDEF0 – Integration Definition for Function Modeling

22. Black Box Test Approach

23. White Box Test Approach

24. Test Approach • Black Box Approach • Boundary Value Analysis (internal code) • Regression Testing (Test Partner and Loadgen) • Data Integrity Testing (Internal Tools) • Stress (Volume) Testing (Loadgen) • Backup and Recoverability (Internal Tools) • Functional Testing (Test Partner and Manual Testing) • Security Testing • Configuration/Compatibility Testing (VMWare images) • User Acceptance Testing (End Users) • Benchmark Testing (Loadgen) • Operational Readiness Testing (Loadgen and Test Partner) • Usability Testing (End Users/508 Compliance Guidelines) • Alpha/Beta Testing (Loadgen) • White Box Approach • Memory Usage (BounceChecker) • General Resource Usage • Error Handling • Gray Box Approach • Combination of Black Box and White Box Approach

25. Boundary Value Analysis Example: Problem: Need to understand what will take place at the boundaries of the behavior for each component. Example: Field Name for the Diagram is required to accept unique name for the diagram consistent of alpha-numeric characters up to 105 characters (with exception for special characters) Boundary Value Analysis: Absence of the name = rejected Special Characters = rejected (this is on the boundary) null = rejected 2alpha5 numeric characters = accepted All alpha characters = accepted All numeric characters= accepted 106 characters = rejected

26. Best Practices • Peer Reviews • Defined Requirements Standards • Defined Quality Standards • Constant improvements to Quality Processes • Implementation of Automation Testing into Regression and Performance Test Phases

27. Merging Industry Standards into our Test Methodology • 508 Compliance • 508 Compliance Checklist • Methodologies: IDEF0, IDEF1, IDEF3 • IDEF0/IDEF1/IDEF3 Compliance Checklist • IEEE Standards • Used as guideline in defining of Quality Assurance Processes • ISO Standards • Used as guideline in defining of Quality Assurance Processes

28. 508 Compliance Check List • Menu items have a unique, functioning access key. • Shortcut or popup menu items have a unique, functioning access key. • Dialog controls have a unique, functioning access key. • Dialogs can be closed by pressing the ESC key. • Shortcut or popup menus can be opened via the keyboard. Most keyboards have what's called an application key. Pressing this key displays the shortcut menu for the selected object. • Drag and drop functionality can also be executed via the keyboard. • Dialogs have a logical and consistent tab order (left to right and top to bottom). • Color alone should not be used to indicate different states. For example, in access control red and green colored arrows were used to represent checked in and checked out items. • The tabs on a tabbed control can be navigated by pressing CTRL+TAB. • Tree controls can be navigated by pressing the arrow keys. • Interfaces containing multiple windows and/or frames allow for focus to be shifted via the keyboard. • Focus is always placed on the most recently opened or accessed window. • The application’s active window should display a visual focus indicator at all times so that users can anticipate the effects of their keystrokes. • The application can read, use, and preserve system-wide user interface (UI) settings when displaying customized controls or window content. The system-wide settings adopted by a user enhance the product’s accessibility; the application should use them where possible, and not disable or disregard them. An example of a system-wide setting is the ToolTip size. • The application does not convey information exclusively by sound. If sound is the default method for conveying information, the application should provide other options to express this information.

29. IEEE Software Engineering Standards

30. Defect Life Cycle

31. Glossary • Organizations: • ARTS – Retail Tech Standards with Industry Data Model • www.nrf-arts.org • BPMI – Business Process Management Standards body • www.bpmi.org • DAMA – Data Management Association • www.dama.org • DSDM – Dynamic Systems Development Method • www.dsdm.org • NASCIO – National Association of State Chief Information Officers • www.nascio.org • OMG – Object and XMI Tech Standards Body • www.omg.org • Supply Chain Management standards with SCM data model • www.supply-chain.org • Telecoms Tech Standards with Industry Process Mode • www.tmforum.org • The Open Group • www.opengroup.org

32. Glossary • FEA • Federal Enterprise Architecture • BRM - Business Reference Model • SRM - System Component Reference Model • PRM - Performance Reference Model • DRM - Data Reference Model • TRM - Technical Reference Model • BPMN • Business Process Modeling Notation • IDEF • Integrated Definition Language, a systems modeling technique using a specific graphical structure • ORM • Object Role Modeling • E/R • Entity Relation • BPEL4WS • Business Process Execution Language for Web Services • UML • Unified Modeling Language, an object-oriented design language • XMI • XML Metadata Interchange • DoDAF • Department of Defense Architecture Framework • C4ISR • Command, Control, Communications, Computers, Intel-ligence, Surveillance, and Reconnaissance Architecture Framework

33. POPKIN SOFTWARE Popkin Software 2 Rector Street New York, NY 10006 Tel: (646) 346.8578 Fax: (646) 346.8501