Project Presentation Document Optimization 11 May 2007

1. Project PresentationDocument Optimization11 May 2007 • Team members: • Chris Catalano • Chun-Yu Chang • Chris Joson • David Matthes • Sponsors: • Huron Consulting Group • Aerospace Corporation

2. EDD Background • Electronic Data Discovery (EDD) is the systematic collection, processing and review of electronic files to support the litigation process. • EDD is used in: • Alleged stock-back dating • Government reviews of mergers and acquisitions • Other dirty deals e.g. blackmail, fraud, embezzlement • The current processing system was designed for component flexibility and variability. • The market place is shifting to an environment that holds speed and automation paramount.

3. Project Objectives • Evaluate the current EDD system against two alternatives. • Client: Huron Consulting Group • Evaluate SysML as an effective modeling language for systems engineering. • Client: Aerospace Corporation

4. Approach • Modeled and compared three EDD systems in SysML. • Evaluated the EDD systems from a capital budgeting perspective • Evaluated quantitatively our experience with SysML.

5. Agenda Approach - SysML Model Analysis - Trade Study Evaluation - SysML Usability

6. Approach - SysML Model • SysML is a modeling language for: • Defining systems • Analyzing systems • Communicating different system viewpoints • EDD Team captured the following viewpoints: • Requirements Diagram • Use Case Diagram • Block Definition Diagram • Activity Diagram

7. Requirements Diagram • Captures EDD requirement hierarchy • Provides traceability to EDD components

8. Use Case Diagram • Describes EDD contextual relationships • Show interactions between entities and the EDD System

9. Describes Behaviors Describes the characteristics Block Definition Diagram Describes the structured composition of EDD

10. Activity Model of Current Process Specifies the flow of inputs/outputs and controls, including sequence for coordinating activities.

11. Partitions Partitions (swimlanes) show responsibility for each activity. Processing Server – 9 programs Engineer – 9 copies/move Unix – 6 Perl scripts Review Team – 6 QCs Mac Client – 1 activity

12. Alternative (Attenex or Autonomy) process used to compare with the current process Processing Server – 1 program Engineer – 2 copies/move Unix – 2 Perl scripts Review Team – 2 QCs Mac Client – 1 activity

13. Advantages of Alternative Process • Fewer manual steps • Reduced probability of error • Simpler to maintain • Easier to train • Less rigid process • Shorter time to process documents

14. Agenda Approach - SysML Model Analysis - Trade Study Evaluation - SysML Usability

15. Net Present Value Probability Distribution • The goal was to model the financial impact of each alternative over three years using Net Present Value (NPV). • NPV is a capital budgeting technique used to estimate and compare cash flows for competing systems and projects. • For each system the Net Cash Flow was decomposed, modeled, and run in a Monte Carlo simulation to generate NPV estimates. • The results are NPV probability distributions for each alternative • t – time • n – total project time • r – discount rate • Ct – net cash flow • Co – Initial capital expenditures at time zero

16. Net Present Value • Compared to the baseline, the alternative systems increase the processing speed and the ability to accept projects. The trade off is increased costs. • Autonomy: • $2,000,000 initial cost • $250,000 annual maintenance cost • Attenex: • $500 per gigabyte processed operational cost • How does the increased ability to accept new projects and the increased costs impact the profitability of the systems?

17. Baseline Attenex Autonomy Probability NPV ($) NPV – Results NPV Probability Density Function The results assume that the alternatives will increase the number of projects allowed into the system by twice the baseline. Autonomy is being pulled into a higher range of profitability! Baseline Mean: $12.0 Million Attenex Mean: $12.3 Million Autonomy Mean: $16.2 Million

18. Conclusions & Recommendations • The model shows that by increasing the opportunity to accept new projects the alternative systems can overcome the increased costs! • The future system for Huron will be a hybrid of the alternatives. • The process used for a particular project will be dependent on the clients’ requirements. • The baseline system, while slower, provides a reliable and cost effective solution. • For clients who choose higher speeds at higher costs Attenex would be an ideal fit. (Huron already owns licenses for the software!) • It is critical to spread the costs of Autonomy across the three EDD groups. In effect distributing the responsibility for recouping the investment!

19. Agenda Approach - SysML Model Analysis - Trade Study Evaluation - SysML Usability

20. Aerospace asked us to evaluate SysML to determine how effectively SysML and Rational System Developer worked Evaluate SysML as a modeling language for designing systems Evaluate SysML maturity Determine how useful SysML is for systems engineering design and evaluation Evaluate IBM Rational System Developer Determine how well it supports SysML usage Purpose of the SysML Evaluation

21. Created a Multi-Attribute Utility Assessment Evaluation Hierarchy survey Survey contained 41 questions developed to assess the strengths and weaknesses of SysML and Rational System Developer Questions were answered on a 1 to 5 Likert scale with 5 indicating a positive response Surveyed 8 OR680 Students using SysML Electronic Data Discovery (EDD) Tactical Surveillance Satellite (TSS) Approach: Survey

22. Multi-Attribute Utility AssessmentEvaluation Hierarchy Overall Utility Effect On Task Performance Usability Process Quality Product Quality General Ease Of Use Interface Decomposed the evaluation criteria to evaluate different aspects of SysML and Rational System Developer Ease of Learning Flexibility * Modified from Adelman & Riedel, Handbook for Evaluating Knowledge Based Systems, 1997

23. Utility Results • Tactical Surveillance Satellite (TSS) • Electronic Data Discovery (EDD) • Values closer to 5 indicate positive responses

24. Survey Analysis • SysML • Strengths • Overall respondents felt SysML was a good language • Scored well in usability and flexibility • Weaknesses • The main weakness in SysML is that it is difficult to learn • Respondents took 20-40 hours to become a functional user • Rational System Developer • Strengths • Rational System Developer scored highest in usability • Survey indicates that people found Rational System Developer fairly easy to use • Weaknesses • Survey indicted low scores for ease of training • The Interface and product quality also scored lower than other areas

25. Recommendation to Aerospace • SysML • SysML is difficult to learn and will require investment in training and time • May not be practical for smaller systems or processes with limited complexity • However, if people are already trained, SysML diagrams ensure consistency and provide effective communication across multiple disciplines • Rational System Developer • Rational supported the creation of models and helped maintain consistency • Process descriptions were created and analysis performed using Rational and SysML • SysML is well suited for complicated systems with significant hierarchical decomposition, systems common in the National Security Space domain

26. Summary • Huron asked us to evaluate their current EDD system and two alternatives • Used SysML and NPV to perform the analysis • Determined that the best solution is a mix of the current system for most clients and Autonomy for clients that require faster processing and can afford the increased cost • Aerospace asked us to evaluate SysML to determine how effectively it can support system engineering design and analysis • Conducted a survey to help answer this question. • The survey found that SysML is a useful tool, but the learning curve is steep

27. Acknowledgements • Heather Howard, Shana Lloyd, and Julie Street, Aerospace Corporation • Chris Genter, Huron Consulting Group • Professor Laskey, George Mason University • Sanford Friedenthal, Lockheed Martin • Professor Adelman, George Mason University • The TSS Team • David Alexander, Kevin Sadeghian, Siroos Sekhavat, and Tom Saltysiak

28. Future Work • Optimize Parametric Diagram to make the model executable • Run executable model • Compare executable model results with results obtained from Microsoft Excel • Distribute SysML survey to future students for a larger sample and further analysis

29. Questions?

30. Backup

31. Decompositions Component Diagram

32. Parametric Diagram • Parametric Diagrams were created to express constraints between value properties and allow to perform an executable model. • Executable model used to provide analysis for performance, safety, reliability, throughput, weight, cost, etc. • High Learning Curve • Lack of Time (Estimation of >20+ additional hours to learn SysML limitations) • Inexperience with Simulation Toolkit (Estimation of >30+ hours to execute with toolkit) • Inexperienced team with Java (Estimation of >70+ hours to learn Java)

33. Sample Survey Questions • Questions focus on either SysML as a language or IBM Rational System Developer as a tool • Most questions will be rated on a scale of 1 to 5 • Responses will be averaged together to determine a score for each category • Sample Questions • Overall, SysML improves the system design process. • Rational System Developer provides feedback when processing user commands. • SysML was easy to learn. • I can easily add model elements to the System model.

34. Survey Example • Survey will have participant answer a series of questions

35. Webpage mason.gmu.edu/~cchang7

37. General Status

38. Schedule

39. NPV Backup

40. NPV - Formula • Where: • t – time • n – total project time • r – discount rate • Ct – net cash flow • Co – Initial capital expenditures at time zero

41. NPV - Assumptions • Number of Projects Limitations: The number of projects entering into the system can not be greater than the maximum level of availability. • Projects Start and Completion Time: All projects started in a month are assumed to be completed within that month. In practice this assumption can be interpreted as larger scale projects are started early in the month while smaller projects are started later in the month. • Minimum Revenue: $2500 is the minimum amount of revenue accepted for a job. • Autonomy Costs: The Autonomy system has an initial cost of $2 million dollars and an operational cost of $250,000 annually. • Attenex Costs: The Attenex system has an operational cost of $500 dollars per GB processed. • Prospective Projects: The level at which prospective projects are found is consistent for all systems. • Availability Parameter: The availability parameter is being used to model the size and availability of the queue for incoming projects. • Pricing Scheme: The pricing scheme is constant for each system over the three year period. No adjustments have been made to the pricing schemes of the higher cost alternatives. • Migration Costs: With the exception of initial software costs, all migration costs are ignored in this model.

42. NPV- Revenue Inputs (1) • Annual Revenue: The annual revenue is the sum of twelve monthly revenue estimates. • Monthly Revenue: The monthly revenue is the sum of the revenue for each job accepted and completed in a month. • Revenue per Project: The revenue per project is the amount of revenue in dollars that a generated by a project. • Projects Accepted: This value is the total number of projects entered into the system each month.

43. NPV- Revenue Inputs (2) • Maximum level of System Availability: The maximum level of system availability is the largest number of projects that can enter into the system each month. • Number of Prospective Projects: The number of prospective projects describes the number of projects that are available to be entered into the system. • Number of Staff: The number of staff plays a critical role in limiting the number of jobs that can be entered into the system each month. • Processing Speed: Processing speed describes the rate at which projects can be pulled through the system.

44. NPV- Cost Inputs • Initial Costs: The costs used to procure new software and equipment for the alternative systems at the onset of the migration. The initial costs are incurred once at the beginning of the project. • Maintenance Costs: Monthly costs associated with maintaining the software and hardware systems. The maintenance costs include repairing machines, software upkeep and spare parts. • Salary Costs: Monthly costs related to employee salaries. • Operational: Monthly costs related to procuring additional equipment, software and the overhead costs related to the building and facilities.

45. NPV – Parametric Diagram