Software Engineering Research Center

1. Software Engineering Research Center An NSF Industry - University Cooperative Research Center Aditya Mathur, Purdue University Director Robert Cowan, West Virginia University Assistant Director Fire invented

2. The SERC Organization • -History of I/UCRCs and NSF mission • -Our Affiliates and the Administration • -Industry and University Partnerships • Current Research Projects • Technology Transfer Examples • Virtual SERC (VSERC) Outline for Presentation

3. Support research and education in science and engineering - Foster the interchange of scientific information among scientists and engineers in the United States and foreign countries - Technology Transfer via industry/university partnerships National Science FoundationPromoting the Progress of Science

4. Industry/University Cooperative Research Centers(IUCRCs) • Developed by NSF in 1970’s • 50+ centers nation-wide • -SERC: I/UCRC for Software Engineering • True partnership between industry and university researchers Secrets of Success

5. Established: 1986 Purdue University • Ball State University University of Florida • University of West Florida Oregon Associated Universities • University of Oregon • Portland State University • Oregon State University • Oregon Graduate Institute West Virginia University Added in 1994 Added in 1998 The SERC IUCRC Tenured Faculty

6. Army Research Lab: Atlanta,GA • AveStar Inc.: Fairmont, WV • Telcordia: Morristown, NJ • British Telecom Labs: Martlesham Heath, England • Dynamix Inc.: Eugene, OR • ManTech:Fairmont, WV • Motorola, Inc.: Schaumburg, IL • NASA IV&V: Fairmont, WV • Northern Telecom: Richardson, TX • Northrop Grumman: Melbourne, FL • Northrop Grumman: Rolling Meadows, IL • Tivoli Systems, Inc: Indianapolis, IN Current Affiliates

7. Metrics,Tools for Measurement of Software Design Quality • Methods for Assessing and Assuring Data Quality • Testing Distributed Systems • Testing Video Games • Evaluation of ROI on V&V Activities • Software Process Improvement Modeling and Tools • Computer-Based Instructional Technologies • Test Coverage Analysis and Test Optimization • Software Reuse Examples of Research Projects

8. Universities Oregon Associated Schools Purdue University Ball State University University of Florida University of West Florida West Virginia University Industrial Advisory Board Technology and Software Industry and University Network Companies Army Research Lab Bellcore British Telecom Labs Dynamix ManTech Motorola, Inc. Nortel Northrop Grumman, IL Northrop Grumman, FL Tivoli Systems, Inc., IN Partnership provide is composed of Collaboration Researchers produce is available to Permission (2)

9. SERC Administration National Science Foundation SERC Policy Board Industrial Advisory Board Director: Aditya Mathur Assistant Director: Robert Cowan Site Director: Stuart Faulk Site Director: Aditya Mathur Ball State University Purdue University University of West Florida Site Director: Steve Thebaut University of Florida Site Director: Ali Mili University of Oregon West Virginia University Oregon State University Portland State University Oregon Graduate Institute . Sharing Facilities

10. Source of fresh research problems • access to industrially generated data and technology • fast turn-around on proposals • publishable research • satisfaction of seeing research used in industry • potential for additional funding from other NSF programs Benefits to Universities The Lab After Hours

11. Influence selection of research projects • exploit innovations that may lead to new products • improve software quality and productivity of software engineers • new software engineering technology….access to expertise • use of unique laboratories, facilities • satisfy corporate commitment to support university research • interaction with peers • technical reports • on-site short courses and seminars • summer interns, potential future employees Benefits to Industry

12. National Science Foundation • Participating Universities • Industrial Affiliates - support faculty and graduate student projects • Membership Fees - $30,000 per year full membership - $5,000 per year limited membership (for small businesses only) SERC Funding Learning the Rules

13. Poster sessions • Researcher presentations • IAB member presentations • Potential company presentations • Software demonstrations • IAB and SERC Admin. meetings Semi-Annual SERC Showcase

14. IAB SERC Staff Industrial Request For Proposals: IRFPs interact with to generate . IRFPs Broadcast IRFPs to SERC Universities Purdue+ Florida+ Oregon+ West Virginia+ Broadcast Proposals from Researchers to Industrial Affiliates Army Research Lab Bellcore British Telecom Labs Dynamix Motorola Nortel Northrop Grumman, IL Northrop Grumman, FLTivoli Systems, Inc. A Lab Mentor and We Learn from One Another

15. Problem: When requirements change, can a new software version be evolved automatically and efficiently? Approach: Genetic programming. Rationale: With genetic programming we can evolve programs by means of natural selection. To go from one software version to the next, it may be much faster to start with the previous version instead of starting from scratch. Status: In our current genetic programming system, medium-sized image-processing software can be evolved from scratch. Goals: Quantitative assessment of how well genetic programming can evolve new versions of software. Guidelines for conducting the evolution process. Study applications of the technology to SERC affiliate projects, beyond image processing. Software EvolutionJens PalsbergPurdue University

16. Problem: Approaching data quality in a more comprehensive way than what is currently being done. Existing scrubbing tools fail to realize that data is a complex entity that is replicated, dynamic, and tightly coupled to the business processes. Approach: (a) Define a framework for data quality: Including dimensions, requirements, policies, and a taxonomy for dealing with and improving data quality, and, (b) Develop a methodology that deals properly with the issues of initial cleanup, continuous cleanup, and process improvements. The methodology must be comprehensive, implementable, and measurable. Goals:Data quality has been a major cause of customer dissatisfaction, lost business, and inability to meet business objectives. While quality has a clearer meaning when it comes to software, it is much more poorly defined when it comes to data. A generally acceptable and operational definition of data quality refers to Accuracy, Consistency, Currency, and Completeness of data. The specific goal of this project is to develop a methodology that deals with all aspects of data quality. Data QualityAhmed ElmagarmidPurdue University

17. Description: “Where in this program is functionality X implemented?” Method: Use test cases as probes to locate functionalities Example: Where is call forwarding implemented? • run small set of test cases that involve forwarding a call • run set that does not involve the call • analyze traces to look for program components that were executed in first set and not second set Goal: • Spotlight code fragments • Tech Transfer: to make Software Recon an affiliate productivity aid on affiliate projects Software Maintenance:Spotlighting the Code Norman Wilde University of West Florida

18. Bellcore Maintenance of OO Programs 1991 - 1993 • Bellcore concerned about the maintainability of OOPs, requested SERC research on this topic • Over two summers, UWF conducted interviews w/Bellcore programmers and performed study of OO tools in use at Bellcore • Study identified maintenance issues and tool needs for OOPs - earliest research on this topic • Results used in defining Bellcore’s programming environment • Bellcore/UWF jointly authored papers in “IEEE Transactions on SE” and “IEEE Software” • UWF student hired by Bellcore to continue work Tech Transfer Examples: Norman Wilde

19. Northrop Grumman (RM) Software Recon tool 1995 - current • trial of Software Recon conducted at RM on a NG in-house C language code management system • several tool improvements made based on feedback from NG programmers • C Software Recon tool now in use • NG request Ada Recon tool, now under develop. Norman Wilde: Technology Transfer (cont.)

20. Northrop Grumman (FL) Recovery of Design Threads 1995 - 1997 • NG software engineers suggest Recon could be used to recover “design threads” from JSTARS, large battle management system • collaborate on a trial, which successfully recovered threads • joint authored paper to appear in “Journal of Systems and Software” Norman Wilde: Technology Transfer (cont.)

21. Description:During the software design process, developers should be able to infer more about the software they are developing. Computing metrics allows one to choose the best design, as well as identify stress points that may lead to difficulty during coding and maintenance. (Currently developing OO design metrics.) Method: • Use metrics De and Di and D(G) to identify error-prone modules in a software design. • Use DB (design balance) and DC (design connectivity) to gain a better understanding of a developing system. • Various strategies have been employed by current affiliates to correct or monitor troublesome modules. Goal: • Improve software development processes and products. Design Metrics Dolores and Wayne ZageBall State University

22. Bellcore, Northrop Grumman, Army Research Lab • NG started by using the metrics’ ability to predict error-prone modules in a missile defense system. • Since then, incorporated DM into software development methodology as NG strives to go from Level 3 to Level 4. Calculating the Zages’ metrics is included in the set of requirements for new projects. • DMAA and DMAC now in use at affiliate sites. Technology Transfer Example: Zages Ball State Research Team SEI Level 4 DM DM DM DM Northrop Grumman

23. Bellcore, Northrop Grumman, Army Research Lab • At Bellcore: DM have been excellent predictors of stress points in telecommunications software • DM technology moving from Bellcore research to Bellcore development groups • Bellcore and BSU joint papers • BSU DM team student interns at NGC and Bellcore in Summer 1997 Technology Transfer Example: Zages

24. Motorola Goal: Apply DM technology to identify stress points in SDL designs and assess the utility and effectiveness of design metrics on such designs. Method: Map DM primitives to SDL design artifacts. Calculate metrics and and determine if stress points are error-prone. Metrics Directed Verification of SDL DesignsDolores and Wayne ZageBall State University

25. Description:Previous research: Process Tradeoff Method (PTA) predicts the impact of process improvements on: • development cost • product quality • task schedule Current Project: • Further develop these methods by applying the techniques to address a specific question: What is the impact of implementing the Personal Software Process (PSP) in a real world development setting? Goal: • Accommodate new processes, process changes, and project environments • Develop graphical and quantitative process models which can generally be used to predict the impact of implementing CMM KPAs and other process changes Predicting the Performance of Software Process ImprovementsDavid Raffo, Portland State University

26. Description: Develop innovative application of TQM principles to software project management using software process models and metrics. Approach: Develop a suite of software process and product metrics along with software process models which together can be used in an integrated framework to support project management and process planning decisions. Goal: Develop a systematic approach to help companies achieve the Quantitative Process Management, Software Quality Management, and the Continuous Process Improvement KPAs of the CMM. Analysis of Software Process Models and Metrics: Moving to Levels 4 and 5 of the CMMDavid Raffo & Warren HarrisonPortland State University

27. Description:Traditional reliability estimation methods depend on data from product testing. However, many software technologies are evolving so rapidly that failure rates of existing systems may not provide a good indication of reliability of systems under design and development, even in the early, conceptual design phase. Approach: • Develop models of software reliability based on expert judgement combined with statistical information. Exploit advances in Bayesian probability theory to develop software reliability models which can combine expert estimates of factors influencing reliability with statistical information. Goal: • Develop methods and tools for predicting the reliability of complex software systems during early design. Estimating Software Reliability During Conceptual DesignBruce D’AmbrosioOregon State University

28. Description:Reuse has been widely discussed and promoted in the software industry, but many aspects of current practice has to be modified in order for reuse to be practical. For example, the change of reliability when a component is used in a new environment has not been well investigated. Without some guidelines, we may not have the confidence to reuse a component. How should the component be re-tested, and if necessary be modified? (cont.) Computer-aided Testing for Reusable Software ComponentsMark YangUniversity of Florida

29. Approach: • To study reliability estimation methods under a changing environment • To figure out the number of new test cases needed in order to meet the reliability requirement in a new environment • To build a tool that takes the old testing results, usage records, new environments, and new reliability requirement as inputs and outputs the number of new test cases • To assess software reliability estimation methods from the reuse point of view. Which one is the best choice for reuse? Goal:To build a tool that can certify reusable components based on past testing history and present requirements Computer-aided Testing for Reusable Software Components(cont.)Mark YangUniversity of Florida

30. Description: • Testing distributed systems composed of software components for tolerance to component failures, security and correctness of implementation. • Developing interface based fault injection methods between components and metrics in order to assess the adequacy of testing. Method: • Identify the source of errors and failures in a distributed system • Develop and validate metrics to assess test adequacy • Study the behavior of system under component failure (cont.) Testing Distributed SystemsAditya Mathur and Sudipto Ghosh Purdue University

31. Goal: • Develop a methodology to assess the tolerance of a distributed system to failures occurring in one or more of its components • Develop and assess software fault injection testing methodology for distributed systems • Develop metrics for the assessment of adequacy of tests of distributed systems • Develop a prototype assessment tool for experimentation Testing Distributed Systems (cont.)Aditya Mathur and Sudipto Ghosh Purdue University

32. Bellcore Fault Injection Testing 1994-1997 • Bellcore explored the possibility of using fault injection testing technology for software systems • Study identified research issues involved in fault injection testing and tool design for fault injection testing • Over 3 years Purdue and Bellcore conducted studies on the application of the fault injection testing technique on Bellcore’s systems • Bellcore/Purdue jointly authored paper in Quality Week 97, Europe • Purdue student hired as a summer intern by Bellcore Technology Transfer Examples: Aditya Mathur and Sudipto Ghosh

33. Description: • Traditional methods for software reliability assessment treat a software system as a “whole” and are applicable very late in the development cycle • We have proposed CBRE (Component-Based Reliability Estimation) to overcome the problems associated with traditional methods. • CBRE can also be applied to obtain early estimates of system reliability, i.e., prior to code availability. • The current focus is on • comparing CBRE with Laprie-Kanoun method of component-based reliability estimation • evolving models of inter- and intra-component dependencies, • determining ways of applying CBRE in early stages of development. (cont.) Architecture Based Estimation of Software Reliability Aditya Mathur, Pietro Michielan, Manuela SchionaPurdue University

34. Collaboration: • This project is supported in part by a tie-project grant from the NSF. The tie is with Professor Kishor Trivedi from the Center for Advanced Communications at Duke University. • Other collaborators include: • Dr. Jose Maldonado, University of Sao Paulo, Brazil. • Professor Nozer Singpurwalla, George Washington University • Dr. Alberto Pasquini of ENEA, Rome, Italy Architecture Based Estimation of Software Reliability(cont.)Aditya Mathur, Pietro Michielan, Manuela SchionaPurdue University

35. Industry must increase its contribution to the U.S. R&D enterprise. • Industry must overcome private sector barriers to partnerships. • Industrial R&D must grow. • Industry must focus its research priorities. • Industry must take timely advantage of theleading-edge results coming out of U.S. universities and government labs. *http://nii.nist.gov/pubs/coc_rd/rd_cover.html “Endless Frontier, Limited Resources”*

36. Re-training Courses • 10-week, 400-hour hand-on course in Software Engineering offered to 30 employees of Raytheon during summer of 1998. • Specialized Course Modules • 1-week modules in Design, Testing, WEB-site Construction and Management, Java, and other languages. • Master in Software Engineering • Oregon Associated Universities (http://www.omse.org/) Educational Activities