Creating Test Reports Test report documents the result of software testing.

Creating Test Reports • Test report documents the result of software testing. • Test reporting involves reporting the results of test execution of a software application to relevant stakeholders. • Test report data is used to fix defects and track the status of the software application under test. Managerial Aspects of Testing

Test Reporting Tools • Help a software tester to effectively generate a test report • These tools enable the software tester to do the following: • Generate test report formats • Provide statistical output of test results • Track the defects found in a software application during testing • Record the result of a test • The following are some test reporting tools: • Analysis Tools • Represent the result of a test in statistical form • Help in analysis of test data • Examples:JMP 5.1 and SysStat 11 • Database Tools • Store the result of a software test Managerial Aspects of Testing

Test Reporting Tools(Contd.) • Defect Tracking Tools • Help to track the resolution of defects • Example:Bugzilla • Test Reporting Formats • Software packages like word processors are used to generate formats for test reports • Sample Test Report Managerial Aspects of Testing

Test Reporting Standards • Each test report should provide the following information: • Details of the defect detected • Type of test performed • Scope of the test • Environment and condition under which defect is detected • Inconsistency between actual and expected result • Sample test report Managerial Aspects of Testing

Test Reporting Standards(Contd.) • Problem Statement:Consider a software application, AutoCal that performs the function of a calculator. A test is done on the application to check the add and subtract functions of the application. While testing the following errors are found: • Subtraction of two digit numbers gives the product of the numbers. • Subtracting a greater number from a lesser number does not show a negative result. • Adding two three digit numbers gives the square of the first number. • Develop a test report for the software test of AutoCal application. Managerial Aspects of Testing

Test Reporting Standards(Contd.) • Solution Managerial Aspects of Testing

Statistical Analysis Tools • Pareto Charts • Ranks the causes of a problem from the most significant to the least significant • Analyzing data using a pareto chart enables a software tester to: • Focus on critical issues of a software application • Prioritize the problems Managerial Aspects of Testing

Statistical Analysis Tools(Contd.) • Histogram • Displays the frequency of occurrence of data • Provides an easy way to analyze and interpret data Managerial Aspects of Testing

Statistical Analysis Tools(Contd.) • Scatter Plot • Shows the relationship between two variables Managerial Aspects of Testing

Statistical Analysis Tools(Contd.) • Scatter Plot Managerial Aspects of Testing

Statistical Analysis Tools(Contd.) • Cause and Effect Diagram • Careful analysis of a cause and effect diagram enables a software tester to: • Identify the various causes of a problem • Determine the root cause of a problem Managerial Aspects of Testing

Statistical Analysis Tools(Contd.) • Cause and Effect Diagram Managerial Aspects of Testing

Statistical Analysis Tools(Contd.) • Problem Statement:Virgtech Inc., a software development team is developing a voice recognition system. The project has already crossed its first deadline and does not seem to meet the forthcoming deadline. Use a cause and effect diagram to determine the causes for the delay in completion of the project. Managerial Aspects of Testing

Statistical Analysis Tools(Contd.) • Solution Managerial Aspects of Testing

Analyzing Statistical Data • Exploratory Analysis • Uses graphical and descriptive statistical analysis to: • Maximize insight into a dataset • Learn about the structure of a dataset • Extract information related to variables in a dataset • Test underlying assumptions • Highlight unusual conditions Managerial Aspects of Testing

Analyzing Statistical Data(Contd.) • Exploratory Analysis • Example: Consider the data set in the following table: Managerial Aspects of Testing

Analyzing Statistical Data(Contd.) • Exploratory Analysis • After quantitative computation of the preceding dataset the following result is obtained: • Mean of X is 9 • Mean of Y is 7.5 • Intercept of the line XY is 3 • Slope of the line XY is 0.5 • Correlation of X and Y is 0.816 Managerial Aspects of Testing

Analyzing Statistical Data(Contd.) • Exploratory Analysis • Graphical representation of the dataset helps to understand its underlying structure. • Analysis of dataset in the preceding scatter plot infers that the tested variables are not closely related Managerial Aspects of Testing

Analyzing Statistical Data(Contd.) • Cluster Analysis • Classifies a set of observations into two or more mutually exclusive groups • Example:The budget and schedule of projects of a company are plotted on a scatter plot. • Analysis of the following scatter plot infers that projects behind schedule and over budget share similar characteristics Managerial Aspects of Testing

Analyzing Statistical Data(Contd.) • Regression Analysis • Used to estimate the nature of relationship between two variables • Example:The scatter plot below represents the relationship between the complexity of a software application and the cost to test the same. Managerial Aspects of Testing

Analyzing Statistical Data(Contd.) • Regression Analysis • The following information can be gathered after analyzing the preceding scatter plot: • As the complexity of the software application increases the cost to test the same also increases. • The cost to test any software application depends upon the complexity of the software application. Managerial Aspects of Testing

Risk Management Process • Risks are events which may or may not occur and whose occurrence may have harmful effects on the project • A risk has the following two features associated with it: • Probability • Loss • Risk Management is the process of identifying, controlling and eliminating or minimizing risks that may affect a project • Risk Management comprises of the following: • Risk Identification • Risk Analysis • Risk Mitigation Managerial Aspects of Software Testing

Risk Management Process(Contd.) • The following are the two major decisions taken during risk management: • Deciding on actions that can prevent risks from occurring • Deciding on actions to tackle the risks that materialize Managerial Aspects of Software Testing

Identifying Risks in Software Testing • Risk Identification is the process of identifying risks that might affect the project and its test process and documenting their features. Managerial Aspects of Software Testing

Classifying Risks • Software Risks • Risks associated with a software application being developed • The following are the factors that lead to software risks: • Skill of developers • Disgruntled employees • Poorly defined project objectives • Software risks are categorized into the following: • Project Risks:threaten the project plan as they affect the project schedule and the cost of the project • Technical Risks:threaten the quality of the software being developed Managerial Aspects of Software Testing

Classifying Risks(Contd.) • Business Risks • Threaten the feasibility of the software being developed • Some contenders of business risks are: • Market risk: Developing an excellent software product that has a few takers • Technology change: Underlying technology of the software product being superseded by a new technology • Product competition: Launch of a competitive product in the market before the completion of the project • Management risk: Loss of management support due to difference in opinion • Budget risk: Project cost estimated to exceed the planned budget Managerial Aspects of Software Testing

Classifying Risks(Contd.) • Premature Release Risks • Occur when software application is released in the market before it is totally complete • The following can be the reasons for premature release risks: • Deadline for the completion of the project has exceeded • Equally competitive product is about to be released by another company • Testing Risks • Affect the process of testing a software application • The following factors can lead to testing risks: • Insufficient budget • Incompatible test environment • Inappropriate tools and procedures • Inexperienced testers Managerial Aspects of Software Testing

Tools and Techniques for Identifying Risks • Checklists • Used to identify potential risks to a project and test process • Include a list of most probable risks that can be encountered by any project and a test process • The following are the generic categories that can be included in a risk identification checklist: • Product size • Business impact • Development environment • Technology used • Staff characteristics • Customer characteristics Managerial Aspects of Software Testing

Tools and Techniques for Identifying Risks(Contd.) • Interviews • People directly or indirectly associated with the project are contacted on a one to one basis to collect information • Software testers who have tested a project or worked on a test environment similar to the current project can also be interviewed • Questionnaires • Contain a list of questions related to a particular risk • Help in the identification of a majority of risks associated with a test process and the project • Sample questionnaire Managerial Aspects of Software Testing

Tools and Techniques for Identifying Risks(Contd.) • Brainstorming • Used to generate a large number of ideas in a short span of time • It is used in the following cases: • Participation of the entire group is desired • Broad range of options or ideas are desired • Intuitive, creative, or unconventional ideas are desired • Observations • Effective method used to collect information related to risks associated with a project and its test process • Reviews • Frequent and periodic reviews of projects are done to identify the potential risks to similar projects and their test processes Managerial Aspects of Software Testing

Analyzing Risks in Software Testing • Risks analysis is used to assess the following: • Probability of occurrence of the identified risks • Impact of the identified risks on the project and test process • Risk analysis comprises of three main tasks: • Risk Probability • Risk Impact • Risk Factor Managerial Aspects of Software Testing

Risk Probability • Determines the probability of occurrence of risks identified for the project and its test process • A risk may have a high probability in one project and a low probability in another project depending on the nature the project. • During the risk analysis process, the result of each task is logged in a risk analysis table. • Risk Analysis Table Managerial Aspects of Software Testing

Risk Impact • Degree of impact that a risk has on a project and test process • Determined on the basis of the probability of occurrence of the risk in the project and during the testing of the project • The impact of the risk on the following aspects of the project need to quantified and graded: • Cost • Schedule • Quantity Managerial Aspects of Software Testing

Risk Factor • Used to prioritize the risk to a project relative to the other risks identified for the project • Determined on the basis of the probability of occurrence and impact of the risk on the project • Risk Factor = Probability of risk * Impact of risk Managerial Aspects of Software Testing

Risk Mitigation • Risk mitigation is the planning process wherein alternative ways are considered to do the following: • Avoid the identified risks from occurring • Reduce the impact of the risk on the project • There are three possibilities for which the risk mitigation strategies are developed: • Risk Avoidance • Risk Monitoring • Contingency Planning Managerial Aspects of Software Testing

Risk Avoidance • The following are the tasks that a project team performs to avoid the occurrence of risks: • Preparation of the risk plan before the commencement of the project • Identification of potential risks to a project and its process of testing • Prioritization of the identified risks to a project and its process of testing. • Preparation of a plan for managing risks. This plan is called the risk management plan • The risk management plan is a document that describes how the risk management processes will be carried out during the project Managerial Aspects of Software Testing

Risk Avoidance(Contd.) Risk Management Plan Managerial Aspects of Software Testing

Risk Monitoring • Risk monitoring has three primary objectives: • To assess whether the predicted risks really occur • To ensure that the risk avoidance steps defined for a particular risk are being implemented • To collect information that can be used for risk analysis in future • Risk monitoring activities commence as soon as the project development begins • Risk monitoring is done periodically Managerial Aspects of Software Testing

Contingency Planning • Contingency planning refers to the development of a plan that identifies alternative strategies used to ensure project success if specified risk occurs • The following are the steps to develop a contingency plan: • Identify and prioritise risks for which contingency planning is necessary • Group similar risks together • Consider possible actions and select the most appropriate action. • Divide actions into the following categories: • Taken prior to the event • Taken in the event of a failure • Necessary if the event does not occur • List possible causes of the risk • Create documentation to be followed in the event of occurrence of a risk • Consider training the concerned people Managerial Aspects of Software Testing

Creating Test Reports Test report documents the result of software testing.