Software Testinf Basics Day 1

1. Software Testing Session Richa Goel

2. Introduction – Fundamentals of Testing

3. What is Testing? What is Software Testing?

4. What is Software Testing? Software testing can be stated as the process of validating and verifying that a software program/application/product: • meets the requirements that guided its design and development; • works as expected

5. What is testing? Testing is a process aimed at: • Finding defects in a controlled manner • Detecting the level of quality of the test object • Demonstrating the gap between specifications and the actual product • Demonstrating that the end product functions as called for in the requirements

6. What is testing? Definition: • Testing is process of trying to discover every conceivable fault or weakness in a work product. • Testing is a process of executing a program with the intent of finding an error. • A good test is one that has a high probability of finding an as yet undiscovered error.

7. Why Software Fail?

8. What is a “bug”? • Error: a human action that produces an incorrect result • Fault: a manifestation of an error in software • also known as a defect or bug • if executed, a fault may cause a failure • Failure: deviation of the software from its expected delivery or service • (found defect) Failure is an event; fault is a state of the software, caused by an error

9. Error - Fault - Failure A person makes an error ... … that creates afault in thesoftware ... … that can causea failurein operation

10. Reliability versus faults • Reliability: the probability that software will not cause the failure of the system for a specified time under specified conditions • Can a system be fault-free? (zero faults, right first time) • Can a software system be reliable but still have faults? • Is a “fault-free” software application always reliable?

11. Why do faults occur in software? • software is written by human beings • who know something, but not everything • who have skills, but aren’t perfect • who do make mistakes (errors) • under increasing pressure to deliver to strict deadlines • no time to check but assumptions may be wrong • systems may be incomplete • if you have ever written software ...

12. What do software faults cost? • huge sums • Ariane 5 ($7billion) • Mariner space probe to Venus ($250m) • American Airlines ($50m) • very little or nothing at all • minor inconvenience • no visible or physical detrimental impact • software is not “linear”: • small input may have very large effect

13. What do software faults cost? • huge sums • Ariane 5 ($7billion) • Mariner space probe to Venus ($250m) • American Airlines ($50m) • very little or nothing at all • minor inconvenience • no visible or physical detrimental impact • software is not “linear”: • small input may have very large effect

14. Absence of errors • If we don't find defects does that mean the users will accept the software? • We must consider the customers for software • The people using soft-ware are more interested in software to meet their needs. Richa Goel

15. Think on the same! • Open Ms Paint application and identify one defect in Ms Paint.

16. Absence of Testing… Richa Goel

17. Defect vs Bug • Error : Is an undesirable deviation from requirements • Bug : Is an error found BEFORE the application goes into production • Defect :Is an error found AFTER the application goes into production • An error would be caught by the developer itself since mostly error is generated out of the coding, and it will caught in the unit testing stage itself. • A fault is some mistake done by the development or testing team but corrected before the release. • A defect is the term which states anomoly between the expected and actual result.

18. Most Common Software problems • Incorrect calculation • Incorrect data edits & ineffective data edits • Incorrect matching and merging of data • Data searches that yields incorrect results • Incorrect processing of data relationship • Incorrect coding / implementation of business rules • Inadequate software performance

19. Confusing or misleading data • Software usability by end users & Obsolete Software • Inconsistent processing • Unreliable results or performance • Inadequate support of business needs • Incorrect or inadequate interfaces with other systems • Inadequate performance and security controls • Incorrect file handling

20. Objectives • Objective of testing is to find all possible bugs (defects) in a work product • Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn't or things don't happen when they should.

21. Objectives of testing • Executing a program with the intent of finding an error. • To check if the system meets the requirements and be executed successfully in the Intended environment. • To check if the system is “ Fit for purpose”. • To check if the system does what it is expected to do.

22. Objectives of testing • A good test case is one that has a probability of finding an as yet undiscovered error. • A successful test is one that uncovers a yet undiscovered error. • A good test is not redundant. • A good test should be “best of breed”. • A good test should neither be too simple nor too complex.

23. Objectives for Testing • Determine the software satisfy requirements. Some of the testing we do is focused on checking product against the specification for the product. We review the design to see if it meets requirements. • Demonstrate software is fit for purpose: looks at whether software does enough to help the user to carry out their tasks. • Detect defects

24. Need for Testing • Testing ensures that what you get in the end is what you wanted to build. • enhances the integrity of a system by detecting deviations in design and errors in the system.  In Short, • To check the reliability of the software.  • To be ensure that the software does not contain any bug which can become a reason for failure. • The software is made according to its specification

25. Need for Testing Common things that creates the need for testing: • Communication gaps between the requirement specifier and the developer • Not proper understanding of all the complexities of the system • Inadequate requirements gathering

26. Why is testing necessary ? • Software systems context are an increasing part of life, from business applications to consumer products • Causes of software defects: Defects occur because human beings are fallible. Failures can be caused by environmental conditions • Role of testing in software development, maintenance and operations: Rigorous testing of systems and documentation can help to reduce the risk of problems occurring in an operational environment • Testing and quality: With the help of testing, it is possible to measure the quality of software in terms of defects found • How much testing is enough? Deciding how much testing is enough should take account of the level of risk

27. Scenario • If the company website has some spelling mistakes in the text, potential customers may be put off the company as it looks unprofessional. • If a software program miscalculates application quantities, the effect could be very significant: suppose a decimal point is wrongly placed so that the application rate is 10 times too large.

28. Testing and quality • Testing helps us to measure the quality of software in terms of the number of defects found, the tests run, and the system covered by the tests. • Projects aim to deliver software to specification. • For the project to deliver what the customer needs requires a correct specification. • Validation : The delivered system must meet the specification ('is this the right specification?') • Verification: Process Based ('is the system correct to specification?'). • The customer also wants the project to be within budget and timescale.

29. Testing and quality • testing measures software quality • testing can find faults; when they are removed, software quality (and possibly reliability) is improved • what does testing test? • system function, correctness of operation • non-functional qualities: reliability, usability, maintainability, reusability, testability, etc.

30. Quality... it’s all about the End-User Does this software product work as advertised? Functionality, Performance, System & User Acceptance ... Testing Will the users be able to do their jobs using this product? Installability, Compatibility, Load/Stress ... Testing Can they bet their business on this software product? Reliability, Security, Scalability ... testing Richa Goel

31. So why is testing necessary? • because software is likely to have faults • to learn about the reliability of the software • to fill the time between delivery of the software and the release date • to prove that the software has no faults • because testing is included in the project plan • because failures can be very expensive • to avoid being sued by customers • to stay in business

32. In other Words… Testing is a process used to identify the correctness, completeness and quality of developed computer software. Testing, apart from finding errors, is also used to test performance, safety, fault-tolerance or security. Software testing is a broad term that covers a variety of processes designed to ensure that software applications function as intended, are able to handle the volume required, and integrate correctly with other software applications. Richa Goel

33. Exercise 1 A field can accept integer values between 20 and 50. What tests should you try? Richa Goel

34. Common answers to the exercise would include: A Test Ideas List for Integer-Input Tests

35. Software Testing Terminologies • Accuracy: The capability of the software product to provide the right or agreed results or effects with the needed degree of precision. • Actual result: The behavior produced/observed when a component or system is tested. • Behavior: The response of a component or system to a set of input values and preconditions. • Component: A minimal software item that can be tested in isolation. • Debugging: The process of finding, analyzing and removing the causes of failures in software.

36. Software Testing Terminologies • Deliverable: Any (work) product that must be delivered to someone other that the (work) product’s author. • Error: A human action that produces an incorrect result. • Expected result: The behavior predicted by the specification, or another source, of the component or system under specified conditions. • Fail: A test is deemed to fail if its actual result does not match its expected result. • Failure: Actual deviation of the component or system from its expected delivery, service or result.

37. Software Testing Terminologies • Integration: The process of combining components or systems into larger assemblies. • Pass: A test is deemed to pass if its actual result matches its expected result. • Priority: The level of (business) importance assigned to an item, e.g. defect. • Severity: The degree of impact that a defect has on the operation of a component or system. • Tester: A technically skilled professional who is involved in the testing of a component or system. • Use Case: The specification of tests that are conducted from the end-user perspective. Use cases tend to focus on operating software as an end-user would conduct their day-to-day activities.

38. How much testing? • It depends on RISK • risk of missing important faults • risk of incurring failure costs • risk of releasing untested or under-tested software • risk of losing credibility and market share • risk of missing a market window • risk of over-testing, ineffective testing

39. Any Questions? If you want you can collect your questions for the end of the session as well.

40. Quiz • When what is visible to end-users is a deviation from the specific or expected behavior, this is called: • an error • a fault • a failure • a defect • a mistake

41. Quiz Deciding How much testing is enough should take into account :- • Level of Risk including Technical and Business product and project risk. • Project constraints such as time and budget. • Size of Testing Team. • Size of the Development Team. • i,ii,iiiare true and iv is false • i,,iv are true and ii is false • i,iiare true and iii,iv are false • ii,iii,ivare true and i is false

42. Quiz • Which of the following will be the best definition for Testing • The goal / purpose of testing is to demonstrate that the program works. • The purpose of testing is to demonstrate that the program is defect free. • The purpose of testing is to demonstrate that the program does what it is supposed to do. • Testing is executing Software for the purpose of finding defects.

43. Exercise 2 • How many tests would you need to do to test a one-digit numeric field?

44. Answer • Valid case • 10 - numeric values • We need to ensure that all the invalid values are rejected. • Invalid case • 26 uppercase alpha characters, • 26 lower case • 6 special and punctuation characters • a blank value. • So there would be at least 68 tests for this example of a one-digit field.

45. Seven Testing Principles

46. Seven Testing Principles • Testing shows presence of defects. • Exhaustive testing is impossible. • Early Testing. • Defect Clustering. • Pesticide Paradox. • Testing is context dependent. • Absence of errors is fallacy.

47. Is the software defect free? Testing Principle 1 - Testing shows presence of defects Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness. • We cannot say 'All swans are white'. • However, as soon as we see one black swan we can say 'Not all swans are white'. • In the same way, however many tests we execute without finding a bug, we have not shown 'There are no bugs'. • As soon as we find a bug, we have shown 'This code is not bug-free'.

48. How much testing is enough? Testing Principle 2 - Exhaustive testing is impossible Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts. • Assessing and managing risk is one of the most important activities in any project, and is a key activity and reason for testing. • Deciding how much testing is enough should take account of the level of risk, including technical and business risks related to the product and project constraints such as time and budget.

49. Re-visit Exercise 2 • How many tests would you need to do to COMPLETELY test a one-digit numeric field?

50. Scenario • The immediate question is, 'What you mean by test completely?'