1 / 31

Software Self-Testing

Software Self-Testing. This work has been supported by the European Social Fund within the project « Support for Doctoral Studies at University of Latvia ». Edgars Diebelis Prof. Dr. sc. comp. Jānis Bičevskis. Self-Testing.

ranit
Download Presentation

Software Self-Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SoftwareSelf-Testing ThisworkhasbeensupportedbytheEuropeanSocialFundwithintheproject «SupportforDoctoralStudiesatUniversityofLatvia». Edgars Diebelis Prof. Dr. sc. comp. Jānis Bičevskis

  2. Self-Testing • Ability to execute stored test examples to ensure that software functionality is correct. • Testing approach • Manual testing; • Automated testing with testing tools; • Self-Testing.

  3. Content • Problem statement. • Concept of Self-Testing. • Implementation of Self-Testing. • Comparison of the Self-Testing Concept with Conventional Testing Support Tools. • Efficiency Measurements of Self-testing. • Conclusions.

  4. ProblemStatement • „Computing systems’ complexity appears to be approaching the limits of human capability” (Kephart, J., O., Chess, D., M.). • IBM Autonomic Computing Manifest, 2001 • outlined four key features that characterise autonomic computing. • SmartTechnologyapproach, 2007 • identifiedseventypesofthesmarttechnology; • self-testingisoneofthetypeofthesmarttechnology.

  5. ConceptofSelf-Testing

  6. Self-TestingFunctionality • Self-testing contains two components: • Test cases of system’s critical functionality to check functions, which are substantial for system using; • Built-in mechanism (software component) for automated software testing (regression testing) that provides automated running of test cases and comparing test results with standard values.

  7. Test storage file (XML) Test storage mode 1. 2. 3.

  8. Test storage file (XML) Test storage file (XML) Self-Testing tool Self-Testing mode 1. 2. 4. 6. 3. 5.

  9. Test Points • Test point is a programming language command in the software text. • Test point ensures that: • particular actions and field values are saved when storing tests; • the software execution outcome is registered when tests are executed repeatedly. • By using test points, it is possible to repeat the execution of system events.

  10. Test Point Types • Test point types: • input field test point; • comparable value test point; • system message test point; • SQL query result test point; • application event test point; • test execution criteria test point; • self-testing test points; • etc. 10

  11. ImplementationofSelf-Testing

  12. Build of Self-Testing • Self-Testing tool modules: • Self-Testing module; • Self-testing test management module. 12

  13. Self-Testing in use

  14. Comparison of the Self-Testing Concept with Conventional Testing Support Tools

  15. Testing Tools Selection • Automated Testing Institute (ATI) opinion. • “ATI Automation Honors ” awards. • Since May 2009, the ATI has been publishing its magazine Automate Software Testing. • Website has a list of 716 automated testing tools. • ATI annual conference on automated testing (Verify/ATI). • ~8000 registred users.

  16. Testing Tools • TestComplete • thebestcommercialautomatedfunctionaltestingtoolin 2010 • FitNesse • thebestopensourcecodeautomatedfunctionaltestingtoolinthe NET sub-categoryin 2010 • Ranorex • thebestcommercialautomatedfunctionaltestingtoolinthe NET andFlash/Flexsub-categoriesin 2010 • T-Plan Robot • thebestopensourcecodeautomatedfunctionaltestingtoolinthe Java sub-categoryin 2010 • RationalFunctionalTester • in2009 and 2010, it was a finalistamongthebestcommercialautomatedfunctionalandperformancetestingtools • HP Unified Functional Testing Software • thebestcommercialautomatedfunctionaltestingtoolin 2009 • Selenium • thebestopensourcecodeautomatedfunctionaltestingtoolin 2009 and 2010

  17. Criteria for Comparison • Testing method (TM) • Test automation approach (TAA) • Test automation framework (TAF) • Testing level • Test recording and playback • Desktop applications testing • Web applications testing • Services testing • Database testing • Testing in production environment • System user can create tests • Simultaneous running of several tests • Performing simultaneous actions • Identifying the tested object • Test result analysis • Test editing • Screenshots • Control points • Object validation • Object browser • Test log • Test schedule planner • Identification of the end of command execution • Plug-ins and extensions • etc.

  18. Comparison

  19. Comparison II 20

  20. Self-Testing further development • New Test automation frameworks. • Test editor and log. • Object validation and object browser. • Load, stress and other testing levels. • Web applications and services testing. • Additional platforms. • Plug-ins and extensions. • Integration with external environment testing.

  21. Self-Testing advantages • White box testing. • Testing in production environment. • Users without in-depth IT knowledge to define and run test cases. • Testing external interfaces. • Perform system testing without specific preparation for running the test.

  22. EfficiencyMeasurementsofSelf-testing

  23. Efficiency Measurements Retrospective analysis of incident notifications in a real project. It is not possible to apply and compare two different concepts in the same conditions. Analysis of incident notifications (1,171 in total) in the CSAS in the period from July 2003 to 23 August 2011. Subjective opinion; however, the high number of incident notifications and the statistics do reflect trends. 24

  24. Statistics of Incident Notifications 25

  25. Bugs Unidentifiable by the Self-testing 26

  26. Bugs Identifiable by the Self-testing 27

  27. Conclusions

  28. Conclusion • self-testing approach offers not only options equal to those offered by other globally recognized testing support tools; moreover, self-testing additionally offers options that other testing tools do not possess: • testing external interfaces; • testing in production environment; • testing with the white-box method; • possibility for users without IT knowledge to capture tests. • Testing support is part of systems developed and it is available throughout the entire life cycle of software.

  29. Conclusion II • Self-testing changes the testing process by considerably broaden the role of the developer in software testing. • Self-testing requires additional work to include the self-testing functionality in the software and to develop critical functionality tests. • Self-testing saves time by repeated (regression) testing of the existing functionality. • Implementation of the self-testing functionality is useful in incremental development models, in particular in systems that are gradually improved and maintained for many years.

  30. ThankYou!

More Related