automated software testing test execution and review n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Automated Software Testing: Test Execution and Review PowerPoint Presentation
Download Presentation
Automated Software Testing: Test Execution and Review

Loading in 2 Seconds...

play fullscreen
1 / 19

Automated Software Testing: Test Execution and Review - PowerPoint PPT Presentation


  • 136 Views
  • Uploaded on

Automated Software Testing: Test Execution and Review. Amritha Muralidharan (axm16u). Contents. What is test execution? Phases of Testing Unit Testing Integration Testing System Testing Test Results Analysis of Regression Tests User Acceptance Testing

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Automated Software Testing: Test Execution and Review


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
contents
Contents
  • What is test execution?
  • Phases of Testing
    • Unit Testing
    • Integration Testing
    • System Testing
  • Test Results Analysis of Regression Tests
  • User Acceptance Testing
  • Defect Tracking and Automated Tracking tool
  • Test Program Status Tracking and EVMS
  • Test Metrics Collection and Analysis
    • White Box
    • Black Box
  • Test Execution Review
what is test execution
What Is Test Execution?
  • Stage 5 of the ATLM
  • Creation of testing environment
  • Tests are carried out on the application
    • Input = test procedures
    • Output = satisfied/altered acceptance criteria
  • Software Problem Reporting
  • E.g. CourseMarker
phases of testing
Phases of Testing
  • Unit, Integration, System and User Acceptance testing.
  • Tests executed according to the Test Plan for each phase.
phases of testing unit testing
Phases of Testing: Unit Testing
  • Consistent with Development Schedule
  • Automated Results Checkout Process
  • White Box Level
    • Involves detailed look at code
  • Best to execute unit tests as soon after the code has been created.
  • Static Analysis
  • Code Profiling
  • Development Test Tools
  • Test Documentation
    • Metric Collection
    • Process Improvement
    • Software development folders (SDF) and Unit Test Evaluation Criteria
phases of testing integration testing
Phases of Testing: Integration Testing
  • “Units that are incrementally integrated are tested together based on control flow”
  • Some integration testing takes place during unit testing
  • Test procedures based on integration test design addressed earlier in the ATLM
  • Results are analyzed and documented
  • Refining Test Procedures
  • Some test scripts can be automated for re-use in system testing
  • End-user approval = end of unit and integration testing
phases of testing system testing
Phases of Testing: System Testing
  • Higher Level
  • Performed by a separate test team
  • May require many individual test procedures
  • Test Execution
    • False Negatives
    • False Positives
  • Software Problem Report
test results analysis of regression tests
Test Results/Analysis of Regression Tests
  • New application baseline
    • Build release notes should accompany the new build
  • Smoke Tests
  • Regression Testing (after defects have been fixed/modified)
    • Investigate high risk areas of code
    • Indicator of developer behaviour
  • Test team investigates areas with larger number of problem reports
  • Analysis is a clear indicator of whether test execution procedures are useful in locating errors.
    • Where more test effort is required
  • System testing is complete when criteria have been met
user acceptance testing
User Acceptance Testing
  • Involves End-user participation
  • Consists of the subset of the suite of tests performed at system level
  • Defects at UAT level will be documented in SPRs
  • Satisfactory resolutions of level 1 and 2 SPRs = conclusion to testing
defect tracking and new build process
Defect Tracking and New Build Process
  • Test team performs problem reporting operations in compliance with a defined process
    • SPR creation
    • E-mail Notification to configuration management(CM) and developers
  • New software is checked via CM tools
  • New software build issuance
  • Defect Priority
    • Fatal, high, medium, low
  • Engineering review Board (ERB)
  • Automated Defect Tracking Tool
    • Ensures reported defects receive proper attention
    • Central defect tracking repository
    • Has several basic characteristics
  • Automated Test Tools
    • Maintain test results/permit automatic generation of test results
  • Recommended that SPRs are generated per life-cycle stage
test program status tracking and evms
Test Program Status Tracking and EVMS
  • Helps to efficiently handle problems that arise during test effort
  • Test engineer must produce meaningful reports based on test plan measures and metrics
    • Test log
    • Test coverage reports
test program status tracking and evms cont
Test Program Status Tracking and EVMS Cont…
  • To effectively monitor testing progress and report to senior management, an earned value approach is used.
  • EVMS = one of the best ways to track progress
  • Tracks value of completed works and compares it to planned costs and actual
    • Provides true measure of schedule and cost status
    • Enables creation of effective corrective actions
  • 4 steps
    • Identify short tasks
    • Schedule each task
    • Assign a budget to each task
    • Measure progress of each task

Earned value for work completed – planned budget = Schedule Variance

Earned value for work completed – actual cost = Cost Variance

test metrics collection and analysis
Test Metrics Collection and Analysis
  • Metrics = key indicators of test coverage progress and quality of test effort
  • Separated into white box and black box testing efforts
  • HOWEVER, gathering too many metrics can be time consuming
test metrics collection and analysis white box testing
Test Metrics Collection and Analysis: White Box Testing
  • Targets application’s internal working
  • Measures the DEPTH of testing
    • Coverage Analysis
  • Code Analysis and Code Profiling e.g. Rational Quantify
    • Objective is to locate complex areas of source code
  • Fault Density (Fd) = Nd/KSLOC
    • Predicts the remaining number of faults
  • Design Complexity Metrics measures the number of ways a module can cal other modules
test metrics collection and analysis black box testing
Test Metrics Collection and Analysis: Black Box Testing
  • Focus on the BREADTH of testing
  • Based on application’s external considerations
  • Each metric is assigned to one of three categories
    • Coverage
    • Progress
    • Quality
test metrics collection and analysis black box testing1
Test Metrics Collection and Analysis: Black Box Testing
  • Coverage
    • Test Coverage
    • System Coverage
    • Functional Coverage
  • Progress
    • Test Procedure Execution Status
    • Error Discovery Rate
    • Defect Aging
    • Defect Fix Rate
    • Defect Trend Analysis
  • Quality
    • Test Success Index
    • Defect Density
    • Test Effectiveness
    • Problem Report Acceptance Criteria Metric
    • Test Automation Metric
test execution review
Test Execution Review
  • Stage 6 of ATLM
  • Review test performance e.g. metrics
    • Helps determine enhancements to be made
  • Document well performed activities
  • Lessons Learned should be acknowledged/documented
    • Aim to ensure best practices are used
  • Integrate a continual iterative learning process
    • Encouraged as improvement opportunities
  • “check pulse”
  • Documentation should be easily accessible to all team members
  • Corrective actions suggested must be evaluated
  • Represents he need for process improvement