The Paradigm Shift – Testing 3.0. Arun Kumar Singh. Agenda. Testing 1.0 Testing 2.0 The changing market space The new needs Testing 3.0 The 3 type of supplier and the opportunity for them. Software fails. Source – IEEE Spectrum. Testing 1.0. Definition :
Arun Kumar Singh
Source – IEEE Spectrum
Software Testing is the process of executing a program or system with the intent of finding errors. [Myers79]
Part of debugging process , dynamic change
Processes (typically) undocumented
Tending to be driven in an ad hoc uncontrolled and reactive manner by users or events.
chaotic or unstable environment for the processes
one team. no differentiation. Development team did testing
Success due to competence and heroics of the people in the organization and not on the use of proven processes.
products and services that work; however, they frequently exceed the budget and schedule of their projects
Any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. [Hetzel88]
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test.
Software testing can be stated as the process of validating and verifying that a software program/application/product:
1. ^Exploratory Testing, Cem Kaner, Florida Institute of Technology, Quality Assurance Institute Worldwide Annual Software Testing Conference, Orlando, FL, November 2006
testing processes are
well characterized and understood, part of SDLC or PLC
described in standards, procedures, tools, and methods. improved over time
Using precise measurements, management can effectively control the software development effort. ……. identify ways to adjust and adapt the process to particular projects without measurable losses of quality or deviations from specifications.Specialized tools and techniques ( statistical models ) are used
planned & dedicated
well defined and separate testing organization. Exec mgmt support.
testing G&Os aligned to corporate vision and mission
Business unit A in geo A
Business unit B in geo B
Business unit C in geo C
Demand consolidation & forecasting
QA Consulting/Best Practice Exchange/domain & product expertise
Shared test environments
^ McConnell, Steve (2004). Code Complete (2nd ed.). Microsoft Press. pp. 29. ISBN0-7356-1967-0.
Typical Software Quality Factors [Hetzel88]
Tracking/measuring and defining correct outcomes & benefits.
Lack of management support (little or no visibility into the effectiveness )
Resistance from suppliers.
Organization change... Limited tools to encourage the managed approach with HR metrics
Defining & doing the right amount of testing
Moving from technology to business process testing
Cost to quality measures
Disconnected from the end customer experience
Offshoring : communications, right skill sets, management-vendor relations, global infrastructure, ever increasing expectations/innovation
Political / Legal factors
Rapid change in business environment
Overall: Discrete or standalone global testing services spending reached $9.4 billion in 2010,
Growth: Worldwide growth estimated at a five-year CAGR of 15.4%. Some analyst have
Geo-wise: United States has led other regions in adoption of discrete testing services, in terms of both traditional and newer cloud-based testing services models…….
an increase in regional interest and adoption of these services in Canada, Latin America, Western Europe,and parts of Asia/Pacific is seen.
Message for testing
huge growth – plan for it. Be ready with right resources
outcome driven delivery and financial models would be needed
move from technology to business process
Source : IDC
Impact on Testing:
Source : Gartner
Trends that are key to growth in this services segment:
Emerging/Future Application Testing Services Trends:
Source : IDC
Source : IDC
IDEAL: initiating, diagnosing, establishing, acting and learning model ( carnegie mellon university )