Software testing and development for intended quality

1. Software testing and development for intended quality Tero Pesonen

2. Objective • To show how software development, software testing and intended quality depend on one another. • The research area includes, in addition to traditional software development and service models, also the emerging XaaS (Everything as a Service) architectures, technologies and service models. • The project results help the participating companies in improving the efficiency of their quality management and software testing and hence the efficiency of their software development as a whole. • Testing techniques • Testing as a service

3. Intended Software Quality Software Development • Software Products • ISO/IEC 12207 • ISO/IEC 15504 • New Services • SaaS ISO/IEC 25010, Software Product Quality Software Testing ISO/IEC 29119, Testing Spice, IEEE Std 1012 StaaS Research Problem Assessment framework through which each OU is evaluated

4. Preliminary Findings • In the software developing OU, requirements management plays an important role in achieving intended quality. • Requirements act as the basic communication medium through which the customer/user, developers and testers concretize different aspects of quality. • Requirements are never throughout, and different stakeholders interpret them differently. Requirements change. • Successful requirements management contributes positively to achieving the targeted quality by reducing overhead caused by doing wrong work.

5. Preliminary Findings • Building ”good” quality is different to building targeted type and level of quality as efficiently as possible. • For many organizations, concretizizing intended quality attributes (also non-functional) is not yet as well implemented as actual development and testing activities are. • Many organizations are also developer centric, and here QA emphasizes functional quality. • Intended quality appears therefore to be also a knowledge transfer issue within the development process, not just a testing or software development challenge, as often perceived.

6. Preliminary Findings • Agile methods favour direct, short development cycles that produce immediate feedback on quality for testers, developers and users. • Agility establishes an efficient communication channel. • Non-agile methods seem to require stricter, more formal quality management approaches for similar level of success. • Large, complex, multistakeholder projects do not benefit from agility to the same extend, as the communication overhead is rather increased than reduced in such a context. • Well-defined waterfall-like processes may fare better.

7. Observations from the data • Developers and testers need to be able to communicate directly, face-to-face, without effort; physical co-location is seen very important for mutual productivity. • Developers should not review their own code; one becomes blind to one’s own mistakes. That time is better spent reviewing a colleague’s work. • Requirements management solutions (e.g. Shared management centers) seem a good starting point in improving the efficiency of the overall development process. • Note: How is non-functional quality communicated in the solution? • The same organization may need to apply different processes to different types of development work, not just have all development governed by one procedure. • Rapid, explorative development work benefits from agility. • Maintenance work can be more strictly phased and documented.

8. Observations from the data • In projects were testing is done as a service, it is paramount that no communication obstacles are created between testers, managers and developers, despite them being part of different organizations. • Knowledge transfer between stakeholders should be considred already in project planning. • Develoeprs and testers should be communicated the formal aspects and be inquired for feedback; they observe, notice and draw conclusions – and frustrate when things do not work. • Cloud architectures support very quick development for targeted quality with module re-use and well-defined API’s • But force vendor lock-in. • Offer poor portability to non-cloud platforms if software grows.

9. Publication 1 / STX • Research question: How do developing organizations build intended quality? Sub questions: • Do different size and type of OU’s build quality differently? • Do cloud environments affect how quality is built? • What is the role of testing methods and tools for quality? • Conference paper. Contents: 1. Introduction 2. Related research 3. Research process • Qualitative, Grounded Theory method. • 18 STX interviews + 10 interviews of supplementary data. • xx sheets of transcripted data.

10. Publication 1 / STX 4. Results 4.1 Quality is communicated as requirements, which are translated into architecture and then into software code. Testing ensures the requirements are, first, correct, and, second, being addressed correctly. 4.2 For most organizations, the potential in improved requirements management seems currently greater than that provided by better testing techniques or other kinds of testing tools. 4.3 Process improvement seems to follow improved requirements management. 4.4 Service-oriented OU’s view quality less formally than product-oriented ones. Knowledge transfer plays a role in organization’s growth, and has to be reckoned with. 4.5 Cloud environments facilitate certain kinds of development work and aid in achieving targeted quality efficiently, but not without major compromises. 5. Discussion and Summary.

11. Research: Next Steps • Assessment of testing • On-going work: • Master’s Thesis by Aoke Li on the status of standardization in China. • Due to February 2012. • A new Master of Science Thesis study for STX will be started this spring. • Research topics: • Software product lines (ISO/IEC 2655x.) • Distributed global software development. • Cloud laboratory (external project with China.) • Quantitative survey.