Software Testing in the Cloud Leah Riungu-Kalliosaari
Contents • Cloud computing • Cloud Computing use and distribution • Nordic countries • Europe • USA • STX Research: Software Testing in the Cloud
Cloud Computing • Cloud Computing represents a collective term for pay-per-use IT services that are delivered over the internet. • On-demand access to services on a pay-per-use model. Source: Nordic Public Sector Cloud Computing – a discussion paper, Nordic Council of Ministers
Nordic Public Sector • There are challenges • Scarce resources • Increased pressure on budgets • Need for more efficiency and innovativeness • Deliver more with less • Increased access to broadband and mobile devices • National IT strategies • Focus on cost efficient IT • Deliver value to the end users Source: Nordic Public Sector Cloud Computing – a discussion paper, Nordic Council of Ministers
Benefits of cloud computing • Cost effectiveness • Reduced capital and maintenance costs • On-demand self-service • Reduced (unnecessary) interaction with service providers • Scalability • Flexibility and pay-per-use • Quality of Service • Possible to monitor, control and report resource usage • Innovation • Cloud architecture supports services in different systems and organizational barriers. • Quick time to market, reliability and business continuity, efficiency, green savings, cheaper security tools Source: Nordic Public Sector Cloud Computing – a discussion paper, Nordic Council of Ministers
Cloud Computing Use and Distribution • Nordic countries are generally ranked highly in e-readiness indices. • Sweden established an eGovernment strategy in 2009 • The Finnish government has a digital agenda for 2011-2020 which includes cloud computing as one of the initiatives • Iceland has got an e-strategy called “Iceland the e-Nation” Source: Nordic Public Sector Cloud Computing – a discussion paper, Nordic Council of Ministers
Cloud Computing Use and Distribution • Denmark has a strategy focused on renewing digital services especially in the public sector. • Norway is working on a national digital agenda • In Europe, cloud services are expected to generate about EUR 35billion by 2014 • USA has a Cloud First Policy • Every federal agency will identify three “must- move” services within three months and move one of those services to the cloud within 12 months and the remaining two within 18 months. Source: Nordic Public Sector Cloud Computing – a discussion paper, Nordic Council of Ministers
Nordic Cooperation on Public SectorCloud Computing • Knowledge sharing can help in overcoming non-technical barriers e.g. addressing legal and regulatory issues • Develop a common view on security and legal issues • Improve the buying power in the Nordic region • Improve procurement processes for cloud-based services • Define common demands and standards • Attract data centres • Encourage public innovation Source: Nordic Public Sector Cloud Computing – a discussion paper, Nordic Council of Ministers
Overview • The study looks at an intersection of cloud computing and software testing • Applications are tested as services by use of cloud- based resources. • Daily operation, maintenance, and testing support through web-based browsers, testing frameworks and servers • Testing is seen as an arena for piloting cloud computing adoption
Objective To understand how organizations can successfully use the cloud for testing. Observe the adoption of cloud computing in different organizational contexts Impact of cloud computing on testing; testing as a service Use of empirical observations, with qualitative research methods
Testing in the Cloud Testing in the cloud affects The acquisition model (cloud based testing emphasizes services The business model (cloud based testing emphasizes pay per use instead of license fees The access model (services are accessed over the internet The technical models of testing (e.g. scalability)
Testing in the Cloud 3. Testing the cloud 2. Testing environments in the cloud 1b. Non-SaaS software 1a. SaaS software • The system or application under test is available online • Testing infrastructure and platforms are hosted in the cloud (Including crowdsourcing/Human as a Service-(Haas)) • Testing of the cloud itself Facets of testing in the cloud Source: L.M. Riungu, O. Taipale, K. Smolander, “Research Issues for Software Testing the Cloud, ”2nd International Conference on Cloud Computing Technology and Science, 2010.
Roadmap towards testing in the Cloud Source: L. Riungu-Kalliosaari, O. Taipale, K. Smolander, “Testing in the Cloud: Exploring the Practice, Accepted, ”Special issue on Software Engineering for Cloud Computing, IEEE Software, March/April 2012. • Develop an understanding of cloud computing Understand the risks and prepare to address them. • Carry out pilot projects. Explore the viability of testing in the cloud and the potential benefits. • Come up with elaborate strategies For example, criteria for the selection of applications suitable for cloud-based testing; criteria for the selection of potential cloud vendors. • Enhance team interaction and prepare for complexities Organizations need to be prepared for additional testing brought about by the complexities and new requirements for cloud-based applications and systems. • Enhance co-operation between research and industry Focus on addressing cloud related issues that are relevant for the software industry (including testing)
Security • Security is seen as a requirement for testing in the cloud • Data security across networks, confidentiality of customer data • Security is seen as an obstacle • Where is the data stored? • Who owns the data? • Who handles the data? • What happens to the data in case of service failure?
Aspects of Security (1) Trust An entity A is considered to trust another entity B when entity A believes that entity B will behave exactly as expected and required (Artz, et al., 2011) Level of certainty to the customer that the cloud provider is capable of providing the subscribed service properly and accurately Governance Management and control over policies, defining roles and responsibilities, standards for application development and special attention for managing security risks/threats (CSA 2009) Design, identification and implementation of organizational structures along with monitoring, control and testing of deployed services in the public cloud (Jansen, et al., 2011) Compliance Compliance is the process of ensuring adherence to policies derived from internal directives, procedures and requirements, or from external laws, regulations, standards and agreements (Proctor, 2011). Involves measuring the effectiveness and adherence of the rules and understanding the followed process. Cloud providers need to provide assurance and proof to the subscribers that they have control over security. Customers need to verify their own internal security measures with their own auditors. Identity and Access Management Provision of privacy and protection of data sensitivity Who has access to the data?
Aspects of Security (2) Availability Service interruption e.g. g-mail had one-day outage in 2008. Distributed denial of services (DDOS) - servers and networks are brought down by the flood of network traffic and prevent users to access the internet based services Incompatibility between the cloud provider’s storage services and applications that need to be tested Data Security Data protection and confidentiality especially in shared multi-tenant environments Change management e.g. skills development Instance Isolation and its Failure Ensure that different instances running on the same physical machine are detached from each other (Ertaul, et al., 2009). Architecture A public cloud may enable one vendor’s SaaS to be hosted within some other vendor’s PaaS or IaaS service. Nested hosting platform and network risks lack of transparency between the customers and actual point of operations even during testing (Lumley, 2010).
Security Approach (2) Define a strategic cloud security roadmap Mainly guided by the requirements of the organization Evaluate the cloud provider’s risks and various types of risk assessment methods Define the business and IT strategy Evaluate the information: what can be public/private? The organization type Identify the risks Point out the risks, threats and vulnerabilities Design some initial control mechanisms to deal with the risks Document the plan Detail all the important aspects in a plan that can be disseminated and effectively communicated Assess the cloud security requirements Map the customer’s security needs to the provider’s ability to meet them Identify the gaps and how to resolve them