The etics service configuration building and testing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 25

The ETICS Service Configuration, Building and Testing PowerPoint PPT Presentation


  • 112 Views
  • Uploaded on
  • Presentation posted in: General

The ETICS Service Configuration, Building and Testing. Guillermo Diez-Andino Sancho (on behalf of the entire ETICS team) Budapest, Hungary – 1-5th October 2007. Contents. ETICS in a nutshell ETICS and QA The Service ETICS Features Metadata Service architecture What it looks like

Download Presentation

The ETICS Service Configuration, Building and Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The etics service configuration building and testing

The ETICS Service Configuration, Building and Testing

Guillermo Diez-Andino Sancho

(on behalf of the entire ETICS team)

Budapest, Hungary – 1-5th October 2007


Contents

Contents

  • ETICS in a nutshell

  • ETICS and QA

  • The Service

    • ETICS Features

    • Metadata

    • Service architecture

  • What it looks like

  • Conclusions

2


Etics in a nutshell

ETICS in a nutshell

  • ETICS stands fore-Infrastructure for Testing, Integration and Configuration of Software

  • Project started on January 1st, 2006 and has a duration of two years

  • Goal is to provide efficient automated tools for building and testing

  • ETICS is born from the experience of developing, integrating and testing grid and distributed software


The etics partners

The ETICS partners

Build system, software configuration, service infrastructure, dissemination, EGEE, gLite, project coord.

Software configuration, service infrastructure, dissemination

The Condor batch system, distributed testing tools, service infrastructure, NMI

Web portals and tools, quality process, dissemination, DILIGENT

Test methods and metrics, unit testing tools, EBIT


Etics and qa i

ETICS and QA (I)

  • ETICS is about providing build/test tools and useful information to help guaranteeing software quality

  • How it is achieved

    • Automation and reproducibility is a first step towards QA

      • Of the Build Process

        • Early error detection and bug fixing

      • Of the Test Process

        • Automatically deploying distributed tests

        • Wrong behaviors are detected in early development stages

    • Collecting software metrics

      • Can be customized (ETICS plugin framework)

    • All the previous points are summarized in Build and Test reports sent to the user


Etics and qa ii

ETICS and QA (II)

  • How is it automated

    • Everything about the software to be built/tested is expressed in the ETICS data model

      • Define the project structure

      • Define the different versions of the components

      • What software depends on what

        • Software dependencies (in code development it is a source of problems)

      • How to build/test

    • On top of that ETICS provides tools to run tests on the code

      • Coding conventions, coverage tests, etc …

      • This tools can be extended/adapted by the users (plugin based framework)

        • Framework allows users with investments in existing build, test, certification and/or software quality assurance tools to leverage this investment

        • Users can select, from a rich set, which metrics are important to them and have ETICS automatically calculate them during build and test, and make them visible


Features of the etics service

Features of the ETICS Service

  • Out-of-the-box automatic build/test system

  • Multiplatform support for development and testing

  • Powerful and flexible automatic dependency management (internal and external dependencies)

  • Trigger coding convention checks, unit test, documentation tools and collect and publish the results

  • Possible to build everything from source or use pre-built binary packages

  • Designed to support several Version Control Systems

  • Possible to register artefacts on the repository

  • Publishes run-time configuration information such as environment variables, runtime dependencies, etc …

  • Produce different package formats (rpms, tar, msi, debs)

  • Produces rich build and test reports

7


Metadata the heart of etics intelligence

Metadata: the heart of ETICS’ intelligence

  • In order to understand how to build and test, ETICS relies on metadata

  • Simple to view and edit by the user, this metadata allows the ETICS Service to perform complex reasoning, hence performing high-level functions on behalf of the user – e.g.

    • Dependency analysis

    • Property processing

8


Service architecture i

Service Architecture (I)

  • ETICS Service build as an N-tier architecture

  • All elements of ETICS are configured, built and tested with ETICS

  • ETICS Service is Open Source: Apache 2.0 license


Service architecture ii

Web Application

Web Service

Service Architecture (II)

Via browser

Build/Test

Artefacts

Report

DB

Project

DB

NMI Execution Engine

Clients

NMI Client

Wrapper

WNs

ETICS Infrastructure

10


Service architecture iii

Service Architecture (III)

  • Build and Test Web Service (WS)

    • Provides single secure interface to the ETICS backend system

    • Performs complex queries on behalf of the clients (WA and CLI)

    • Authentication is performed using digital certificates (x.509)

    • Authorisation is role-based

    • Submit build/test jobs to the underlying execution engine (NMI)

    • Clients connecting without certificate are mapped to Guest, with minimal privileges (most likely just read access)

  • Build and Test Command-Line client (CLI)

    • Build and test locally

    • Browse metadata

    • Edit metadata

    • Submit build and test jobs

    • State-full and secure application

    • Interfaces with the ETICS Build and Test Web Service


Service architecture iv

Service Architecture (IV)

Execution engine

NMI is an execution engine internally used by ETICS to manage remote build and test jobs on different platforms

NMI is a layer built on the top of Condor

Following features were added to NMI/Condor to better support the ETICS Execution Engine

Private resources: projects can provide specific resources for exclusive usage

Freeze: lock the node for further analysis in case of failure

Root-enabled environment: build or test jobs can request to run as root

12


Service architecture v

Service Architecture (V)

  • ETICS composed of following Web Applications:

    • Build and Test (WA)

    • Repository

    • Administration

  • Build and Test Web Application (WA) allows users to

    • Interfaces with the ETICS Build and Test Web Service

    • Browse metadata

    • Edit metadata

    • State-full and secure application

    • Submit build and test jobs

  • Report and repository applications

    • Provide to users access to build products, logs and test results (artefacts)

    • Provides permanent and volatile artefact storage

  • Administration

    • Assign roles to users (Developer, Integrator, ProjectAdministrator, …)

    • Visualize the active users/roles in a given module/configuration


Some screenshots

Some screenshots

What it looks like


Getting started

Getting started


Etics registration

ETICS registration

https://etics.cern.ch:8443/eticsAdmin/

/public/registration/requestRegistration.jsp


Project metadata i

Project Details

Project Metadata (I)

  • All ETICS registered projects are available from a central entry point: https://etics.cern.ch:8443/etics

  • Project details are shown upon selection in a drop-down list

17


Project metadata ii

View/Edit Panel

Browse Panel

Configuration List

Project Metadata (II)

18


Configuration metadata

Configuration Metadata

  • ETICS defines configuration similar to version in CVS

  • Configurations can be defined differently for different platforms

  • Configurations include what to do to checkout, build and test software modules:

    • Version control system (VCS) - e.g. CVS, SVN - commands

    • Build commands

    • Test commands

    • Properties and environment

    • Dependencies

  • Properties and environment variables are inherited and propagated

  • Each command includes clear set targets. e.g. build: clear, init, checkstyle, doc, compile, package, publish

19


Remote build test

Remote build/test

20


Etics reports i

ETICS Reports – (I)


Etics reports ii

ETICS Reports – (II)

  • Trees

    • Easy browsing trees

    • Organised by Project or by Build Date

  • Detailed reports

    • Package Summary

    • Build Summary

    • Build Configuration

    • Build Logs

http://etics.cern.ch:8080/reportBrowser/


Etics repository

ETICS Repository


Conclusions

Conclusions

  • ETICS defined, integrated and delivered a novel Configuration, Build and Test Service

  • Valuable and useful information is provided out of the build/test process

  • The service leverage the rich knowledge foundation of the consortium partners in the domain of software quality assurance, development, integration and test of complex Grid and distributed software

  • The ETICS service was used in an incremental fashion to build and test the ETICS service, providing rapid feedback to the team

24


Thanks

Thanks

http://www.eu-etics.org

25


  • Login