DEISA perspectives Workshop on Biomedical informatics, Brussels, March 18-19 2004 - PowerPoint PPT Presentation

Deisa perspectives workshop on biomedical informatics brussels march 18 19 2004 l.jpg
Download
1 / 16

DEISA perspectives Workshop on Biomedical informatics, Brussels, March 18-19 2004 . Victor Alessandrini www.deisa.org. D istributed E uropean I nfrastructure for S upercomputing A pplications - Consortium. IDRIS – CNRS, France ( coordinator ) FZJ – Juelich, Germany

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

DEISA perspectives Workshop on Biomedical informatics, Brussels, March 18-19 2004

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Deisa perspectives workshop on biomedical informatics brussels march 18 19 2004 l.jpg

DEISA perspectivesWorkshop on Biomedical informatics, Brussels, March 18-19 2004

Victor Alessandrini

www.deisa.org


D istributed e uropean i nfrastructure for s upercomputing a pplications consortium l.jpg

Distributed European Infrastructure for Supercomputing Applications - Consortium

  • IDRIS – CNRS, France (coordinator)

  • FZJ – Juelich, Germany

  • RZG – Garching, Max Planck Society, Germany

  • CINECA, Italy

  • EPCC (HPCx project), Edimburgh, UK

  • CSC, Helsinki, Finland

  • SARA, Amsterdam, The Netherlands

  • ECMWF (european organization), Reading, UK


Deisa mission statement l.jpg

DEISA : mission statement

  • To contribute to a significant enhancement of capacities and capabilities of HPC in Europe, by the integration of leading national supercomputing infrastructures.

  • To deploy and operate a distributed multi-terascale European computing platform, based on a very strong coupling of existing national supercomputers. DEISA plans to operate as a virtual European supercomputing centre.

  • To contribute to the deployment of an extended, heterogeneous Grid computing environment for HPC in Europe, needed to interface the DEISA research infrastructures with the rest of the European IT infrastructures.


Deisa management operation as a virtual european supercomputing centre l.jpg

DEISA management (operation as a virtual Europeansupercomputing centre)


Artist s view of the facility l.jpg

Artist’s view of the facility

The DEISA super-cluster (the “core” project)

Dedicated network interconnect

(reserved bandwidth)

National computing

facility

Site B

Site A

Site C

Site D

Extended Grid

services


Deisa and grid technologies l.jpg

DEISA and Grid technologies

  • Large scale integration of IT systems is, for DEISA, primarily an strategic issue

  • DEISA is technology neutral. Technology choices follow from their capability to adapt to a pre-established operational model, and to provide real services to end users.

  • Three criteria for technology choices:

    • The necessity of disposing as soon as possible of a stable and reliable European production platform (the “core” platform). Grid technologies work in the background through global file systems and multi-cluster batch managers.

    • The necessity interfacing this platform with other systems in Europe (the “outer” Grid environment). Deployment of traditional Grid environments, like the Globus Toolkit or Unicore.

    • The preparation of their future evolution by the integration of new Grid technologies as they reach maturity (a specific JRA activity).


The first generation core infrastructure operational early 2005 l.jpg

The first generation “core” infrastructure(operational early 2005)

125 cabinets like the ones shown above (IBM 690, 690+, 655+)

4000 processors (5 – 8 Gigaflops per processor)

About 10 Terabytes memory

A lot of useful teraflops.

Inclusion of huge Linux and vector systems planned for 2005.


Global file systems l.jpg

Global File Systems

Sophisticated software environment,

necessary to provide single system

image if a clustered computing

platform.

Global file system

They provide global data management.

Data in the GFS is “symmetric” with

respect to all computing nodes.

GFS encapsulate sophisticated distributed computing and Grid technologies.

Applications do not need to be modified to benefit from GFS services.

Grid technologies are working in the background, and they are not directly

seen by end users.


The deisa integration concept core infrastructure l.jpg

The DEISA integration concept (core infrastructure)

Site A

Site B

Global distributed GPFS file system

with continental scope.

Global resource pool is dynamic: nodes

can enter and leave the pool without

disrupting the national services..

Network interconnect

(reserved bandwidth)

Site C

Site D


The deisa operational model l.jpg

The DEISA operational model

  • Global management of an “ European resource pool”, that will provide:

    • An integrated supercomputing environment for trans-national collaborations, providing global data management through global file systems.

    • No need to grid enable huge applications (unless they are grid enabled by design). Applications can benefit from important computational resources being run “as such”.

    • Implementation of job migration across sites (transparent to end users) as a way of releasing significant resources in one particular site for demanding DEISA applications. We are load balancing computational throughput at a European scale.

    • Support of distributed applications (grid enabled by design).

  • With this operational model, the European super-cluster is not very different from a huge local supercomputing cluster (which will be partitioned anyhow because of fault tolerance).


Scientific perspectives l.jpg

Scientific perspectives

  • We (the DEISA partners) have decades of experience in the production of leading edge computational science.

  • We dispose of competent staff for user support, system integration,…

  • Integration of human competences is as important as the integration of computing resources.

  • We have developed, over the years, very efficient interfaces with the different scientific communities (climat modeling, quantum chemistry, material sciences, …). Each one has its own specific requirements.

  • We are eager to develop special interfaces with the Bio-Medical community, in those areas in which we can provide added value to their research.

  • DEISA includes a Life Science Joint Research Activity.


Prospective services first guess l.jpg

Prospective services (first guess)

  • Using the global distributed file systems of the “core” facility to deploy shared data repositories.

  • Deploying distributed heterogeneous file systems with external organizations to dispose of “enhanced privacy” access to dada.

  • Fine tunning new generation BMI applications to clusters of large, high physical memory, SMP clusters. Our “intelligent” operating systems can use large memory address space to create memory mapped file systems and dramatically reduce input/output overhead in data driven applications.

  • Running very demanding applications in what concerns processing power and memory size.

  • Running very demanding applications in what concerns execution times (hundreds of hours). This requires process migration and checkpoint-restart facilities for QoS (which are present in our platforms).

  • Interfacing with other organizations, which may act as “portals” to the DEISA environment.

  • Deploying high level Web-like services that will allow computational monitoring and steering of complex applications, hiding the DEISA environment from end users.


Portals web like services l.jpg

Portals, Web-like services

The DEISA super-cluster (the “core” project)

ASP services. Users connect to an application,

not to the computing environment.

Provides computational monitoring and

steering services.

Technology already available for the DEISA

users.

Site B

Site A

Users are not

IDRIS-DEISA aware

Demanding applications

are rerouted to IDRIS

Site C

IDRIS

InfoBioGen


Asp services three tier architecture for computational monitoring and steering l.jpg

ASP services: three tier architecture(for computationalmonitoring and steering).

Firewall

Internet

Intranet

Gateway: bridge

between the Internet

clients and the legacy

environment. Handles

all AA services.

Clients connect to J2EE

application running on the

gateway.

Back end server: runs

high performance

“legacy” application


Extended heterogeneous data management l.jpg

Extended, heterogeneous data management

Global distributed file system with:

- Single sign on

- Intrinsic security model

- Access control

- Internal cryptographic mechanisms

- Operating across firewalls

- Web and NFS access to data.

(candidate: AVAKI DataGrid)

Site B

Site A

Other organizations disposing of

confidential data sets.

Site D

Site C


Conclusion l.jpg

Conclusion

  • DEISA is focused on European computational science.

  • Strong integration of national supercomputing infrastructures should be pursued, as the only way forward (today) for matching the US and Japanese efforts in this area.

  • The DEISA consortium is committed to enhance the production capabilities of HPC in Europe, and to search for a very large user consensus and for a high impact in everyday creation of scientific results. This is its only measure of success.

  • We are eager to work with the BMI community to identify the areas in which we can provide leading edge services adapted to their needs.


  • Login