1 / 32

The Contrail Demonstrator and other use cases

The Contrail Demonstrator and other use cases. contrail is co-funded by the EC 7th Framework Programme under Grant Agreement nr. 257438 . Christian Temporale, Hewlett Packard christian.temporale@hp.com. contrail-project.eu. Why use cases?.

brit
Download Presentation

The Contrail Demonstrator and other use cases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Contrail Demonstrator and other use cases contrail is co-funded by the EC 7th Framework Programme under Grant Agreement nr. 257438 Christian Temporale, Hewlett Packard christian.temporale@hp.com contrail-project.eu

  2. Why use cases? • To evaluate, test and demonstrate the technology developed in the core research Work Packages. • Doing experiments and extensive tests on the CONTRAIL system • Report usability issues, suggest improvements, identify bottlenecks • Demonstrate that the project have both a scientific and business impact • Identify new requirements • Refine existing requirements • Users’ feedback collection and analysis

  3. Contrail use cases • Distributed Provision of Geo-referenced Data • Implementation of a Virtual 3D Tourist Guide with an embedded 3D Digital Globe, that let end users to navigate through data provided by federated Content Providers • Multimedia Processing Service Marketplace • Implementation of a web portal - that serves as interface for end users, whereas the backend interact with federated Content and Technology providers – offering multimedia services to end users • Scientific Data Analysis • Implementation of a scientific data analysis service, mainly oriented to neutron scattering community • Electronic Drug Discovery – Bioconductor (†) • The Electronic Drug Discovery enables companies in the life science sector to use modern bioinformatics tools/applications on a federated cloud system

  4. UC1 - Distributed Provision of Geo-referenced Data • Virtual 3D Tourist Guide (Google Earth like) • Users navigate the 3D digital globe via web browser • Detailed maps and points of interest • Data set (from Data providers) • Geo-spatial data provided by SDIs (Spatial Data Infrastructures) • Geo-referenced data provided by Content Providers • SDIs and Content Providers • Become part of the Contrail Federated Cloud • Get IT infrastructure from the Contrail Federated Cloud

  5. UC1 – Data and content layers

  6. Internet Application Provider Configuration file Contrail Federated Cloud Public cloud Private cloud Data Provider Data Provider Data Provider Data Provider Data Provider Data Provider Private Cloud VIN UC1 - Distributed provision of geo-referenced data

  7. UC1 - Architecture

  8. UC1 – Deployment in Contrail

  9. UC1 – Main benefits from Contrail • Fully decentralized approach to 3D Virtual World rendering • Enables Data providers to autonomously manage their proprietary data in the federation • Choice of location storage inside the cloud • Elasticity: scale services for traffic spikes • Peak load management • Reduction of initial investments in IT infrastructures for small data providers • Quick and efficient processing of complex raw geospatial data • Support of constraints on data storage • Data protection

  10. Goals of the demonstrator • Transform the selected Use Case into a market oriented demonstrator, open to final users • Prove the concept of Contrail in a nearly production scenario, with all security and usability features • At the end of the project the demonstrator will become full featured and can be easily transformed in commercial products by the industrial partners

  11. UC2 - Multimedia processing service marketplace: end user standpoint sign-up to System subscribe to services download use contents streaming enrich contents end user advanced features advanced search search contents contents proposition contrail-project.eu

  12. UC2 - Actors Content Provider searches / requests contents provides contents requests digital transformation TechnologyProvider MultimediaMarketplace searches / uses / enriches contents provides adapted contents Final user administers users, services, contents Administrator contrail-project.eu

  13. UC2 - Architecture Client web server Content providers s + AC File Payments Presentation layer (V) Business layer (C) DAO layer (M) Media digital processing broker User data processing Data mining system Advanced content search system No SQL DB Technologyproviders

  14. UC2 - Advanced content search component Advanced content search system images DB (faces, keys) Model training New content (video) Image and video processing Machine learrning Face detection, face recognition contrail-project.eu

  15. UC2 - Advanced content search – Example 1 1 front-end activities “Peter Sellers” Marketplace front end back-end pre-processing Doctor Strangelove link 6 2 5 DoctorStrangelove.avi Marketplace DB Get contents by tags (Sellers) Marketplace Business layer 3 4 Doctor Strangelove 4 5 3 8 retrieve name Media digital processing broker Content Provider update tags 9 7 images DB Name, code video recognitionrequest 6 314 1 machine learning Advanced Content Search Sellers.jpg 2 contrail-project.eu

  16. UC2 - Advanced content search – Example 2 1 Marketplace front end Doctor Strangelove link 12 2 11 DoctorStrangelove.avi Marketplace DB Get contents by tags (Sellers) Marketplace Business layer 9 Doctor Strangelove 10 3 8 Sellers 7 retrieve name Media digital processing broker Content Provider images DB 5 image recognitionrequest 4 314 6 Advanced Content Search Update DB contrail-project.eu

  17. Data mining: Apache Mahout • Mahout is a rich set of distributed scalable machine libraries that extracts insights from “Big data” • Mahout’s core algorithms for clustering, classification and batch-based collaborative filtering are implemented on top of Hadoop using Map Reduce • Mahout features used in the demonstrator: • Recommendation mining - based on users’ behavior • Data classification – based on existing categorized contents contrail-project.eu

  18. UC2 - System deployment (3rd rel.) - Current plan conpaasweb application hosting service Contraildistr. storage s Contrail Fed. / conpaas + AC File Payments Presentation layer (V) WS Content Provider GAFS Business layer (C) Contrail VEP DAO layer (M) Media digital processing broker User data processing Data mining system HTTP conpaasSQL service SQL server manager conpaasmap-reduce service Contrail VEP Contrail Fed. / conpaas contrailDaaS Advanced content search SQL DB conpaasKV store MySQL Technology Provider

  19. UC2 - Deployment examples DB deployment on ConPaas Web application deployment on ConPaas

  20. UC2 - Main benefits from Contrail • Security • Confidentiality and protection of whatever users’ data (contents, personal data, usage data) • Legal constraints on data storage (e.g. geographical location) • Reliability • Content storage reliability (→ SLA) • VM reliability (→ SLA) • Performance • Bandwidth availability (→ SLA) • System response time (→ SLA) • Contrail Federation • Technology providers immediately enabled to sell their know-how directly to external operators (e.g. the Marketplace) without needing to invest on development of products accessible by end users • Content providers need trustable storage infrastructures to preserve their data (their business comes from the commercial value of their contents) contrail-project.eu

  21. UC2 - Other Contrail benefits • ConPaaS services • Marketplace deployment (elasticity + scalability) • Structured/distributed storage + GAFS • Contents localization • Legal restrictions on stored contents • SQL servers (High Level Services) • No need to set up new DBs. • Map/Reduce services • Advanced usage data processing • Extraction of statistical information from log files • Search indexes building • VIN / VCP • Advanced configuration of network topology contrail-project.eu

  22. UC3 - High Performance Real-Time Scientific Data Analysis Small Angle Neutron Scattering Analyses data from neutron scattering experiments on biological samples at the ISIS facility at STFC Runs a large number of model-fitting simulations to determine geometrical shapes of chemical samples Presents a figure of merit for each simulation marking its closeness to the experimental data

  23. UC3 - Neutron source data Complex parameter space for modeling analysis of biological structures 1,000 to 100,000 simulations to determine the molecular structure of biological samples Currently, only the most experienced researchers know how to navigate the parameter space

  24. UC3 - Scientific data • Experiments are expensive • Booked months in advance • Need to locate “interesting” parameters • Feed experiences back into model • Manual search can take weeks • Large amount of time taken in trial-and-error • “Close enough” may result in only 10% of the experimental data being used – wasteful

  25. UC3 - Architecture • SDA portal: • Simulations of target structures based on complex parameter exploration • Statistical analysis of generated data, allowing ranking on multiple aspects • Feedback to user as to whether more experimental data is required in order to fully categorize the sample

  26. UC3: System deployment GAFS is used to store experimental data, simulation results, ranked data, and metadata (parameter settings, etc) User defines parameters and experimental data ConPaaS Bag-of-Tasks service Function: Run a simulation task for each combination of parameters Location of simulation results in GAFS is a parameter to Map/Reduce ConPaaS Map/Reduce service Function: Analyze the simulation results and provide a ranking compared to experimental data GAFS SDA Portal ConPaaS web application hosting service

  27. UC3 - Main benefits from Contrail • “Horizontal scaling” • Allocate additional VM instances as needed (not like Grid, previously used) • Estimate expected completion time • Predicable turnaround time → productivity • Performance • Grid technologies did not guaranteed system responsiveness • Single-Sign On • ConPaaS • Bag-of-Tasks • Web application • MapReduce • GAFS for sharing data and simulation output • Infrastructure reliability • Security • SLA Enforcement

  28. WP12: Use Cases and Applications –CONTRAIL features in Use Cases Features exploited in release 2.0 of Use Cases

  29. WP12: Use Cases and Applications – CONTRAIL features in Use Cases Features exploited in release 3.0 of Use Cases

  30. Thank you!

  31. contrail is co-funded by the EC 7th Framework Programme http://contrail-project.eu Funded under: FP7 (Seventh Framework Programme) Area: Internet of Services, Software & Virtualization (ICT-2009.1.2) Project reference: FP7-IST-257438 Total cost: 11,29 million euro EU contribution: 8,3 million euro Execution: From 2010-10-01 till 2013-09-30 Duration: 36 months Contract type: Collaborative project (generic) contrail-project.eu

  32. Backup slides

More Related