Apache Airavata

1. What Is Apache Airavata ? ● A framework that supports execution and management ● Of computational scientific applications and workflows ● Mainly for grid based systems but will support – Remote clusters, cloud-based systems – Can be extended via implementation ● Can be a middleware for scientific developers ● Offers an extensible architecture ● Open source Apache 2.0 license ● Written in Java, C++

2. Airavata Architecture

3. Airavata Architecture ● Airavata API – Users / gateway developers can communicate with Airavata ● Orchestrator – Manages application AMQP worker queue ● Workflow interpreter – Manages submitted work flows ● Application Factory (Gfac) – Communicates with remote resources

4. Airavata Architecture ● Registry – The Airavata data store ● Messaging – Publishes notifications for application statuses ● API written using Apache Thrift – Supports many languages

5. Airavata AppCatalog Data Model

6. Airavata AppCatalog Data Model ● Used for application registration and contains ... ● ComputeResourceModel – Abstraction of remote compute resource – Contains information about resource queues – Host addresses, job submission protocols – Useful information related to compute resource ● ApplicationModule – Remote resources contains modules / applications – Already installed in the systems – User defined applications possible

7. Airavata AppCatalog Data Model ● Used for application registrationand contains ... ● ApplicationInterfaceModel – Interface which defines application I/O – Can have multiple interfaces defined ● ApplicationDeploymentModel – Bridges the compute resource and the module ● GatewayProfileModel – Abstraction of the science gateway ● GatewayPreferenceModel – Model to specify gateway compute resources preferences – Includes information like login usernames, allocation ids etc

8. Airavata ExperimentCatalog Data Model

9. Airavata ExperimentCatalog Data Model ● Used for application execution ● Application might use single node or work flow ● Airavata internally creates models to represent process – Process model, task model, job model ● Work flows contain a list processes ● Tasks can have many types – Input staging, job submission – Job monitoring, output staging etc ● Users can develop their own tasks for the task chain

10. Airavata UI ● Has a Java based application called XBaya – Create workflows, – Submit and manage multiple applications – Works best with Airavata 0.14 ● Also has a web-based interface (PHP Gateway) – Written using Airavata PHP client library – Users can deploy it in their own systems – Use it to register applications – Run and monitor applications

11. Available Books ● See “Big Data Made Easy” Apress Jan 2015 – See “Mastering Apache Spark” ● Packt Oct 2015 – See “Complete Guide to Open Source Big Data Stack ● “Apress Jan 2018” – ● Find the author on Amazon www.amazon.com/Michael-Frampton/e/B00NIQDOOM/ – Connect on LinkedIn ● www.linkedin.com/in/mike-frampton-38563020 –

12. Connect ● Feel free to connect on LinkedIn –www.linkedin.com/in/mike-frampton-38563020 ● See my open source blog at open-source-systems.blogspot.com/ – ● I am always interested in – New technology – Opportunities – Technology based issues – Big data integration