Organized by PBS GridWorks, Altair Presentation by S.THAMRAI SELVI Professor and Head Department of Information Technol

1. Virtualization and Grid @ GRID 08, Bangalore Organized by PBSGridWorks, Altair Presentation by S.THAMRAI SELVI Professor and Head Department of Information Technology MIT Campus, Anna University Chromepet, Chennai on 31.08.2008 GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

2. Agenda • Introduction • Virtualization • Trust management in Grid • Semantic discovery of Resources in Grid • Conclusion GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

3. The laws influencing Information Technology • The laws that are generally accepted as governing the spread of technology: • Moore's Law • Gilder's Law • Metcalfe's Law • Disk Law GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

4. Service Oriented Architecture Web Services Virtual Computing Infrastructure Consolidation IT Architecture Trends Flexibility Range of solutions GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University Resource Sharing

5. Service Oriented Architecture model GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

6. XML Web Services FoundationSimple, Open, Broad Industry Support GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

7. Web 2.0: Evolution Towards a Read/Write Platform GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

8. Core concepts used in Web 2.0 • AJAX- Asynchronous JavaScript + XML (User interface/Portal) • JSON - JavaScript Object Notation (SOAP) • REST Representational State Transfer (WSDL). REST proposes to use Internet as programming platform with only HTTP • Widgets, Gadgets, Start page for Portal GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

9. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University Source: http://blogs.zdnet.com/Hinchcliffe/

10. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

11. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

12. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

13. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

14. internet internet access tier edge routers routing switches authentication, DNS,intrusion detect, VPNweb cache 1st level firewall web tier load balancingswitches web servers web page storage(NAS) 2nd level firewall application tier switches applicationservers files(NAS) switches database tier databaseSQL servers storage areanetwork(SAN) Today’s IT architecture is complex and unmanageable… GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

15. Moore’s Law. Transistors on a single chip doubles approximately every 18–24 months. Gilder’s Law. Aggregate bandwidth triples approximately every year. Metcalfe’s Law. The value of a network may grow exponentially with the number of participants(community Law). Disk Law Storage doubles every 12 months Putting together network, storage and computing speed …rates of growth of digital power Source: Cambridge Energy Resource Associates 10616-17 GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

16. “Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations” The Need for New Technology GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

17. Technologies influencing a new era in computing… • Microprocessor Technology • Optical Networking Technology • Storage Technology • Wireless Technology • Sensor Technology • Global Internet Infrastructure • WWW and Web Services (SOA) • Open Source Movements GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

18. Grid A kind of open standard distributed infrastructure that enables flexible, secure, coordinated resource sharing among dynamic collections of trusted resources belonging to diverse organizations across the globe ensuring user’s Quality of Service requirements. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

19. Grid enables … • Virtual Organization • Dynamic Resource sharing GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

20. Building Blocks • Network • Computing Nodes • Storage PARAM PADMA GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

21. Context jobs / legacy code /binary executables Middleware Resources Map to resources GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

22. Internet GRID Application Application Collective Resource Transport Connectivity Internet Fabric Link Overall GRID Architecture GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University Source: The Anatomy of the GRID, Foster, Kesselman and Teucke

23. Protocols, Services,and APIs Occur at Each Level Applications Languages/Frameworks Collective Service APIs and SDKs Collective Service Protocols Collective Services Resource APIs and SDKs Resource Service Protocols Resource Services Connectivity APIs Connectivity Protocols Local Access APIs and Protocols Fabric Layer GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

24. Local “metacomputers“ • Distributed file systems • Site-wide single sign-on • "Metacenters" explore inter-organizational integration • Totally custom-made, top-to-bottom: proofs of concept 1 We are here! • Utilize software services and communications protocols developed by grid projects: • Condor, Globus, UNICORE, Legion, etc. • Need significant customization to deliver complete solution • Interoperability is still very difficult! 2 • Common interface specifications support interoperability of discrete, independently developed services • Competition and interoperability among applications, toolkits,and implementations of key services 3 Three Generations of Grid Standardization is key for third-generation grids! GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University Source: Charlie Catlett

25. Grids – Where to ? Grid 1.0 – concerned with the virtualization, aggregation and sharing or compute resources Grid 2.0 – focused on the virtualization, aggregation and sharing of all compute, storage, network and data resources The promise is that in Grid 2.0 the resources will be easier to define, test, install, transport and adjust on demand. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

26. Grid 2.0 Emerging • Grid 2.0* • Virtualized Compute, Storage, Network, Data • Service Oriented • Policy Driven Automation • Distributed across firewalls • Parallel, stateless, stateful and transactional apps SOA Software Services with SLA & QoS Metrics Grid 1.0 Compute Intensive Cycle Aggregation Virtualization Consolidation of Resources GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University *The 451 Group: 'grid 2.0' is focused on the virtualization, aggregation and sharing of all compute, storage, network and data resources. It is both Service-oriented and automated.

27. Grid 3.0? Not just stitching Web and Grid together (Semantic Web and Web 2.0 ARE the Web) The Grid is part of the Web. The Web is a platform for the Grid. A rich and relevant experience for users and developers. Smashing and mashing. User Applications Semantic Grid Web 2.0 (Grid) Middleware Resources GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

28. Platform – Infrastructure Rapid Application development Agile, Loosely coupled Designed for Reuse Grid 3.0? The Participatory Grid. People - Social • Keep it Simple • Users count GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

29. Next Generation Grid(s) – 3-fold VisionExpert Group Reports: NextGenerationGrids • End-user empowerment • Life-support to business processes Simplification Abstraction Software Vision End-User Vision • Continuously changing requirements • Grid services development environments Architectural Vision • Societal behaviour (millions of self-organising nodes) • Computational semantics, ontologies, meta-descriptions • Pervasive virtual organisations Virtualization GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

30. Virtualization GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

31. Virtualization Virtualization is a framework or methodology of dividing the resources of a computer into multiple execution environments, by applying one or more concepts or technologies such as hardware and software partitioning, time-sharing, partial or complete machine simulation, emulation, quality of service (Source: Wiki) GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

32. Benefits from Virtualization • Load balancing • Create VM for new work loads • Increased Resource utilization • Create VM in the under utilized resources • Reusable • One VM created can be reused somewhere else • Isolation • Destruction of one VM doesn’t affect the other VM GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

33. Continued… • High Availability • Reduction in downtime • Improved ROI • More production with less Investment, operational and maintenance • Reduced TCO • Eliminate the need of new Infrasturctures such as Server GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

34. Continued… • Live migration • Support suspend resume and migration of created VMs • Persistent and Non Persistent • Support for keep/drop the changes made in VM images • Undoable and Append • Allows undo and append options in VM imagaes GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

35. “If you want people to use your system then make it as easy as possible to use.”Werner Vogels, Amazon CTOSimple to use is not the same as simpleYes, Grids are doing complicated hard stuff. But why make the developer and user suffer so? Also, Trust is another major issue Motivation for our Research GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

36. CARE Resource Broker (CRB ) A Grid Meta Scheduler GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

37. CLI or Job Submission Portal CARE Resource Broker Job Pool User Pool Request Handler CONTROLLER Job Tracker Scheduler Host Identifier Host Pool CRB with Virtualization support DB RESOURCE MANAGER Information System Virtual Machine Manager Physical Machine Manager Virtual Cluster Service Drivers Transporter Aggregator Executor GT Components GridFTP FTP MDS4 Pre-WS GRAM WS GRAM Grid Resources GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

38. CRB Request Handler: • Gets the Job request, create Job instance and store it in the Job Pool. Controller: • Initiates the Aggregator to collect the information about all available resources and updates the Host object and Host Pool object. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

39. Continued … Scheduler: Gets the unordered resource list and orders the resources based on their trust value and return back to Controller. Virtual Cluster Service: The controller gives the resource list to the VCS in order to create virtual cluster. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

40. Continued… Transporter Initializes the RFT and transfer the input and executables into selected host Executor: Initializes the GRAM to submit the job into local scheduler. Aggregator Initializes MDS to get the resource information in the Grid GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

41. Trust Management System GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

42. We define Trust… The degree of belief in the resource provider’s competence to complete user’s task dependably, securely and reliably in a specific context at a given time Agent / Resource Broker users Resources GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

43. Modify or update the value of trust periodically for each resource provider • Identify suitable parameters with which the respective trust can be defined Trust Integration Trust Metric Identification • Integrate the trust mechanism in the resource broker / Scheduler to find out the most trustworthy resource provider for successful job execution/task completion • Apply suitable methodology to determine the value of those metrics TMS Trust Value Updation Trust Metric Evaluation Trust Value Calculation • Determine the overall trust value using the values for various trust metrics obtained Trust Management Life Cycle GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

44. Describes relying party’s trust in a service provider. The trustor trusts the trustee to provide a service that does not involve access to the trustor’s resource • It describes trust in principals for the purpose of accessing resources owned by the relying party. A trustor trusts a trustee to use resources that he owns or controls. • It measures whether a resource provided by the resource provider is trustworthy. • The QoS is the main concern in this case • It is the belief that information provided by the Information provider is reliable and accurate. • It is a measure of belief that a resource broker has discovered a trustworthy resource • Measures whether a resource provider is willing to offer his services to the user. • The previous behaviour / payment record may be considered for this trust Types of trust GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

45. These metrics reflect the throughput of the resources and their QoS Resource Registration Metrics Resource Performance Metrics Availability, Bandwidth, Latency, Estimated Execution Time, ActualExecution time, Success, Failure Equipment Provision Trust User Feedback Metrics Dependency Metrics Government / Private, Registration Number These metrics reflect the reputation of the organization . It is used to identify initial trust value of the resource provider These metrics reflect reputation of the resource in the user community Recommendation, Reputation GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

46. Block Diagram of Trust Management System Determines Overall trust value by using suitable mathematical model Determines values of metrics using different methodologies Periodically updates the trust value and stores in the database Trust Metrics Identification Database User Feedback Trust Updation User Resource Registration Resource Provider Trust Metrics Evaluation Trust Calculation Resource Performance Underlying Middleware, Resources GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

47. Conventional Gridway Flow Trust Enabled Gridway Flow <job template> <job template> Job Submit Job Submit Invokes Scheduling Operation Invokes Scheduling Operation Gathers Available Resource Gathers Available Resource Selects Most Trusted Resource Performs Matchmaking Performs Matchmaking Trust DB Matches Against JobReq Invokes TMS TMS Matches Against JobReq Selects and submits Selects and submits R2 R1 R3 R2 R1 R3 GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

48. PATENT Pending • Filed Application for patent (Patent application No.593/CHE/2007 A) • Title of the Invention: Trust Resource Broker • International Classification:G06F 17/30 • Patent has been published in Journal No. 16/2007 dated 20/04/2007. GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

49. Integrated with Gridway Meta Scheduler • http://www.gridway.org/doku.php?id=related:related Developed by other projects • Trust Management • Description - This project provides GridWay with a mechanism to evalute the trust of the computational resources. The Trust module consists of modified host attribute parsers and match-making algorithms. Additionally the Trust module needs to modify the information providers of Grid resources. • Contact - This component has been developed by the team of Prof Dr.S.Thamarai Selvi (stselvi@annauniv.edu) at Madras Institute of Technology GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University

50. Semantic Component GRID08 - Grid Virtualization by S.T.Selvi, MIT, Anna University