Transparency in Distributed Operating Systems Vijay Akkineni

1. Transparency in Distributed Operating SystemsVijay Akkineni

2. Centralized Operating systems • Network Operating Systems • Distributed Operating Systems • Cooperative Autonomous Systems • Cloud Computing Operating Systems Generations

3. Partitioning of COS

4. Peer to Peer communications. • Seven layer OSI architecture. • Examples Remote Login, File Transfer, Messaging, Network Browsing, Remote Execution. Network Operating System

5. Loosely coupled systems. • Sharing or resources and coordination of resources. • Transparency – Key difference between NOS and DOS. • Distributed resources and activities are to be managed and controlled. Distributed Operating System

6. Distributed Operating Systems

7. Characterized by Service Integration. • Middleware – Cobra, JMS, RMI. Cooperative Autonomous Systems

8. Hide irrelevant system dependent details from the users • Higher Implementation Complexities • Single System Image • Minimal Knowledge Transparency

9. User has no awareness of object locations,objects are mapped and referred to by logical names. • WebServices UDDI, Federated Services - SOA Location Transparency

10. Ability to access local and remote system objects in uniform way. • The physical separation of system objects is concealed from the user. • Examples – Accessing a file from the local file system and from a cloud drive. Access Transparency

11. Logical Resources and Physical processes migrated by the system, from one location to another in an attempt to maximize efficiency, reliability, availability or security should be automatically controlled by the system • Example – Application Servers using JNDI Migration Transparency

12. Exhibit consistency of multiple instances of files and data. • System elements are copied to remote points in the system in an effort to possibly increase efficiencies through better proximity or provide increased reliability through duplication. • Examples – Google's Big Table, HDFS. Replication Transparency

13. Sharing of Objects without interference. • Similar to Time sharing. • An important challenge when designing distributed systems is how to deal with concurrent accesses. • Example – An important design goal for distributed database. Transactional Integrity and ACID properties during multiple transactions happening concurrently. Concurrency Transparency

14. Failure Transparency tries to mask failures so that they are not seen or noticed by the users. • It is difficult to identify between a resource that has failed and a resource which is performing badly (slowly). • Consider opening a webpage - is it dead or painfully slow, how long should the browser wait? • Examples - Map Reduce Frameworks, DFS Replication on Data Nodes. Failure Transparency

15. Attempt to achieve a consistent and predictable performance level even with changes to system structure or load distribution. • When parts of the system experience significant delay or load imbalance, the system is responsible for the automatic, rapid, and accurate detection and orchestration of a remedy. • Examples - Load balancing, Speculative execution in Map Reduce. Performance Transparency

16. A system's geographic reach, number of nodes, level of node capability, or any changes therein should exists without any required user knowledge or interaction. • Research Area - Currently there is lot of ongoing research on running Map Reduce job across the data centers. Partition compute jobs based on geographical locality. Size/Scale Transparency

17. System occasionally have need for system-software version changes and changes to internal implementation of system infrastructure. • Examples - Linux Kernel Upgrades and how it should not effect the existing software applications on the OS. Revision transparency

18. System occasionally have need for system-software version changes and changes to internal implementation of system infrastructure. • Examples - Linux Kernel Upgrades and how it should not effect the existing software applications on the OS. Revision transparency

19. The most difficult aspect of transparency,”Holy Grail” of distributed system designers. • Systems parallel execution of processes throughout the system should occur without any required user knowledge. • Examples – Parallel Algorithms on multicore processors and Map Reduce tasks on multiple systems. Parallelism Transparency

20. Transparency Summarized

21. Major Research Areas

22. Permit autonomous management of its resources on behalf of its users and applications. • Cloud OS operations must continue despite loss of nodes, entire clusters, and network partitioning. • The Cloud OS must be operating system and architecture agnostic. • The Cloud must support multiple types of applications, including legacy. • Cloud OS management system must be decentralized, scalable, have little overhead per user and per machine and be cost effective Towards Cloud OS

23. Logical Model of Cloud OS

24. Heterogeneous Nature of Cloud hindering adoption of Cloud Technologies. • IBM is researching into Altocumulus middleware, which offers a uniform API for using Amazon EC2, Eucalyptus, Google AppEngine, and IBM HiPODS Cloud, aiming to provide an API which is Cloud agnostic. • http://www.almaden.ibm.com/asr/projects/cloud/ Cloud Middleware

25. “HP performance-optimized datacenter (POD).” Data Sheet, 2008. • “Amazon EC2.” [Online] http://aws.amazon.com/ec2. • Apache hadoop Map Reduce References