Distributed Systems Fall 2010

2. Distributed Systems Fall 2010 Course introduction

3. 5DV020 Defining distributed systems “A distributed system is one in which components located at networked computers communicate and coordinate their actions by passing messages.” (Coulouris, Dollimore, Kindberg, 2005) ”A distributed system is one in which the failure of computer you didn't even know existed can render your own computer unusable.” (Leslie Lamport, 1987)

4. 5DV020 Outline Staff presentation Course presentation Lessons from last year This year's course Basics and challenges of distributed systems The big assignment

5. 5DV020 Staff presentation Daniel Henriksson Lars Larsson Questions about the assignments? Send to 5dv020-staff@cs.umu.se Questions about lectures? Send email to the appropriate teacher!

6. 5DV020 Course presentation Students should obtain: Knowledge of theoretical models for distributed systems Knowledge of problems and solutions in designing and in the implementation of distributed systems

7. 5DV020 Course presentation • The course covers: • Architectural models of distributed systems • Client-Server, peer-to-peer, transactions, transparency, naming, error handling, resource management, and synchronization … and much more! • Computer security in a broad perspective • Distributed programming and middlewares

8. 5DV020 Course presentation Theoretical part (4.5 ECTS) Theory, methods, algorithms, and principles Practical part (3 ECTS) Practical obligatory assignments

9. 5DV020 Lessons from last year (positive) Students were very happy with the level of assistance on the assignment Students acquired a lot of new knowledge The average grade given to the course was “very good” Students were very happy with the practical assignment – hard and rewarding

10. 5DV020 Lessons from last year (negative) Common pitfall: students start with the assignment too late Split opinions regarding what to do with extra time during lectures Very limited computer access due to other course Unclear how level 1 was easier than level 2 on the assignment Too large/annoying post-course evaluation!

11. 5DV020 This year's course Added an assignment on Java RMI Note that any programming language may be used for the second, big assignment Let’s vote regarding our policy on what to do about extra time during lectures! Department hopes that computer access will be better this year (but don’t count on it) Clearer info about the big assignment New evaluation form/procedure Keep up good work

12. Extra time during lectures We want to be able to adopt the schedule to fit your needs Some lectures are somewhat “sparse” If our in-class discussion of topics take longer time, that’s fine (within reason) If not, we can start covering upcoming topics to have more time during denser lectures Consider topics a reading guide, not a guarantee If this poses a problem, please give us feedback, and we will try to remedy the situation 5DV020 5DV020 12

13. 5DV020 What to learn? Book is dense with information See reading guide on web page – it is actually accurate Extremely good, but no easy read Start now! You will be busy later... Understand the problems and solutions Learn the general ideas of algorithms and how/why they work, not every minute step Definitions are very important!

14. 5DV020 Basics of distributed systems Architectural model Placement of components across the network and their relationships, and the network itself System model Client/Server HTTP, FTP, … countless others! Peer-to-Peer (P2P) BitTorrent, Freenet, Direct Connect, … Combinations: multiple servers, mobile code, mobile agents, thin clients

15. 5DV020 Design of distributed systems Performance issues Responsiveness Throughput Load balancing Quality of Service (QoS) Reliability, security, performance (time-critical) Adaptability

16. 5DV020 Design of distributed systems • Caching and replication • Web caching and Content Distribution Networks • Dependability • Correctness, security, and fault tolerance

17. 5DV020 Distributed systems challenges Heterogeneous systems Networks, hardware, OS, languages, protocols Openness Security Scalability Control physical resource costs and performance loss, resource conservation, bottleneck avoidance

18. 5DV020 Distributed systems challenges • Failure handling • Detection, masking, tolerance, recovery, redundancy • Concurrency • No global time, simultaneously running processes • Transparency • Access, location, concurrency, replication, failure, mobility, performance, scaling

19. 5DV020 Distributed systems challenges How are parameters passed and how is data converted? Marshalling, unmarshalling How are distributed resources (functions, methods, objects) published and discovered? How are errors handled? Security! Memory handling – distributed garbage collection?

20. 5DV020 Middleware Distributed systems often utilize middleware to aid development Offers layer of abstraction Extends upon traditional programming models: Local procedure call → Remote procedure call OOP → Remote Method Invocation Event-based programming model

21. 5DV020 Middleware Applications, Services RMI, RPC Middleware Request/Reply protocol Marshalling, Unmarshalling UDP, TCP

22. 5DV020 Operation invocation Data structures must be “flattened” and serialized for transport External formats, e.g. XML, JSON, Java Object Serialization, ... Use interface Procedures having either input, output, or both No pointers Service interface: provided services Remote interface: operations accessible from other processes Cross-language/platform interfaces: IDL, WSDL

23. 5DV020 Semantics (Local call = exactly once) Maybe once Omission failures (lost packets, crashes) At-least-once Crash failures, arbitrary failures (multiple executions) Used by Sun RPC At-most-once Executed exactly once or not at all Used by Java RMI, Corba

24. 5DV020 Security Distributed system = increased exposure Client- and Server-authentication Client authorization Is the client allowed to perform X? Proof of execution Server must be able to prove that something has been executed Also, non-repudiation: it should not be possible to claim that something did not happen if it did

25. 5DV020 Distributed systems: a mess! Communication performance variations Latency (delay), bandwidth (throughput), jitter (variation in time) Clocks and timing Clock drift Interaction models Asynchronous, synchronous Event ordering Delays cause replies to arrive to some process before the request

26. 5DV020 Distributed systems: a mess! • Failures • Distributed systems are much more likely to fail unexpectedly • Lost packets, bit errors, local failures, no response, method does not exist, etc … If you can write stable programs in spite of these difficulties, you are a great programmer!

27. 5DV020 The small assignment Previously a group exercise Solved individually You teach yourself Java RMI Deadline next Friday (Sept 10) 13:15 Session for (final) questions that very same day http://www.cs.umu.se/kurser/5DV020/HT10/assignment.html

28. 5DV020 The big assignment Group communication middleware Presentation of working implementation at the end of the course Solved in pairs (2 students per team) Pair up among yourselves before Friday! Deadlines: Design report: September 17 13:15 Whole system: October 26 13:15 Presentations: October 27 – 29 http://www.cs.umu.se/kurser/5DV020/HT10/assignment.html

29. 5DV020 The big assignment Apply concepts from theory Group handling Message ordering (Reliable) Multicast of messages Not security, however

30. 5DV020 Rules and grading Done in pairs 3 levels Bonus points for the exam (if non-bonus points give you ≥ 30p of 60p total)! Valid for this year's exams only Level 1: basic system (no bonus) Level 2: + dynamic groups (3p bonus) Level 3: + tree-based reliable multicast (6p bonus)

31. 5DV020 Levels • You may change level at any time • Level 1 is easiest, but in practice only if you aim for it from the beginning • Many problems can be avoided due to greatly lowered fault-tolerance of the system

32. 5DV020 Constraints May use any programming language and any tools you like ...as long as they do not provide a too big advantage (check with us!) Currently, we can only help with Java RMI Do not use plain sockets All normal rules apply Thou shall not cheat, etc.

33. 5DV020 Test and debug application(s) Test application A user-level application that shows the functionality of the system Debug application Used to demonstrate the correctness of your implementation These programs can, and likely will, be one and the same! But make the debug parts non-essential to use the application Must be GUI applications!

34. 5DV020 Deliverables Deliverable 1 (project plan) Your interpretation of the assignment Requirement analysis Project and time plan Basic design of the system Yes, really Deliverable 2 (report) Refers back to Deliverable 1 Describe your system ...the usual Make something to be proud of! One of your biggest projects during your time here at CS

35. 5DV020 Live demonstration You will demonstrate your system to us at the end of the course Written test protocol

36. 5DV020 Good luck! Students have done this before, and succeeded Certainly not easy Hard work, big payoff All students that attempted the entire assignment passed! Hints Start on time. No really. Read the whole specification We know it’s long, but it helps you

37. 5DV020 Next lecture Time, and the lack thereof No global time, and its implications Algorithms for (roughly) synchronizing processes