Distributed Systems and Concurrency: Concurrency Control in Local and Distributed Systems

1. Distributed Systems and Concurrency: Concurrency Control in Local and Distributed Systems MajeedKassis

2. Transactions • Definition: A sequence of operations that perform a single logical function • Examples: • Withdraw money from bank account • Making an airline reservation • Making a credit-card purchase • Usually used in context of databases

3. Definition: Concurrency Control • Definition: Concurrency control is the activity of coordinating concurrent accesses to a database in a multi-user database management system (DBMS) • Problems: • Lost update problem • Temporary update problem • Incorrect update problem

4. Types of Concurrency Control • Local concurrency control • Two-phase locking • Timestamps • Distributed concurrency control • Two-phase commit

5. Concurrency in a DBMS Users submit transactions, and can think of each transaction as executing by itself. Concurrency is achieved by the DBMS, which interleaves actions (reads/writes of DB objects) of various transactions. Each transaction must leave the database in a consistent state if the DB is consistent when the transaction begins. Issues:Effect of interleaving transactions, and crashes.

6. Transaction Fundamental Principles • Atomicity • A transaction must happen completely or not at all • A crash, power failure, error, or anything else won't allow you to be in a state in which only some of the transaction was applied. • Consistency • Data will be stay consistent after handling the transaction • None of the constraints on the data will ever be violated. • This does not guarantee correctness of the transaction • Consistent: (definition) • Consistent state: any given database transaction must change affected data only in allowed ways. • Inconsistent state example: if the transaction was partially applied on the database.

7. Transaction Fundamental Principles • Isolation • Each transaction runs as if alone • If two transactions are executing concurrently, each one will see the world as if they were executing sequentially • Durability • Once a transaction is complete, it is guaranteed that all of the changes have been recorded to a durable medium (such as a hard disk) • It cannot be undone, once completed!

8. Problems to avoid • Lost updates • Another transaction overwrites your change based on a previous value of some data • Inconsistent retrievals • You read data that can never occur in a consistent state • partial writes by other transactions • writes by a transaction that later abort

9. Lost update example

10. Temporary update problem One transaction updates a DB item and then the transaction fails for some reason. The updated item is accessed by another transaction before it is changed back to its original value.

11. The Incorrect Summary Problem One transaction is calculating an aggregate summary function on a number of records while other transactions are updating some of these records, the aggregate function may calculate some values before they are updated and others after they are updated.

12. Transaction States

13. Transaction API • begin() • To begin a transaction • commit() • Attempts to complete the transaction by changing the database • Note: The system might refuse the commit attempt, and initiate abort procedure. This means the transaction has failed. • rollback()/abort() • Aborts the transaction

14. Example • Problems due to concurrency: • Lost updates – your result was overwritten • Inconsistent retrievals – another transaction is modifying fields you are requiring

15. Solutions? • Poor: Global lock! • Only let one transaction run at a time • isolated from all other transactions • make changes permanent on commit or undo changes on abort, if necessary • Not efficient • We do wish to allow concurrent access. • Better: Lock objects as you need • Other transactions can execute concurrently • Easy to implement

16. Lock-Based Protocols A lock is a mechanism to control concurrent access to a data item Data items can be locked in two modes : exclusive (X) mode. Data item can be both read as well as written. X-lock is requested using lock-Xinstruction. shared(S) mode. Data item can only be read. S-lock is requested using lock-S instruction.

17. Locks alone are insufficient – outdated value!

18. Two-Phase Locking • Two phases in a transaction life: • Growing phase: acquire locks – incrementally! • Shrinking phase: release locks • Rule: You cannot lock anymore after your first release • Locks are implicitly released upon commit/abort. • Issues: • Deadlocks might happen! Resource ordering is important • Cascading Rollbacks: • If (T1) causes a failure and a rollback must be performed. • Other transactions dependent on T1's actions must also be rollbacked due to T1's failure, thus causing a cascading effect. • That is, one transaction's failure causes many to fail.

19. Example using two phase locks

20. 2 Phase Locking might suffer deadlocks t2.lock(bar); t2.lock(foo); • t1 might get the lock for foo, then t2 gets the lock for bar, then both transactions wait while trying to get the other lock t1.lock(foo); t1.lock(bar);

21. Strict vs Regular Two-Phase Locking

22. Preventing deadlock • Proposals: • Each transaction can get all its locks at once • Each transaction can get all its locks in a predefined order • Both of these strategies are impractical: • Transactions often do not know which locks they will need in the future!

23. Detecting deadlock • Construct a “waits-for” graph • Each vertex in the graph represents a transaction • T1 → T2 if T1 is waiting for a lock T2 holds • There is a deadlock iff the waits-for graph contains a cycle

24. “Ignoring” deadlock • Automatically abort all long-running transactions • Not a bad strategy, if you expect transactions to be short • A long-running “short” transaction is probably deadlocked

25. Timestamp Concurrency Control Algorithms Approach MajeedKassis

26. Timestamp Concurrency Control Algorithms • A transaction’s timestamp to coordinate concurrent access to data. • A timestamp is a unique identifier given by DBMS to a transaction that represents relative start time of a transaction: • Uniqueness: no equal timestamp values can exist. • Monotonicity: timestamp values always increase. • For each data Q, the protocol maintains two values: • W-timestamp(Q): largest timestamp of any transaction successfully wrote Q. • R-timestamp(Q): largest timestamp of any transaction successfully read Q.

27. Timestamp-based Transaction Execution Rules • Access Rule • If two transaction wish to access same data at the same time, older transaction gets priority. • Late Transaction Rule • If a younger transaction has written a data item, then an older transaction is not allowed to read or write that data item. • Younger Transaction Rule • A younger transaction can read or write a data item that has already been written by an older transaction.

28. When Xact T wants to read Object O • If TS(T) < WTS(O), this violates timestamp order of T w.r.t. writer of O. • So, abort T and restart it with a new, larger TS. (If restarted with same TS, T will fail again! • If TS(T) > WTS(O): • Allow T to read O. • Reset RTS(O) to max(RTS(O), TS(T)) • Change to RTS(O) on reads must be written to disk! This and restarts represent overheads.

29. When Xact T wants to Write Object O • If TS(T) < RTS(O), this violates timestamp order of T w.r.t. writer of O; abort and restart T. • If TS(T) < WTS(O), violates timestamp order of T w.r.t. writer of O. • Else,allow T to write O and update WTS(O) to TS(T).

30. Optimistic Algorithms for Concurrency Control • Based on the assumption that conflicts of database operations are rare. • Better to let transactions run to complete • Check for conflicts before they commit! Not while executing. • Does not require locking. • A transaction is executed without restrictions until it is committed following these phases: • Read phase. • Validation or certification phase. • Write phase.

31. Read Phase: Optimistic Algorithms • Updates are prepared using private (or local) copies (or versions) of the data members • The transaction reads values of committed data from the database. • Executes the needed computations • Makes the updates to a private copy of the database values. • It is conventional to allocate a timestamp to each transaction at the end of its Read to determine the set of transactions that must be examined by the validation procedure.

32. Validation Phase: Optimistic Algorithms • The transaction is validated to assure that the changes made will not affect the integrity and consistency of the database.  • The transaction goes to the write phase, on validation success. • The transaction is restarted, on validation failure. • The list of data members is checked for conflicts.  • If conflicts are detected in this phase, the transaction is aborted and restarted.   • The validation algorithm must check that the transaction has: • seen all modifications of transactions committed after it starts.

33. Write Phase: Optimistic Algorithms The changes are permanently applied to the database. The updated data members are made public.

34. Test 1 • For all i and j such that Ti < Tj, check that Ti completes before Tj begins. Ti Tj R V W R V W

35. Test 2 • For all i and j such that Ti < Tj, check that: • Ti completes before Tj begins its Write phase + • WriteSet(Ti) ) ReadSet(Tj) is empty. Ti R V W Tj R V W Does Tj read dirty data? Does Ti overwrite Tj’s writes?

36. Test 3 • For all i and j such that Ti < Tj, check that: • Ti completes Read phase before Tj does + • WriteSet(Ti) ReadSet(Tj) is empty + • WriteSet(Ti) WriteSet(Tj) is empty. Ti R V W Tj R V W Does Tj read dirty data? Does Ti overwrite Tj’s writes?

37. Final Notes: Optimistic Algorithms • Advantages: • Very efficient when conflicts are rare. • The rollback involves only the local copy of data. • Disadvantages: • Conflicts are expensive to deal with, since the conflicting transaction must be rolled back. • Longer transactions are more likely to have conflicts and may be repeatedly rolled back because of conflicts with short transactions. • Applications: • Only suitable for environments where there are few conflicts • Suitable for systems with the majority of their transactions are short. • Acceptable for mostly Read or Query database systems that require very few update transactions.

38. Distributed Transactions MajeedKassis

39. Distributed Database Management System A collection of multiple, logically interrelated databases distributed over a computer network. A distributed Database Management system is as the software system that permits the management of the distributed database and make the distribution transparent to the users. Data may be replicated and found at different sites!

40. Distributed transactions • Data stored at distributed locations • Failure model: • messages might be delayed or lost • servers might crash, but can recover saved persistent storage

41. The coordinator and the participants • Begins transaction • Assigns unique transaction ID • Responsible for commit/abort • Many systems allow any client to be the coordinator for its own transactions • The participants: The servers with the data used in the distributed transaction.

42. Problems with simple commit • “One-phase commit” • Coordinator broadcasts “commit!” to participants until all reply • What happens if one participant fails? • Can the other participants then undo what they have already committed?

43. Two-phase commit (2PC) • The commit-step itself is two phases • Phase 1: Voting • Each participant prepares to commit by writing log records, and votes on whether or not it can commit • Phase 2: Committing • Each participant actually commits or aborts

44. 2PC operations • canCommit?(T) -> yes/no • Coordinator asks a participant if it can commit • doCommit(T) • Coordinator tells a participant to actually commit • doAbort(T) • Coordinator tells a participant to abort • haveCommitted(participant,T) • Participant tells coordinator it actually committed • getDecision(T) -> yes/no • Participant can ask coordinator if T should be committed or aborted

45. The voting phase • Coordinator asks each participant: canCommit?(T) • Participants must prepare to commit using permanent storage before answering yes • Objects are still locked • Once a participant votes “yes”, it is not allowed to cause an abort • Outcome of T is uncertain until doCommit or doAbort • Other participants might still cause an abort

46. The commit phase • The coordinator collects all votes • If unanimous “yes”, causes commit • If any participant voted “no”, causes abort • The fate of the transaction is decided atomically at the coordinator, once all participants vote • Coordinator records fate using permanent storage • Then broadcasts doCommit or doAbort to participants

47. 2PC sequence of events

48. 2PC continued • Problem: • If the coordinator fails, then everyone is stuck • For instance, if everyone voted for commit but did not receive an answer, it is unknown whether the coordinator committed or aborted before failing. • Problem is solved in 3PC: • Send your vote to everyone • Wait for everyone’s vote, or until someone is suspected • If at least one node voted ‘no’ or at least one node is suspected to have failed, invoke Consensus(Abort) • Else, start Consensus(Commit)