Module 3 Transaction Processing Monitors

1. Module 3Transaction Processing Monitors COP 6730

2. Transaction Processing Monitor • Transactional RPC. • Functional Principles of the TP Monitor. • Managing Request and Response Queues. • Other tasks of TP Monitors • Load Balancing • Authentication and Authorization • Restart Processing

3. Layering of Services in a Transaction Processing System • Basic Operating System (BOS) • Transaction Processing Operating System (TPOS) • Transaction Processing Services (TRAPS) TP Monitor

4. Basic Operating Systems • The Basic Operating System (BOS) uses the hardware directly. • It knows little or nothing about transactions • Services include: Process/Thread scheduling, file system, simple sessions, authentication, etc.

5. Transaction Processing Operating System(TPOS) • TPOS renders the BOS objects and services in a transactional style. • Example Services: • Processes are bound to transactions • Messages must not be delivered until the transaction has committed

6. Transaction Processing Services(TRAPS) • TRAPS uses both BOS and TPOS to create a transaction-oriented programming environment. Services include: load control, resource managers, database, etc. • The application typically uses the TRAPS and TPOS, depending on the application's sophistication. Note: These layers do not reflect a strict implementation hierarchy. Rather, it describes a separation of concerns among different system components (e.g., TPOS can implement some of its data structure using SQL).

7. Transaction Processing (TP) Monitor

8. Transactional Remote Procedure Calls (1) Transaction control must be exercised over all resource manager interactions within one transaction (a “web” of the transaction’s calls): Control of participants: • The Transaction Manager (TM) must track which resource managers have been called. • At commit time, the TM must go out and ask each resource manager involved whether they agree to successfully terminate the transaction. Ready ? Yes Yes Yes

9. Transactional Remote Procedure Calls (2) • Preserving transaction-related information: • The same resource manager can be invoked more than once by a transaction • At commit each process running the same resource manager code is asked individually about its commit decision. • Sometime, the resource manager can only vote on commit when information pertaining to that transaction is in one place. • If the different invocations left their traces in different servers, the TRPC mechanism must provide a means to relate multiple invocations of the same resource manager to each other. Resource Manager (Server class) Server VOTE Server VOTE Server VOTE Resource Manager (Server class) VOTE

10. Transactional Remote Procedure Calls (3) Support of the transaction protocol: • The resource managers must stick to the rules imposed by the ACID paradigm. Resource Manager Regular code Transaction Manager Code for participating in transaction model Other Resource managers

11. CALLBACK • Each resource manager has a service interface that defines what the resource manager does for living. • Furthermore, it declares a number of callback entries, which can be called when certain events happen. Resource Manager Regular code Service interface Transaction Manager Code for participating in transaction model Callback entries Other Resource managers

12. CALLBACK • Each resource manager has a service interface that defines what the resource manager does for living. • Furthermore, it declares a number of callback entries, which can be called when certain events happen. • If the RM maintains any durable objects, it must be able to accept rollback and prepare/commit callbacks from the TM. • The RM exports the callback entries to the transaction manager when it first checks in with the TP monitor. • The number of callback entries provided by a RM depends on its level of sophistication. Note: This can result in multiple threads executing the RM’s code at different entry points.

13. CALLBACK ENTRIES • Boolean rm_Prepare(); /* invoked at phase 1 of commit. Returns vote. */ • Boolean rm_Rollback_Savepoint (Savepoint); /* rollback to requested savepoint number */ • Boolean rm_Commit (Boolean); /* invoked at phase 2 commit; param is decision */ • Boolean rm_Savepoint (); /* invoked when the transaction takes a savepoint */ • (void) rm_UNDO (&buffer); /* asks the resource mgr. to undo the effects */ /* described in the log record passed as parameter */ • (void) rm_Abort (); /* invoked at end of abort phase */ /* after all undo steps have been performed */ • (void) rm_REDO (&buffer); /* asks the resource mgr. to redo the effects */ /* described in the log rec. passed as a param */ • (void) rm_Checkpoint (); /* invoked by TM for taking a check point */ • (void) rm_restart (LSN); /* first callback from the transaction manager at */ /* restart; passes the address of the resource */ /* manager’s recent checkpoint record */ The invocation of a resource manager by the transaction manager at such callback entry is just another transactional remote procedure call.

14. Resource Manager’s Interfaces Three groups of interfaces: • The interface the resource manager exports, • the interfaces it uses other general resource managers, and • the system resource manager interfaces it uses to support the transaction protocol.

15. Install A New Resource Manager (1) RMID rmInstall (RMNAME, &rm_callbacks, AccessControlList, /* authorize requests*/ stuff); /*other info. For TP monitor*/ • RMNAME is the user-supplied global external ID. • &rm_callbacks is an array of pointers to the callback entries. • These callback entries are specified as relative addresses in the load module for that RM. • The TP monitor keeps this array in the TRPC stub of the respective RM’s address space.

16. Install A New Resource Manager (2) RMID rmInstall (RMNAME, &rm_callbacks, AccessControlList, /* authorize requests*/ stuff); /*other info. For TP monitor*/ • AccessControlList allows the TP monitor to check whether an incoming request for that RM is acceptable from a security point of view. • Stuff denotes remaining parameters: • The location from which the code for the resource manager can be loaded. • Which other resource managers need to be available before this one can start, etc.

17. Complex Interaction If a client C repeatedly invokes server class C, how should that be handle? We need to consider three scenarios. Give me the next record

18. Complex Interaction If a client C repeatedly invokes server class C, how should that be handle? We need to consider three scenarios. Give me the next record Mail must have header, body, and trailer

19. Case A: Independent Invocations • A server class S can be called arbitrarily often, and the outcome of each call is independent of whether or not the server class has been called before. Example: • The server reads records from a database. • It performs some statistical computations on them, and returns the result. • Since the server keeps no state about the call, it can declares its consent to the transaction’s commit upon return.

20. Case B: Invocation Sequence • The client wants to issue requests that explicitly refer to earlier service requests. Example: The mini-batch example updates a number of records at a time (in a subtransaction) using a cursor. • The final decision of the server class depends on the outcome of the last invocation. I can decide for this server class

21. Case C: Complex Interaction • Each call to S is independent, except for some global integrity constraints that cannot be checked until commit and that may depend on the results of all previous invocations. • The server class must remember the results of all invocations by the client until commit.

22. Complex Interaction – Example • A mail server stores the various parts of a message in a database. • It maintains a consistency constraint that says that all mails must have at least a header, a body, and a trailer. There must be some way of relating all the updates done by the same client when the server is called (back) at rm_Prepare.

23. Implementing state associated with a client-server interaction • Context is maintained by a communication session. • Context is passed back and forth. • The servers write context information to a database. • The context is stored in a segment of shared memory for all servers of the same class. Context information about multiple invocations of the same server class can be maintained in four different ways:

24. Context Management: Communication Session • Such a session has to be managed by the TPOS. • All the client or the server has to do is to declare that it needs one. • Typically, the request for a session is issued by the server. Applications do not want to be bothered with TPOS. • The price for this is considerable overhead in the TPOS for maintaining and recovering sessions. Subsequence requests go to the same server Session Client Server S1 Request/ Response Context In local variables Context In local variables

25. Session Management • If context maintenance is handled thru communication session, then the TP monitor is responsible for binding a server process to one client. • A BindID uniquely identifies an association between a client instance and a server instance. typedef struct{ /* */ rmInstance EndA; /* one end of stateful interaction */ rmInstance EndB; /* other end of stateful interaction */ Uint SeqNo; /* sequence no. to distinguish between multiple stateful interactions between the same servers. */ }BindId; /* handle for a stateful interactions */ BindID rmBind (rmInstance); /* function passes the ID of the client to which */ /* a session with the server has to be established. */ /* gets back a handle which identifies the interaction */ /* between this server and this client. */ /* returns the Null instance if binding fails. */ Boolean rmUnbind (BindID) /* inverse to rmBind, waives the future use of the */ /* specified binding, returns FALSE if no binding */ /* with that identifier was in effect */

26. Context Mgmt. Technique: Context Passing (1) • Context is passed back and forth explicitly between client and server upon each request and reply. This approach relieves the TPOS of the problem of context management.

27. Context Mgmt. Technique: Context Passing (2) I do not have “context” Since rm_Prepare call from the TM carries no context in its parameter list, the commit may have to be done as follows: • The client issue a “final” service call (with the “final” context), which tells the server that the client is about to call commit. • When the server returns normally, the client can call Commit_Work, and the server will not be called back by the Transaction Manager Context ? Transaction Manager Prepare Server S3

28. Context Mgmt. Technique: Keep context in Database (1) • The servers keep the context in a database. An arbitrary instance of the server can be invoked to vote on commit. • The state information must be supplied with the key attributes: • TRID • Client RMID • Sequence number of the invocation in order to uniquely identify which thread of control it belongs to (see next slide).

29. Key Attributes to Access Context Information Server Class Trans. T, Client A1, 2nd call A resource manager Transaction T 1 A1 2 Transaction B A2 B1 B2

30. Context Mgmt. Technique: Keep context in Database (2) • Some of the work involved in maintaining the context can be off loaded to either an SQL database system or to the context management service of the TP monitor. SQL DBMS

31. Context Mgmt. Technique: Shared Memory (1) • All servers of the same class share a segment of read/write memory in which invocation context is kept. • Synchronization on the shared memory is done by the server.

32. Context Mgmt. Technique: Shared Memory (2) • This scheme is similar to the solution using a context database, except that now the whole responsibility is with the server (class). • This solution is used by very sophisticated resource managers, such as SQL database system.

33. Client-Oriented Context • Client-Oriented Context reflects the state of an interaction between a client and a server. • In the last few pages, we assumed this type of context.

34. Transaction-Oriented Context This type of context is bound to the SoC established by a transaction rather than to an isolated client-server interaction. time t1 time t2 Server Class S1 Server Class C Server Class S3 Server Class S2 time t3 time t4(needs the context established by the earlier call to S3 from S1) Note: The context needed by S3 is not bound to any of the previous interactions with S2, but it is bound to the transaction as such.

35. General Context Management • A general context management scheme must be able to cope with both type of state information. • The ground rule: • Each server that manages persistent objects must be implemented such that it can keep transaction-oriented context. • The TRPC mechanism must provide means for servers of any type to establish a stateful interaction with a client in case this should be needed (client-orient context). Note: Communication sessions can only be used to support client-oriented context. (We will discuss transaction-oriented management techniques in more details later.)

36. Managing Resource Managers • The TP monitor’s primary function is to handle TRPCs • TPOS maintains a core data structures to manage the resource managers and the resources pertaining to them This is what a typical TP monitor will put atop a conventional O.S.

37. Anchor of TPOS TPOS_Anchor is one well-defined point in TPOS, from which all the system data structures can be reached. typedefstruct anchor * TPOS_AnchorP; typedefstruct anchor /* TPOS anchor for the global control blocks */ { ... TPAnchorPTPMonCBs; /* pointer to the anchor of the TP monitor */ TMAnchorP TM_CBs; /* pointer to the anchor of transaction mgr. */ CMAnchorP SM_CBs; /* pointer to the anchor of session (comm.) mgr. */ LMAnchorP LM_CBs; /* pointer to the anchor of log mgr. */ IMAnchorP IM_CBs; /* pointer to the anchor of isolation (lock) mgr. */ } TPOS_Anchor • Each system RM (e.g., transaction manager) has its own local anchor, where this RM’s data structures are rooted. • The declaration shows five system RMs. This is a basic set; real system might have more.

38. Anchor of TPOS - Rationale • TPOS comes up before any of the RMs is activated upon system startup • Keeping the pointer to the anchors of system RMs in TPOS_Anchor ensures that addressability can be established in an orderly manner upon system startup. TPOS TPOS _Anchor System RM System RM System RM

39. TP Monitor’s Anchors • The anchor of a system resource manager has no fixed layout. • The TP monitor’s anchor might have the following structure: typedefstructtpa * TPAnchorP; typedefstructtpa /* TP monitor anchor for its control blocks */ { char *MyVersion’ /* version of the TP monitor running */ char Repository[64]; /* name of the repository I work with */ /* repository holds description of RM’s */ handle Repos_Handle; /* handle for calls to repository */ char ContrextDB; /* name of database for keeping context */ handle CtxDB_Handle; /* handle for calls to context database */ RMID NextRMID; /* next RMID to be handed out */ RMCB FirstRMCB; /* pointer to 1st res. mgr. control block */ PCB FirstPRCB; /* pointer to 1st process control block */ SECB FirstSECB; /* pointer to 1st session control block */ } TPAnchor; • These system control blocks are implemented as semaphores to make sure that a process can operate on these control data structures without being disturbed by other processes Description of resources managed by TP Monitor

40. Central Control Blocks • Descriptive data about processes, transactions, and RMs are kept in central control blocks. • There is one system RM that is responsible for and encapsulates each type of control block. Examples (also see next page): • TP monitor is responsible for the resource manager control blocks, the process control block, and the session control blocks (see last page). • The transaction manager is responsible for the transaction control block, and so on.

41. Central Data Structure A system RM A central control blocks There is one system RM that is responsible for and encapsulates each type of control block.

42. Accessing Central Data Structure of TPOS • The following function can be called from any process: PID MyProcid (); /* returns the identifier of the process the caller is running in */ /* this is actually a call to the basic OS */ TRID MyTrid (); /* returns the transaction identifier the caller is executing within */ RMID MyRMID (); /* returns the RMID of the RM that has issued the call */ RMID ClientRMID (); /* returns the RMID of the caller’s client */ • The following function can only be called from system resource manager: PCB MyProc (); /* returns a copy of the central process control block */ /* describing the process the caller is running in */ TransCB MyTrans (); /* returns copy of the central transaction control block */ /* describing the transaction the caller is working for */ RMCB MyRM (); /* returns copy of the central resource manager control block */ /* describing the res. mgr. that issued the call */ RMCB ClientRM (); /* like MyRM, but for the caller’s client */ PCB * MyProcP (); /* returns a pointer to the central process control block */ /* describing the process the caller is running in */ TransCB * MyTransP (); /* returns a pointer to the central transaction control block */ /* describing the transaction the caller is working for */ RMCB * MyRMP (); /* returns a pointer to the central resource mgr. control block */ /* describing the res. mgr. that issued the call */ RMCB * ClientRMP (); /* like MyRMP, but for the caller’s client */

43. MyTrans vs MyTransP • MyTrans gets a copy of all descriptive data for the current transaction. • MyTransP provides, in addition, linkage information to other relevant control blocks.

44. Protection Domains • Subsystems typically want to protect themselves from faults in the application (i.e., caller.) • They want to encapsulate their data so that only they can access them directly. • Such an encapsulated environment is called a protection domain. • The system RMs should operate as protection domains

45. Two Ways to Provide Protection Domains • Process as protection domain • Address space as protection domain

46. Process as Protection Domain • Each process has private address space. • Service requests are handled by switching processes; that is, by sending a message to the server process. Note: A domain switch requires a process switch, and it may require copying parameters and results between the processes. Example: UNIX RM1 Process 1 Address 1 Service request Copy parameters Address 2 Process 2 RM2

47. Address Space as Protection Domains • A process has many address spaces, one for each protected callee and one for the caller. • A service request is handle by switching address spaces, i.e., caller’s process can execute code of the callee • The address space protection domain of the callee contains: • some of the caller’s segments, and • the address space belonging to the callee.

48. Switching Address Space Running my own code Call RM2 Switching address space Process 1 Primary Address 1 (RM1) Switch Address 2 (RM2) Address 3 (RM3) Primary process of RM 1 This is more efficient than process switch. Example: IBMAS/400, IBM MVS/XA, VAX-VMS.

49. Association of Processes with RMs • Each process belongs to a fixed server class, and has a RMID associated with it (i.e., ID of the primary RM) • At run time, however, a process can switch to other address spaces (by issuing TRPCs) which run for other RMIDs • This ability of a process to execute in different spaces requires the system calls MyRMID and ClientRMID (example on next page).

50. Association of Processes with RMs Note: A process cannot switch to any address space for both scheduling and protection reasons.