Processes: code migration

1. Processes: code migration

2. Process Concept • An operating system executes a variety of programs: • Batch system – jobs • Time-shared systems – user programs or tasks • Process – a program in execution; process execution must progress in sequential fashion • A process includes: • program counter • stack • data section

3. Esempio: kernel linux 2.6.20.4 • struct task_struct { • volatile long state; /* -1 unrunnable, 0 runnable, >0 stopped */ • … • #ifdef CONFIG_SMP • #ifdef __ARCH_WANT_UNLOCKED_CTXSW • int oncpu; • #endif • #endif • int prio, static_prio, normal_prio; • cpumask_t cpus_allowed; • unsigned int time_slice, first_time_slice; • … • /* task state */ • struct linux_binfmt *binfmt; • long exit_state; • int exit_code, exit_signal; • … • pid_t pid; • pid_t tgid;

4. Esempio: kernel linux 2.6.20.4 • #ifdef CONFIG_CC_STACKPROTECTOR • /* Canary value for the -fstack-protector gcc feature */ • unsigned long stack_canary; • #endif • … • /* process credentials */ • uid_t uid,euid,suid,fsuid; • gid_t gid,egid,sgid,fsgid; • struct group_info *group_info; • kernel_cap_t cap_effective, cap_inheritable, cap_permitted; • … • /* CPU-specific state of this task */ • struct thread_struct thread; • /* filesystem information */ • struct fs_struct *fs; • /* open file information */ • struct files_struct *files; • /* namespaces */ • struct nsproxy *nsproxy; • /* signal handlers */ • struct signal_struct *signal; • struct sighand_struct *sighand; • … • };

5. Diagram of Process State

6. CPU Switch From Process to Process

7. Single and Multithreaded Processes

8. User level threads • Cheap to create and destroy threads • Thread administration is in user-space • Create: allocating memory for a thread stack • Destroy: freeing memory of the stack • Create and Destroy are cheap • Switching between threads is fast • Store the CPU registers of the current thread • Load the ones for the next thread

9. User level threads: Problem? • Invocation of a blocking call • immediately block the entire process • including all the threads of that process • eg: blocking on I/O • A thread blocking on I/O should never prevent other parts from being executed

10. Kernel Level threads • Implement threads in the OS’s kernel • Create, delete, synchronization, … • all take place via a system call • Switching threads is as expensive as switching processes • So, lose the benefit of using a thread instead of a process

11. Windows XP Threads • Implements the one-to-one mapping • Each thread contains • A thread id • Register set • Separate user and kernel stacks • Private data storage area • The register set, stacks, and private storage area are known as the context of the threads • The primary data structures of a thread include: • ETHREAD (executive thread block) • KTHREAD (kernel thread block) • TEB (thread environment block)

12. Linux Threads • Linux refers to them as tasks rather than threads • Thread creation is done through clone() system call • clone() allows a child task to share the address space of the parent task (process)

13. Thread Usage in Nondistributed Systems • Context switching as the result of IPC

14. Thread Implementation • Combining kernel-level lightweight processes and user-level threads.

15. Multithreaded Servers (1) • A multithreaded server organized in a dispatcher/worker model.

16. Multithreaded Servers (2) • Three ways to construct a server.

17. The X-Window System • Provides GUI interface to UNIX based systems • A protocol, not a product • Operating system independent • X win implementation: • Motif - Open Software Foundation (OSF) • Open Look - Sun/AT&T • The Client/Server terminology is REVERSED for X Windows • Client: the application which requests the GUI display and provides the service. Located on the server node • Server: Program that provides the GUI display

18. The X-Window System • The basic organization of the X Window System

19. Servers: General Design Issues • Client-to-server binding using a daemon as in DCE • Client-to-server binding using a superserver as in UNIX 3.7

20. Reasons for Migrating Code • The principle of dynamically configuring a client to communicate to a server. The client first fetches the necessary software, and then invokes the server.

21. Degrees of Mobility

22. System Examples

23. Control Function/object transfer Function Object f( ) Argument transfer Return value Remote Execution • Procedure code is sent together with arguments. • Server behaves like a general cycle server. • Server can evolve itself. Client Server Main Program Dispatcher Function Object Remote execution Arguments

24. Control Request a remote function/object Function Object Function/object itself is returned. Locally executed Code on Demand • Server behaves like a general remote object server. • A remote function/object is sent back as a return value. • Client executes the function/object locally. • Client execution control stays in local while suspended upon a request to a server. Client Server Main Program func( ) Dispatcher Remote Function Object

25. Time Source Site Destination Site Process P1 : : : : Execution suspended Freezing time Transfer of control Execution Resumed : : : : Process P1 Process Migration • Selecting a process to be migrated • Selecting the destination node • Suspending the process • Capturing the process state • Sending the state to the destination • Resuming the process • Forwarding future messages to the destination

26. Process MigrationBenefits • Better response time and execution speed-up • Dynamic load balancing among multiple nodes • Using a faster CPU • Higher throughput and Effective resource utilization • Migrating I/O and CPU-bound processes to file and cycle servers. • Reducing network traffic • Migrating processes closer to the resources they are using most heavily. • Improving system reliability • Migrating processes from a site in failure to more reliable sites • Replicating and migrating critical processes to a remote.

27. Process MigrationState Capturing • CPU registers • Captured upon a freeze • Address space • Difficult to restore pointers • I/O state: • Fast I/O Operations • Completed before a process migration • Durable I/O Operations like files and user interactions • Difficult to carry files in use and to freeze/restore system calls. • Necessity to maintain a connection with I/O established at the source node. • Some popular files available at the destination node

28. Code Migration Issues • Not always possible to migrate resources • A TCP/IP connection, for example • Transferring a reference is not always a problem • Eg: A URL to a file • 3 Types of process-to-resource bindings • Binding by Identifier • Binding by Value • Binding by Type

29. Code Migration Issues • Binding by Identifier (Strongest form) • process precisely requires the identified resource • Eg: refer to an FTP server by its IP address • Binding by Value • only the value of the resource is needed • another resource can provide the same value (C libraries) • Binding by Type (Weakest form) • just need a resource of specific type • Eg: a local printer, monitor, …

30. Code Migration Issues • Unattached Resources • can be easily moved between various machines • Eg: data files associated with the program • Fastened Resources • Very expensive to move these • Eg: Local databases, complete Web sites • Fixed Resources • intimately bound to a specific machine • Eg: local devices

31. Initiation of Migration • Operating system • when goal is load balancing • Process • when goal is to reach a particular resource

32. Migration • Must destroy the process on the source system • Process control block and any links must be moved

33. What is Migrated? • Eager (all):Transfer entire address space • no trace of process is left behind • if address space is large and if the process does not need most of it, then this approach my be unnecessarily expensive

34. Precopy • Precopy: Process continues to execute on the source node while the address space is copied • pages modified on the source during precopy operation have to be copied a second time • reduces the time that a process is frozen and cannot execute during migration

35. What is Migrated? • Eager (dirty): Transfer only that portion of the address space that is in main memory • any additional blocks of the virtual address space are transferred on demand • the source machine is involved throughout the life of the process • good if process is temporarily going to another machine • good for a thread since the threads left behind need the same address space

36. Copy on reference • Copy-on-reference: Pages are only brought over on reference • variation of eager (dirty) • has lowest initial cost of process migration

37. What is Migrated? • Flushing: Pages are cleared from main memory by flushing dirty pages to disk • later use copy-on-reference strategy • relieves the source of holding any pages of the migrated process in main memory

38. Process MigrationAddress Transfer Mechanisms Pretransferring Transfer-on-reference Total Freezing Source node Source node Source node Destination node Destination node Destination node Suspended Migration decision Migration decision Migration decision Suspended Freezing time Transfer of address space On-demand transfer Transfer of address space resumed Freezing time Suspended Freezing time resumed resumed Merits: quick migration Demerits: large memory latency Merits: easy implementation Demerits: long delay time Merits: freezing time reduce Demerits: total time extended

39. Origin Receiver Sender Resend Migrate Dest 1 Resend again Migrate again Dest 2 Process MigrationMessage Forwarding Mechanisms Resending messages Ask origin site Origin Receiver Sender Send Send Migrate Forward Dest 1 Migrate again Dest 2

40. Link Link Process MigrationMessage Forwarding Mechanisms (Cont’d) Link traversal Link Update Origin Origin Receiver Sender Receiver Sender Send Send New location Forward Send Migrate Migrate Send Dest 1 Dest 1 New location Forward Migrate again Send Migrate again Current location Dest 2 Dest 2

41. Process MigrationHeterogeneous Systems • Using external data representation • Floating-point data • External data representation must have at least as much space as the longest floating-point data representation • Process migration is restricted to only the machines that can avoid the over/underflow and the loss of precision. • Architectural-dependent data representation • Signed-infinity and signed-zero • In general, process migration over heterogeneous systems are too expensive • Conversion work • Architectural-dependent representation handling • Always interrupting external data representation • Java

42. Process migration examples • Early work • XOS, Worm, Butler, DEMOS/MP • Transparent migration in Unix-like systems • Locus, OSF/1 AD, MOSIX, Sprite • OS with Message-passing interface • Charlotte, Accent, V Kernel • Microkernels • RHODOS, Arcade, Chorus, Amoeba, Birlix, Mach • User-space migrations • Condor, Migratory PVM, LSF, … • Application-specific migrations • Freeman, Skordos, Bharat & Cardelli (Migratory Applications) • Mobile objects • Emerald, SOS, COOL • Mobile agents: derived from 2 fields • AI • Distributed systems • Telescript, Agent Tcl, TACOMA, Mole, ..

43. Distributed Global States • Operating system cannot know the current state of all process in the distributed system • A process can only know the current state of all processes on the local system • Remote processes only know state information that is received by messages • these messages represent the state in the past

44. Thread Migration • Only thread stack and registers copied • All shared resources of the process remain on the source machine • Assumptions: • A process for the same program has been started in the destination machine • The code is in the same virtual memory area on both machines • Problems: • Pointers to stack • Pointers to heap • Heap accessed by multiple threads • …

45. Thread Migration Problems • Heap Pointers • Disallow the use of the heap • Virtual common heap by a DSM • Stack Pointers • Pointer manipulation • Preventive stack reservation

46. Thread migration goals • Exploitation of resource locality • Accessing more processing power • Resource sharing • Fault resilience • Load balancing

47. Thread Migration Examples • Emerald • Ariadne • Amber • Millipede • UPVM • Active Threads • …

48. Process Migration Mechanism • Condor (Univ. of Wisconsin) • Harvesting Idle Machines from Cluster of Workstations • Checkpoint-based Migration • Homogeneous system • Checkpointing/Migration • No source code change • Link with Condor Checkpointing Library • New signal handler • System call wrapper • Migration Invocation • Asynchronously using signal: sig_checkpoint

49. Stack Heap Uninitialized data Data initialized data Text Process Migration in Condor(1) • Process Address Space • Text, data and stack

50. Process Migration in Condor(2) • Open Files • State: file descriptor number, mode, offset, dup info • Use system call augmentation • Wrapper open(), dup(), lseek(), close(), etc. • Catch info and store at process’ data space, • Then, call original system call • Signals • Signal handling attr • block/ignore, default/custom_handler • Store using sigprocmask(), sigaction() • Pending signals • Use sigispending()