1 / 14

Checkpoint and Restart Mechanisms (CPR)

Checkpoint and Restart Mechanisms (CPR). Yavor Todorov. Contents . Introduction How it works OS level checkpointing Application level checkpointing CPR for parallel programing CPR functionality References . Introduction.

dale
Download Presentation

Checkpoint and Restart Mechanisms (CPR)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Checkpoint and Restart Mechanisms (CPR) YavorTodorov

  2. Contents • Introduction • How it works • OS level checkpointing • Application level checkpointing • CPR for parallel programing • CPR functionality • References

  3. Introduction • Basically, checkpoint/restart mechanisms allow a machine that crashes and is subsequently restarted to continue from the checkpoint with no loss of data, just as if no failure had occurred. • At some firms and supercomputing centers, it's common practice to break up long-running computational programs into several batches. Programs such as a gene- sequencing application search through enormous databases and execute complex algorithms that can take several weeks to complete. • But while the concept is easy to understand, the technical mechanism to checkpoint and restart an operating system or application is quite complex.

  4. How it works • Checkpointing can occur either within the operating system or at the application level. • Most high end mainframes have automated CPR utilities • CPR at the operating system level saves the state of everything that's being done within a given application at periodic checkpoints and allows the system to restart from the last point. • On very large computers with hundreds or thousands of processes running, saving the entire state of an operating system can take a long time

  5. How it works (cont’d) • It also takes a long time to later restart the machine at that state - on large jobs, it could take several hours. • The recovery is delayed because a large amount of data must be stored, whether or not the application requires that information to fully restart it. • When a process is checkpointed register set, file handlers

  6. Process image • Text • Data • Stack • Heap • Register set values • Status of open files • Sockets • Signals.

  7. Checkpoint at OS level • "Checkpointing at the operating system is useful but very costly, in that the operating system does not know what data the application really needs to restore it later, so it blindly saves everything," according to James Kasdorf , director of supercomputing center • At OS level CPR saves system state. That includes unneeded copies of data, program code and system libraries • Most supercomputing centers try to avoid CPR at OS level

  8. Checkpointing at OS level drawbacks • Since the whole process image is save its an expensive operation • Takes more time • Usually needs kernel modifications since most OS like Linux were not built with CPR functionality

  9. Checkpoint at application level • The application uses OS hooks to save information needed for restart • More efficient in a way that it takes less time to checkpoint and it is faster to restart the application • It allows you to choose optimal point which is typically at the end of a loop • Only needed data gets saved

  10. CPR at app level drawbacks • Difficult in some cases. E.g. application has an open communications channel to an external device or the application runs on a clustered computer. • Distributed application’s state is hard to save as programs state is changing across multiple nodes • CPR for apps with large buffer memory takes longer

  11. CPR for parallel programs • Each process is responsible for taking its own checkpoint. • Checkpoint timing is responsibility of a coordinating process. • CPR data includes: in-transit message data, data section, file offsets, signal state, executable information, stack contents and register contents, CPU state, info about open files, pending signals. • Checkpoint file can be stored either on local or global storage. • When program is restarted each process initiates its own restart.

  12. CPR for parallel programs • All migrating processes have to be stopped at the time, to avoid loss of a signal • Socket IP addressing space have to be taken in consideration( it can be virtualized) • I/O speeds are pivotal for any CPR process ( the faster the better)

  13. CPR functionality • Process migration • Load balancing • Crash recovery • Rollback transaction • Job control

  14. Sources • Duell, J. (2005). The design and implementation of Berkeley Lab's linux checkpoint/restart. Berkeley: Lawrence Berkeley National Lab. • Litzkow, M., Tannenbaum, T., Basney, J., & Livny, M. (1997). Checkpoint and Migration of UNIX Processes in the Condor Distributed Processing System. • Zhong, H., & Nieh, J. (2001). CRAK: Linux Checkpoint/Restart As a Kernel Module.Depertment of CS Columbia University. • http://www.computerworld.com/s/article/68930/Checkpoint_and_Restart

More Related