Asynchronous Tasks in C++11

Asynchronous Tasks in C++11 Bartosz Milewski

Task-Based Parallelism • Cilk (MIT, Cilk Arts, Intel Cilk+) • Parallel Haskell • Microsoft UMS Threads (User-Mode Scheduling) (Only 64-bit and Server) • Microsoft PPL (Parallel Patterns Library) • Intel TBB (Threading Building Blocks) • JVM/.NET-based languages and libraries

Task-Based Parallelism • Programmer: What can be run in parallel • Identify tasks • System: What will be run in parallel • Assign tasks to threads/processors • System: Load balancing, work stealing • System: Dealing with blocked tasks • Take them off thread, reuse thread • Create new UMS thread

Oversubscription Core 1 Core 2 Core 3 Core 4 Oversubscribed Idle Idle Idle Task 1 Task 2 Task 3 Task 4 Task 5 Task 6

Work Stealing Core 1 Core 2 Core 3 Core 4 Busy Busy Busy Busy Task 1 Task 3 Task 5 Task 6 Task 2 Task 4

Thread Reuse Core 1 Core 2 Core 3 Core 4 Busy Busy Idle Idle Task 2 Task 4

Tasks-Based vs. Thread-Based • Abstraction level • Threads: low • Tasks: high • Resource usage • Threads: heavy-weight • Tasks: lightweight • Problem solving • Threads: improve latency • Tasks: improve throughput • Level of parallelism • Threads: large grain • Tasks: fine grain

C++11 Threads and Tasks

The Thread, Promise, Future Combo void thFun(std::promise<std::string> & prms) { std::string str("Hello from future!"); prms.set_value(str); } void test() { std::promise<std::string> prms; std::future<std::string> ftr= prms.get_future(); std::threadth(&thFun, std::ref(prms)); std::cout << "Hello from main!\n"; std::string str = ftr.get(); th.join(); std::cout << str << std::endl; }

The Async Task, Future Combo std::string taskFun() { std::string str("Hello from task!"); returnstr; } void test() { std::future<std::string> ftr = std::async(&taskFun); std::cout << "Hello from main!\n"; std::string str = ftr.get(); std::cout << str << std::endl; }

C++11 Tasks (30.6.8) The template function asyncprovides a mechanism to launch a function potentially in a new thread and provides the result of the function in a future object with which it shares a shared state. • Launch Policies • launch::async • launch::deferred • launch::any (default) • Deferred tasks execute in the context of the forcing thread • What if the future is not forced?

thread_local variables • Global, file-static, class-static, function-static • Each thread initializes them separately • Each thread destroys them at the end • Tasks with launch::async • thread_locals must be destroyed • Before future::get (or future::wait) returns, or • Before future is destructed without forcing • Tasks with launch::deferred • thread_locals follow the lifetime of the forcing thread

Thread Reuse? • thread_local “as if” task_local • At task completion, destroy thread_locals • Starting a new task on an old thread: re-initialize all thread_locals • Problems • Library/runtime hooks for thread_local management • TlsAlloc • DLL_THREAD_ATTACH/DETACH calls to DllMain

Thread Sharing? • Take a blocked task off a thread • Detect blocked tasks • Save task state, including thread_locals • Restore a taks to a thread • Detect unblocked tasks • Restore task state, including thread_locals • Problems • Mutexes are thread-aware • Potential for spurious deadlocks (not really)

Conclusions • C++ tasks are ill-fitted for task-based parallelism • When designing a task-parallel library, consider • thread_localvs task-local • mutex (im-)mobility • See description of Lockable in the Standard • Blocking tasks

Bibliography • Blog: http://blog.corensic.com/2011/10/10/async-tasks-in-c11-not-quite-there-yet/ • C++11 tutorial: http://www.corensic.com/Learn/Resources/ConcurrencyTutorialPartFive.aspx • Proposals: • http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2009/n2880.html • http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3038.html

Asynchronous Tasks in C++11