Asynchronous programming with futures and await

1. Asynchronous programming with futures and await Artur Laksberg Microsoft May 8th, 2014

2. Agenda Asynchrony with callbacks Futures and Continuations Composition with futures await

3. Hollywood Principle Don’t call us, we’ll call you!

4. Read From FIle: Naïve Solution Return a string, block on call: stringread_string_from_file(string file); string s = read_string_from_file("myfile.txt"); cout << s; Problem: blocking call

5. Read From FIle: Callback Solution Does not return a string, takes a callback that accepts a string: template<typenameFunc> voidread_string_from_file(stringfile, Func&& func); ... read_string_from_file("myfile.txt", [](strings) { cout << s; });

6. Concatenate Files: Naive Solution Read from one file, then read from another: template<typenameFunc> voidconcatenate_files(stringfile1, stringfile2, Func&& func) { read_string_from_file(file1, [func](stringstr1) { read_string_from_file(file2, [func](stringstr2) { func(str1 + str2); }); }); }

7. Concatenate Files: A “Better” Solution template<typenameFunc> voidconcatenate_files(stringfile1, stringfile2, Func&& func) { auto results = make_shared<result_holder>(); read_string_from_file(file1, [=](stringstr) { if (results->str) { func(str + *results->str); } else{ results->str = make_unique<string>(str); } }); read_string_from_file(file2, [=](stringstr) { if (results->str) { func(*results->str + str); } else{ results->str = make_unique<string>(str); } }); } structresult_holder { unique_ptr<string> str; }; ? Spot the defect!

8. std::future<T>std::shared_future<T>

9. Long-Running Work Counting lines in a file // Launch a task: future<int> work = async([] { returnCountLinesInFile(…); }); // Collect the results: cout << work.get(); Problem: blocking call

10. Proposal: Continuation (shared_)future::then // Launch a task: future<int> work = CountLinesInFileAsync(...); work.then([] (future<int> f) { // Get the result cout << f.get(); });

11. Binding Multiple Continuations future<T1> t = async([]() { return func1(); }).then ([](future<T1> n) { return func2(n.get()); }).then ([](future<T2> n) { return func3(n.get()); }).then ...

12. Advanced Composition Sequential Composition: f.then(A).then(B); Parallel Composition (only shared_future): f.then(A); f.then(B); Join and Choice (more on next page): autof = when_all(a, b); autof = when_any(a, b);

13. Join Create a future that completes when all arguments complete vector<future<int>> futures = get_futures(); autofutureResult = when_all (begin(futures), end(futures)) .then([](future<vector<future<string>>> results) { for (auto& s : results.get() ) // will not block { cout << s.get(); // will not block } });

14. Join Now with heterogeneous arguments future<int> future1 = ...; future<string> future2 = ...; autofutureResult = when_all(future1, future2) .then([](future<tuple<future<int>, future<string>>> results) { auto pair = results.get(); // will not block ... } });

15. Concatenate Files: The Right Solution Terse, efficient and safe future<string> concatenate_files(stringfile1, stringfile2) { auto strings = when_all(read_string_from_file(file1), read_string_from_file(file2)); returnstrings.then([]( future<tuple<future<string>, future<string>>> strings) { autopair = strings.get(); returnpair.get<0>.get() + pair.get<1>.get(); }); }

16. Concatenate Files: The Right Solution Terse, efficient and safe (with polymorphic lambdas) future<string> concatenate_files(stringfile1, stringfile2) { auto strings = when_all(read_string_from_file(file1), read_string_from_file(file2)); returnstrings.then([]( autostrings) { autopair = strings.get(); returnpair.get<0>.get() + pair.get<1>.get(); }); }

17. Choice Create a future that completes when at least one of arguments completes: vector<future<int>> futures = get_futures(); autofutureResult = when_any (begin(futures), end(futures)) .then([](future<vector<future<string>>> results) { for (auto& s : results.get() ) // will not block { if(s.ready()) cout << s.get(); // will not block } });

18. make_ready_future future<int> compute(intx) { if (x < 0) returnmake_ready_future<int>(-1); if (x == 0) returnmake_ready_future<int>(0); future<int> f1 = async([]() { returndo_work(x); }); return f1; }

19. is_ready future<int> f1 = async([]() { returnpossibly_long_computation(); }); if(!f1.is_ready()) { //if not ready, attach a continuation and avoid a blocking wait fl.then([] (future<int> f2) { int v = f2.get(); process_value(v); }); } // if ready, no need to add continuation, process value right away else { int v = f1.get(); process_value(v); }

20. awaitand resumable functions

21. Branches and loops .then .get future<string> read(stringfile) { return open(file) .then([=](istreamfi) { string ret, chunk; while ( ??? string read(stringfile) { istream fi = open(file).get(); string ret, chunk; while ((chunk = fi.read().get()).size()) ret += chunk; return ret; }

22. Branches and loops .then .get future<string> read(stringfile) { return open(file) .then([=](istreamfi) { auto ret = make_shared<string>(); auto next = make_shared<function<future<string>()>>( [=] { fi.read() .then([=](stringchunk) { if (chunk.size()) { *ret += chunk; return next(); } returnmake_ready_future(*ret); }); }); return (*next)(); }); } string read(stringfile) { istream fi = open(file).get(); string ret, chunk; while ((chunk = fi.read().get()).size()) ret += chunk; return ret; }

24. Await Example Get a value, convert to string: future<string> f() resumable { future<int> f1 = async([]() { returnpossibly_long_computation(); }); intn = await f1; returnto_string(n); }

25. Branches and loops, with await await .get String read(string file) { istreamfi = await open(file); stringret, chunk; while ((chunk = (await fi.read()).size()) ret += chunk; returnret; } string read(stringfile) { istream fi = open(file).get(); string ret, chunk; while ((chunk = fi.read().get()).size()) ret += chunk; return ret; }

26. Resumable Side Stack Simulation Thread #1 Thread #2 void foo() { task<bool> t = async_bar(); somefunc(); bool b = t.get(); } task<bool> async_bar() resumable { do_work(); ... // sync calls awaitasync_work(); // More sync calls ... returntrue; } task<void> async_work() resumable { do_work(); awaitcreate_task( [] { longrunning (); }); return; } Main Stack Side Stack Side Stack <completed> <blocked> done! somefunc(); bool b = t.get() t = async_bar() <suspended> foo(); async_work is done! Longrunning is done! <completed> … … return task<bool> <suspended> … … … • … … return task<void> • await create_task(); • await async_work(); do_work(); do_work(); … More sync calls … async_bar(); async_work(); return true; return;

27. References N3857: Improvements to std::future<T> and Related APIs N3858: ResumableFunctions N3970: Working Draft, Information technology – Programming languages, their environments and system software interfaces – C++ Extensions for Concurrency

28. “Concurrent programs wait faster” Tony Hoare