Get off my thread: Techniques for moving work to background threads Anthony Williams Just Software Solutions Ltd https://www.justsoftwaresolutions.co.uk September 2020Get off my thread: Techniques for current thread? How do we move work off the current thread? Final Guidelines and QuestionsWhy do we need to move work off the current thread?Why do we need to move work off the current thread? Many environments environments have a dedicated thread for processing events: GUIs Client-Server applications Performing extensive processing on the event thread prevents other events from being handled.Why do we need

Back to Basics: Concurrency STD::THREAD int main() { std::thread my_thread{[](int z){ std::cout << "Hello from thread: " << z << "\n"; }, 42}; my_thread.join(); } Hello, world! https://godbolt Calls to acquire() and release() can happen on different threads https://en.cppreference.com/w/cpp/thread/counting_semaphoreCppCon 2023 9 David Olsen - Back to Basics: Concurrency CONCURRENCY “Multiple Make sure your code is thread safe with no data races SummaryCppCon 2023 42 David Olsen - Back to Basics: Concurrency STD::THREAD https://en.cppreference.com/w/cpp/thread/threadCppCon 2023 43 David

programmers? In this talk we provide a gentle introduction to concurrency with the modern C++ std::thread library. We will introduce topics with pragmatic code examples introducing the ideas of threads fundamental building blocks to write for instance a multi-threaded program ○ We are going to focus on std::thread for creating concurrent programs. ■ We will review briefly other methods of writing concurrent programs fundamental building blocks to write for instance a multi-threaded program ○ We are going to focus on std::thread for creating concurrent programs. ■ We will review briefly other methods of writing concurrent programs

Terrible task selection “fairness” Task Thread Thread Thread Task Task Task Task Task Task Task Back Front Task Queue Execute Task() Thread Thread PoolRethinking: Task Queues Problem #2 - starvation, load balancing, work stealing Task Thread Thread Thread Task Task Task Task Task Task Task Back Front Task Queue Execute Task() Thread Thread PoolRethinking: Task Queues Problem #3 - prioritization, etc, as enumerated previously Task Thread Thread Thread Task Task Task Task Task Task Task Back Front Task Queue Execute Task() Thread Thread PoolRethinking: Task Queues Summary: ● Queues

multithreaded programs? 4 Thread A int x = 0; start_thread_b(); x = 1; Thread B while (x != 1) {} Will this loop ever terminate? Will this write ever become “visible” to Thread B?The compiler can rewrite unambiguously “yes.” No other thread is allowed to look at the variable x while this thread is modifying it; and without some kind of synchronization, there’s no way to ensure that this thread isn’t modifying it new thread In C++03 pthreads, you’d create a new thread by calling a third-party library function. In C++11, the standard library “owns” the notion of creating new threads. To create a thread, you

channel.Reader(); thread xfutures/promises: “buffered channel” Channel channel; Channel::Reader reader = channel.Reader(); reader.Read(); // Blocks! thread xfutures/promises: Reader(); reader.Read(); // Blocks! Channel::Writer writer = channel.Writer(); thread x thread yfutures/promises: “buffered channel” Channel channel; Channel::Reader Channel::Writer writer = channel.Writer(); writer.Write("..."); // Non-blocking! thread x thread yfutures/promises: “buffered channel” Channel channel; Channel::Reader

Each core : 1 thread of execution (MIMD) • Cores share main memory, can share caches • Simultaneous multithreading (x86 / x64) -> 2N virtual cores • Query number of cores: std::thread::hardware_concurrency() System - CppCon 2021 7 / 39 Job Systems and Thread Pools • Starting threads is expensive -> Thread Pool • Job System = Thread Pool + Job Queues • Main Thread calls parallel_for() • Put jobs into queue queue • Threads take jobs out of queue Main Thread P P P P parallel_for() Main Thread I/O or Render Thread (load / store data from disk) Blocking waitHelmut Hlavacs – Combining Co-Routines and

Patterns Dealing with Sharing ▪ Copied Value ▪ Thread-Specific Storage ▪ Future Dealing with Mutation ▪ Scoped Locking ▪ Strategized Locking ▪ Thread-Safe Interface ▪ Guarded Suspension Concurrent Patterns Dealing with Sharing ▪ Copied Value ▪ Thread-Specific Storage ▪ Future Dealing with Mutation ▪ Scoped Locking ▪ Strategized Locking ▪ Thread-Safe Interface ▪ Guarded Suspension Concurrent synchronize when a thread takes its arguments by copy and not by reference. Data races or lifetime issues are not possible. copiedValueDataRace.cppThread-Specific Storage Thread-specific storage enables

‘tracer-tracee’ style application • Type reconstruction / Code Generation • Code injection • Process/thread management • Result processing • Non-interactive, configuration driven Object Introspection Inject OI into the target process 2. Wait for a thread to hit our trap... 3. Redirect the thread to run our code 4. Put the redirected thread back on its tracks 5. Process the collected data Inject OI into the target process 2. Wait for a thread to hit our trap... 3. Redirect the thread to run our code 4. Put the redirected thread back on its tracks 5. Process the collected data

// y = x; int r1 =rlx x; y =rlx r1; // x = 42; int r2 =rlx y; x =rlx42; Thread 1: Thread 2: 7 r1 = r2 = 42 is fine! reads from reads fromProprietary + Confidential atomic disallowed // y = x; int r1 =rlx x; y =rlx r1; // x = y; int r2 =rlx y; x =rlx r2; Thread 1: Thread 2: 8 r1 = r2 = 42 should be disallowed! reads from reads fromProprietary + Confidential int r2 =rlx y; // guess 42 x =rlx r2; // Confirm speculation Thread 1: Thread 2: 9 Formally each load observes the other thread’s store, as before.Proprietary + Confidential atomic