C++ Memory Synchronization Variables

1. Lecture14 The C++ Memory model Synchronizationvariables Implementingsynchronization

2. Memorystrides • Some nearestneighborsthatarenearbyinspacearefar apart inmemory • Stride=memorydistancealongthe directionweare moving:Nalongthe verticaldimension • Missratemuchhigherwhenmovingverticalstripsof datathanhorizontalones–thepaddingcode Dr D1mr ------------------------------------------------------------------- 1,381,488,01750,566,005 solve.cpp:solve(...) 10,000 516,000 1,999 for (i = 0; i < (n+3);i++) 66,000 // Fillsin // TOPRIGHT Eprev[i] =Eprev[i +(n+3)*2]; 10,000 516,000 0 504,003 for(i=(n+2);i<(m+3)*(n+3);i+=(m+3))//RIGHTSIDE Eprev[i] =Eprev[i-2]; Scott B. Baden / CSE 160 /Wi '16

3. What problems may arise when copying the left and right sides, assuming each thread gets a rectangular region and it shares values with neighbors that own the outer dashedregion? A. Falsesharing Dataraces A & Bonly All for(i=(n+2);i<(m+3)*(n+3);i+=(m+3)) Eprev[i] =Eprev[i-2]; B. Poor data reuse incache Scott B. Baden / CSE 160 /Wi '16

4. What problems may arise when copying the left and right sides, assuming each thread gets a rectangular region and it shares values with neighbors that own the outer dashedregion? A. Falsesharing Dataraces A & Bonly All for(i=(n+2);i<(m+3)*(n+3);i+=(m+3)) Eprev[i] =Eprev[i-2]; B. Poor data reuse incache Scott B. Baden / CSE 160 /Wi '16

5. What problems may arise when copying the top and bottom sides, assuming each thread gets a rectangular region and it shares values with neighbors that own the outer dashedregion? False sharingSome false sharing is possible, thoughnot signficant Poor data reuse incache C. Dataraces A & Bonly None for (i = 0; i < (n+3); i++) Eprev[i]=Eprev[i+(n+3)*2]; Scott B. Baden / CSE 160 /Wi '16

6. What problems may arise when copying the top and bottom sides, assuming each thread gets a rectangular region and it shares values with neighbors that own the outer dashedregion? False sharingSome false sharing is possible, thoughnot signficant Poor data reuse incache C. Dataraces A & Bonly None for (i = 0; i < (n+3); i++) Eprev[i]=Eprev[i+(n+3)*2]; Scott B. Baden / CSE 160 /Wi '16

7. What does indivisibilitymean? The variable is incremented as if anindivisible operation The variable is incremented as if in acritical section The variable is incremented as if in acritical section followed by abarrier E.A&C D.A&B std::atomic<int> value; value++; Scott B. Baden / CSE 160 /Wi '16

8. What does indivisibilitymean? The variable is incremented as if anindivisible operation The variable is incremented as if in acritical section The variable is incremented as if in acritical section followed by abarrier E.A&C D.A&B std::atomic<int> value; value++; Scott B. Baden / CSE 160 /Wi '16

9. Theinterface • C++ provides atomic variants of the major built intypes • Usual arithmetic operations • Assignment operators such as +=, ++ where appropriate • Assignment isdifferent • No copy or assignmentconstructors • Instead, there are load( ) and store( ) operations • We can assign or copy from or to a non-atomic type • atomic<int> x=7; int y =x; • atomic <int>z=x+2; • We will use the sequentially consistent variant(default) • memory_order_seq_cst Scott B. Baden / CSE 160 /Wi '16

10. What can go wrong if the assignment returned a reference? g The modification of y could affectx Both atomic<int> x =3; int y =x; y++; A. Another thread could modify x beforeassignin itsvalue Scott B. Baden / CSE 160 /Wi '16

11. What can go wrong if the assignment returned a reference? g The modification of y could affectx Both atomic<int> x =3; int y =x; y++; A. Another thread could modify x beforeassignin itsvalue Scott B. Baden / CSE 160 /Wi '16

12. Summary of Atomics • Assignmentinvolvingatomicsisrestricted • No copy or assignment constructors, these areillegal // Some C++ documentation permitsthis! atomic<int> x=7; atomic<int> u =x atomic<int>y(x); • We can assign to, or copy from, or to a non-atomictype • x=7; • int y =x; • We can also use direct initialization involvingconstants • atomic<int>x(0) • Wewillusethesequentiallyconsistentvariant(default) • memory_order_seq_cst • We only need to use the atomic::load() and store() functions if we requireanothermemoryconsistencymodel;thedefaultcanpenalize performancehttp://en.cppreference.com/w/cpp/atomic/memory_order • memory_order_relaxed Scott B. Baden / CSE 160 /Wi '16

13. Memorymodels • Earlier we discussed cache coherence andconsistency • Cache coherence is a mechanism, a hardware protocol to ensure that memory updates propagate to other cores. Cores will then be able to agree on the valuesof information stored in memory, as if there were no cache atall • Cache consistency defines a programmingmodel: when do memory writes become visible to other cores? • Defines theorderingof of memoryupdates • Acontractbetweenthehardwareandtheprogrammer:ifwe follow therules,the theresultsofmemoryoperationsare guaranteed bepredictable Scott B. Baden / CSE 160 /Wi '16

14. The C++11 Memorymodel • C++ provides a layer of abstraction over thehardware, so we need another model, i.e. a contract between the hardware and the C++11programmer • Ensurethatmultithreadedprogramsareportable:theywill runcorrectlyondifferenthardware • Clarifywhichoptimizationswillorwillnotbreakourcode • Weneedtheserules,forexample.tounderstandwhenwe canhaveadatarace,sowecanknowwhenourprogramis correct, and that it will run correctly by all compliant C++11compilers • For example, wemight ask: • “If X=Y=1, is itpossiblefor the • outcomeofthisprogramtober1=r2=1?” Scott B. Baden / CSE 160 /Wi '16

15. Preliminaries • TheC++11memorymodeldescribesanabstractrelation betweenthreadsandmemory • Providesguaranteesabouttheinteractionbetweeninstruction sequences and variablesin memory • Everyvariableoccupies1memorylocation • Bitfieldsandarraysaredifferent;don’tloadallofc[]asa32bitword • Awritetoonelocationcan’taffectwritestoadjacentones • struct s { charc[4]; • int i:3,j:4; • struct in { doubled; • }id; • }; Scott B. Baden / CSE 160 /Wi '16

16. Whydon’twewanttoloadallofthec[]array as oneword? A. Because each element is consideredan “variable” Becauseanotherthreadcouldbewritingasingle element Becauseanotherthreadcouldbereadingasingle element A andB B andC struct s { charc[4]; int i:3,j:4; struct in{ doubled; }id; }; 18 Scott B. Baden / CSE 160 /Wi '16

17. Whydon’twewanttoloadallofthec[]array as oneword? A. Because each element is consideredan “variable” Becauseanotherthreadcouldbewritingasingle element Becauseanotherthreadcouldbereadingasingle element A andB B andC struct s { charc[4]; int i:3,j:4; struct in{ doubled; }id; }; 18 Scott B. Baden / CSE 160 /Wi '16

18. Communication • Memorywritesmadebyonethreadcanbecomevisible,but…. • …specialmechanismsareneededtoguaranteethatcommunication happens betweenthreads • Withoutexplicitcommunication,youcan’tguaranteewhichwritesgetseen byotherthreads,oreventheorderinwhichtheywillbeseen • TheC++atomicvariable(andtheJavavolatilemodifier)constitutesa specialmechanismtoguaranteethatcommunicationhappensbetween threads • Whenonethreadwritestoasynchronizationvariable(e.g.anatomic)and anotherthreadseesthatwrite,thefirstthreadistellingthesecondaboutall ofthecontentsofmemoryupuntilitperformedthewritetothatvariable • Ready is a synchronization variable • In C++ we use load andstore All the memory contents seen by T1, before it wrote toready, must be visible toT2, after it reads the value true forready. http://jeremymanson.blogspot.com/2008/11/what-volatile-means-in-java.html memberfunctions Scott B. Baden / CSE 160 /Wi '16

19. The effects ofsynchronization • Synchronizationcanbecharacterizedintermsof 3properties • Atomicity, Visibility,Ordering • All changes made in one synchronized variable or code block are atomicandvisiblewithrespecttoothersynchronizedvariablesand blocksemployingthesamelock,andprocessingofsynchronized • methodsorblockswithinanygiventhreadisinprogram-specified order • Outoforderprocessingcannotmattertootherthreadsemploying synchronization • Whensynchronizationisnotusedorisusedinconsistently,answers are morecomplex • Imposesadditionalobligationsonprogrammersattemptingtoensure objectconsistencyrelationsthatlieattheheartofexclusion • Objectsmustmaintaininvariantsasseenbyallthreadsthatrelyon them,notjustbythethreadperforminganygivenstatemodification Scott B. Baden / CSE 160 /Wi '16

20. The 3Properties • Ofmostconcernwhenvaluesmustbetransferredbetween mainmemoryandper-threadmemory • Atomicity.Whichinstructionsmusthaveindivisibleeffects? • Visibility.Underwhatconditionsaretheeffectsofonethread visible to another? The effects of interest are: writes to variables,asseenvia readsofthose variables • Ordering.Underwhatconditionscantheeffectsofoperations appear out of order to any given thread? In particular, reads and writes associated with sequences of assignment statements. Scott B. Baden / CSE 160 /Wi '16

21. Whatkindsofvariablesrequireatomicupdates? A. Instance variables and staticvariables Array elements. Depends on the accesspattern Local variables insidemethods A &B B &C Scott B. Baden / CSE 160 /Wi '16

22. Whatkindsofvariablesrequireatomicupdates? A. Instance variables and staticvariables Array elements. Depends on the accesspattern Local variables insidemethods A &B B &C Scott B. Baden / CSE 160 /Wi '16

23. Dataraces • Wesaythataprogramallowsadataraceonaparticularsetofinputs • ifthereis asequentiallyconsistentexecution,i.e.aninterleavingof operations of the individual threads, in which two conflicting operationscanbeexecuted“simultaneously”(Boehm) • Wesaythatoperationscanbeexecuted“simultaneously”iftheyoccur next to each other in the interleaving, and correspond to different threads • Wecanguaranteesequentialconsistencyonlywhentheprogram avoids dataraces • Considerthisprogram,withx== y== 0initially Scott B. Baden / CSE 160 /Wi '16

24. Doesthisprogramhaveadatarace? A. Yes B. No x = = y = = 0initially Scott B. Baden / CSE 160 /Wi '16

25. Doesthisprogramhaveadatarace? A. Yes B. No x = = y = = 0initially Scott B. Baden / CSE 160 /Wi '16

26. Dataraces • Wesaythataprogramallowsadataraceonaparticularsetofinputs • ifthereis asequentiallyconsistentexecution,i.e.aninterleavingof operations of the individual threads, in which two conflicting operationscanbeexecuted“simultaneously”(Boehm) • Wesaythatoperationscanbeexecuted“simultaneously”iftheyoccur next to each other in the interleaving, and correspond to different threads • Wecanguaranteesequentialconsistencyonlywhentheprogram avoids dataraces • Thisprogramhasadatarace(x==y= =0initially) Execution x =1; r1 = y; y =1; r2 = x; // r1 = 1 ∧r2==1 Scott B. Baden / CSE 160 /Wi '16

27. “Happens-before” • Fundamentalconceptinunderstandingthememorymodel • Consider these2threads,withcounter=0 • A:counter++; • B: prints outcounter • EvenifBexecutesafterA,wecannotguaranteethatBwill see1 … • … unless weestablisha happens-beforerelationship • betweenthesetwostatementsrunningindifferentthreads • Whatguaranteeismadebyahappens-beforerelationship? A guarantee that memory writes by one specific statement are visible to another specificstatement • Differentwaysofaccomplishingthis:synchronization, atomics,variables,threadcreationandcompletion,e.g. tA.join(); tB.join(); thread tA =thread(A); thread tB =thread(B); Scott B. Baden / CSE 160 /Wi '16

28. Establishing a happens-beforerelationship • C++ and Java provide synchronization variables to communicate between threads, and are intended to be accessed concurrently: the atomic types, mutexes • Such concurrent accesses are not considered dataraces • Thus, sequential consistency is guaranteed so long as the only conflicting concurrent accesses are to synchronizationvariables • Any write to a synchronization variable establishes a happens-before relationship withsubsequentreadsofthatsamevariable: x_ready=true happens-before the read of x_ready in Thread 2. • A statement sequenced before another happens-before it x=42happens-before x_ready-true • Happens-before is transitive: everything sequenced before a write to synchronizationvariable also happens-before the read ofthatsynchronization variablebyanotherthread. Thus, assignment x=42 (T1) is visibleafterthe read of x_ready by Thread2,e.g the assignment tor1 Scott B. Baden / CSE 160 /Wi '16

29. Doesthisprogramhavea race condition? A. Yes B. No Scott B. Baden / CSE 160 /Wi '16

30. Doesthisprogramhavea race condition? A. Yes B. No Scott B. Baden / CSE 160 /Wi '16