mmnet Summer School

1. mmnet Summer School • Tuesday 20 – Wednesday 21July, 2004, Canterbury • Speakers: • David Bacon, IBM TJ Watson. • Emery Berger, UMass. • Robert Berry, IBM Hursley. • Hans Boehm, HP. • Dave Detlefs, Sun Microsystems. • Rick Hudson, Intel. • Eliot Moss, UMass. UKC, February 2004

2. Birrell’s Reference Listing Revisited Richard Jones University of Kent R.E.Jones@ukc.ac.uk Luc Moreau University of Southampton Peter Dickman University of Glasgow

3. Outline • Distributed reference counting – benefits & issues. • Birrell’s algorithm – example. • Weaknesses of Birrell’s description. • Our approach • Graphical notation • Formalisation & Proof • Extensions • Fault tolerance • Optimisation • Conclusion. UKC, February 2004

4. Problems of a distributed world • Concurrency everywhere • must avoid race conditions, etc • Communication is costly • changing the reference count of a remote object may cost 10,000 times as much as changing the count of a local object • Not easy to get complete knowledge of object graph • synchronisation is expensive • Faults everywhere • communications, processes UKC, February 2004

5. Terminology • Processes: partition computational and storage resources. • Messages pass in point-to-point channels between processes. • Channels have properties, such as FIFO or lossy. • A reference is local if it refers to an object allocated in the same process; alternatively, it is remote (or global). • The owner of a reference is the process that initially allocated the object to which the reference refers. UKC, February 2004

6. Distributed Reference Counting/Listing • Most widely used DGC technique • Maintain a count of remote references to each global object • Reference listing alternative • Benefits • Scalable solution • Easy to implement • But… • Cannot reclaim garbage cycles • Easy to implement wrong! UKC, February 2004

7. Birrell’s algorithm • Birrell, Evers, Nelson, Owicki, and Wobber. Distributed Garbage Collection for Network Objects. DEC SRC technical report 116, 1993. • Widely used: Modula-3 Network Objects; Java RMI. • Based on reference listing, avoids race conditions of naïve implementations, fault tolerance. UKC, February 2004

8. WireRep: unique ID of owner, plus index of object at the owner. • Marshalling • Object table: maps a wirerep w(o) to the local instance of the object. • Client has surrogate for o concrete o in object table • Dirty set: identifiers of processes that have surrogates. • Dirty-set =   o can be removed from the object table. • Concrete and surrogate objects. • Client invokes the surrogate, whose methods perform RPC to owner. Birrell’s description Process P: owner of O Process Q: a client of O weak ref UKC, February 2004

9. P marshalls o to Q P • P pushes o onto its stack;sends w(o) to Q. • Q looks it up in its object table. Present: use the object w(o)=NIL: surrogate being created; suspend. • Absent: enter w(o)=NIL in object table; send dirty(o) to owner(o); • Owner adds Q to its dirtySet(o) and dirty(o) returns. • Q creates surrogate(o) and adds it to its object table. • Q deletes surrogate(o) and sends clean(o) to owner(o). O Q UKC, February 2004

10. copy ack dirty Dirty calls A B Logkeep ref on stackcopydirtyackremove from stack {A} {A,B} UKC, February 2004

11. Weaknesses • Tightly bound to RPC • Acknowledgement mechanism. • Implementation specific. • Assumes method invocation pushes arguments onto stack; • Unique surrogate per process (object-listing) • Under-specified • Critical sections • Race conditions • Other scenarios • Informal proof • Depends on hard-to-formalise aspects (e.g. stack) UKC, February 2004

12. Our contribution • Novel graphical notation. • Formalisation. • Discovered requirement for pivotal new states. • Proof. UKC, February 2004

13. New graphical notation • Intuitive. • Precise. • Uniformity of ‘direction’ of transitions. • ‘Obvious’ where transitions are needed. UKC, February 2004

14. Lifecycle of references • Obvious where transitions are needed • E.g. Receive reference at state ccit. • ccitnil critical for correctness. UKC, February 2004

15. Slicing UKC, February 2004

16. Fault tolerance • Slicing • Owner is aware we have a reference. • Owner is not aware we have a reference. UKC, February 2004

17. Benefits • Intuitive – fault-tolerant version literally encapsulates failure-free version. • Identify precisely when failures can be detected. • Define states reached after failures detected. • Remedial actions. UKC, February 2004

18. Formalisation • Abstract machine • Processes communicating by asynchronous message passing. • Atomic transitions involve 1 process at a time. • Receipt of message changes only a process’ internal state • Trigger sending of a another message? • Store some info in a to do table? UKC, February 2004

19. Benefits • Inputs and outputs desynchronised. • Size of critical sections explicit and minimised. • Asynchronous outputs (e.g. background daemon processes to do tables). • Suitable for mechanical proof. UKC, February 2004

20. Formalisation guard • Rule name: guard  pseudo-statements. • make_copy (p1,p2,r): • p1p2receive_T(p1,r)=OK locallyReachable(p1,r) •  • { • id := new Identifier; • dirty_T(p1,r) := dirty_T(p1,r) U (p1,p2,id); • post(p1, p2, copy(r,id)); • } name table message UKC, February 2004

21. More formally • Tables defined as functions whose first argument is a process. • Channels are bags of messages between pairs of processes. • A configuration of the abstract machine is a tuple of all tables and message channels. • Pseudo-statements act as configuration transformers: • Given a configuration <…,table_T,…, k>, • table_T(a0,…an):=V denotes <…,table_T',…,k> wheretable_T'(x0,…xn) = table_T(a0,…an) if (x0,…xn) (a0,…an)table_T'(a0,…an) = V • post(p1,p2,m)denotes<…,table_T,…,k'> wherek'(p1,p2) = k(p1,p2)  {m}k'(pi,pj) = k(pi,pj), (pi,pj)  (p1,p2) UKC, February 2004

22. Proof style • Safety & Liveness • Invariance-based proof • Induction on length of transitions. • Case analysis of transitions. • Termination measure. • Benefits • Systematic. • Less error prone than temporal reasoning. • E.g. establishing fine details such as mutual exclusivity complicated in a formalism based on temporal reasoning. UKC, February 2004

23. Example proof • Lemma: For any processes p1, p2, for any reference r, for any identifier id and for any configuration, the following implication holds: • If <p1,p2,id>  dirty_T(p1,r) then receive_T (p1,r) = OK • Proof: In the initial configuration, dirty tables are empty and the implication trivially holds. We consider the four rules that add/remove entries to/from dirty tables and that modify the content of receive tables to/from OK. • make_copy (p1, p2,r): make_copy adds an entry <p1,p2,id>, and its guard ensures that the receive-table is in the OK state. • … UKC, February 2004

24. permanent temporary Key Lemmas Safety Lemma 1: Usable Reference For any processes p1 and p2, for any reference r with p1=owner(r) and p1p2, and for any configuration, the following implication holds: If receive_T(p1,r)=OK, then p1  dirty_T(p2,r). Safety Lemma 2: Reference in Transit For any processes p1, p2, for any reference r, for any identifier id and for any configuration, the following implication holds: If copy(r,id)  k(p1,p2), then p1 dirty_T(owner(r),r), if p1  owner(r)or <owner(r),p2,id>  dirty_T(owner(r),r), if p1 = owner(t) Safety Lemma 3: Unusable Reference For any process p1, for any reference r and for any configuration, the following implication holds: If receive_T(p1, r)=nil receive_T(p1, r)=ccitnil, then there exists p such that p dirty_T(owner(r), r)or there exist p,id such that <owner(r),p,id>  dirty_T(owner(r), r). UKC, February 2004

25. Birrell’s algorithm is Safe • A DGC algorithm is safe if the collector cannot reclaim live objects. For Birrell's algorithm, there must be an entry in the owner's dirty table for every live object. • The proof follows directly from the 3 safety lemmas. Birrell's Safety Requirement For all references r, and for all processes p1 and p2 and all identifiers id, If receive t(p1,r)=OKreceive_T(p1,r)=nil receive_T(p1,r)=ccitnil  copy (r,id) k(p1,p2), then there exists p such that pdirty_T(owner(r),r)or there exist p,id such that <owner(r),p,id> dirty_T(owner(r),r). UKC, February 2004

26. Liveness • Liveness guarantees that if all references to an object are deleted, the owner’s dirty table will eventually become empty. • To prove this, • We show that whenever there’s a message in a channel, a transition can be fired to consume it. • We introduce a termination measure on the configurations that shows how far the abstract machine is from completing, and show that DGC transitions cause this measure to decrease. • Hence all transition paths terminate. UKC, February 2004

27. Termination measures size of tables • termination_measure(c) = tab_measure +   msg_measure(m) +  rt_measure(receive_T(p,r)) • tab_measure = 9|dirty_call_todo_T| + 7|dirty_ack_todo_T| + 2|copy_ack_todo_T| + 2|clean_ack_todo_T| + 2|blocked_T| • and messages between pairs of processes states of references in processes msg_measure(copy) = 14msg_measure(dirty) = 8msg_measure(dirtyack) = 6msg_measure(clean) = 3msg_measure(copyack) =1msg_measure(cleanack)= 1 rt_measure(OK) =5 rt_measure(ccitnil) =2rt_measure(ccit) = 1rt_measure(nil) = 1rt_measure() = 0 values chosen‘arbitrarily’ UKC, February 2004

28. Example: receive_dirty_ack Dmeasure= -1 • receive_dirty_ack (p1, p2,r) :dirtyack(r)  k (p1, p2){ receive(p1,p2,dirtyack(r));//-6 copyack_todo_T(p2) := copyack_todo_T(p2) blocked_T(p2,r); //-X// Deserialisation code to be resumed for each entry in blocked_T(p2,r) blocked_T(p2,r) :=  ; //-X receive_T(p2,r) := OK; //+5} • Thus, termination measure decreases by 1. UKC, February 2004

29. Optimisations • FIFO channels • Less synchronisation needed • Fewer messages: no clean_ack • Fewer tables. • Sender is owner • No need for dirty_call and copy_ack • But need message ordering to avoid races • Receiver is owner • Fewer dirty table entries • Again need message ordering UKC, February 2004

30. Future work • Convince ourselves of appropriateness of Birrell’s remedial actions. • Correctness proof of fault-free version. • Explore applicability of our techniques • Graphical notation • Proof-techniques • Generality • Auto-generation of code from formalism. UKC, February 2004

31. Conclusion • Intuitive graphical notation. • Formal, implementation-independent specification and proof of a widely used algorithm. • Discovered weaknesses in original presentation. • A widely applicable technique? UKC, February 2004

32. Questions?

33. FINIS UKC, February 2004