200 likes | 310 Views
Efficient Decentralized Monitoring of Safety in Distributed Systems. Koushik Sen Abhay Vardhan Gul Agha Grigore Rosu. University of Illinois at Urbana-Champaign, USA. Software Reliability. Software Validation Rigorous and Complete Methods Model Checking Theorem Proving
E N D
Efficient Decentralized Monitoring of Safety in Distributed Systems Koushik Sen Abhay Vardhan Gul Agha Grigore Rosu University of Illinois at Urbana-Champaign, USA
Software Reliability • Software Validation • Rigorous and Complete Methods • Model Checking • Theorem Proving • Infeasible for large-scale open distributed systems • Non-determinism and Asynchrony • Testing • Widely used • Ad-Hoc • Good Test Coverage Required • Runtime Monitoring • Adds rigor to Testing
Centralized Monitoring Approach • Monitoring – Use Formal Methods in Testing • Synthesize light-weight Monitors from Specification • Automata, Rewriting-based Monitors, State machines • Instrument code to insert monitors • Execute instrumented code • Distributed System Monitoring • Global state is distributed • For every state update send state to a central monitor • Central monitor assembles them to form consistent execution traces (Vector Clocks) • Sequence of global states • Monitor execution traces
An Example • Mobile node a requests certain value from node b • b computes the value and sends it to a • Property: no node receives a value from another node to which it had not sent a request
Centralized Monitoring Example “If a receives a value from b then b calculated the value after receiving request from a” valRcv → (valComputed valReq) valReq valReq valComputed valReq valRcv → (valComputed valReq) (valComputed valReq) Monitor b valComputed a valReq valRcv
Decentralized Monitoring Approach “If a receives a value from b then b calculated the value after receiving request from a” valRcv → @b((valComputed @a(valReq))) valComputed @a(valReq) @a(valReq) (valComputed @a(valReq)) b valComputed a valReq valRcv valReq valRcv → @b((valComputed @a(valReq)))
Past time Distributed Temporal Logic (pt-DTL) • Past Time Linear Temporal Logic [Pnueli] • Extended with a Operator from epistemic logic (@) • [Aumann76][Meenakshi et al. 00] • Properties with respect to a process, say p • Interpreted over sequence of knowledge that p has about global state
Remote Formulas in pt-DTL • @a F at process b • @ makes remote formula F at process a local to process b “Alarm at process b implies that there was a fire at a” alarm → @afire a formula with respect to process b
Remote Expressions in pt-DTL • Remote expressions – arbitrary expressions related to the state of a remote process • Propositions constructed from remote and local expressions “If my alarm is set then eventually in past difference between my temperature and temperature at process b exceeded the allowed value” alarm → ((myTemp - @btemp) > allowed)
Safety in Airplane Landing “ If my airplane is landing then the runway that the airport has allocated matches the one that I am planning to use” landing → (runway = @airportallocRunway)
Leader Election Example “If a leader is elected then if the current process is a leader then, at its knowledge, none of the other processes is a leader” elected → (state=leader → /\i≠j(@j(state ≠ leader)))
pt-DTL syntax • Fi ::= true | false | P(Ei) | : Fi | FiÆ Fi propositional | ¯ Fi | ¡ Fi | Fi | Fi S Fi temporal | @jFj epistemic • Ei ::= c | vi2 Vi | f(Ei) functional | @jEj epistemic
Interpretation of @jEj at process i p3 m4 m1 m2 p2 @ 1(x=9) m3 p1 x=7 x=9
Monitoring Algorithm • Requirements • Should be fast so that online monitoring is possible • Little memory overhead • Additional messages sent should be minimal; ideally zero
KnowledgeVector Let KV be a vector • one entry for each process appearing in formula • KV[j] denotes entry for process j • KV[j].seq is the sequence number of last event seen at process j • KV[j].values stores values of j-expressions and j-formulae
Monitoring using KnowledgeVector • Maintain KnowledgeVector about global state at each process • Attach KnowledgeVector with outgoing messages • Update KnowledgeVector with incoming messages • At each process monitor local KnowledgeVector
KnowledgeVector Algorithm • [internal event]: (at process i) store eval(Ei,si) and eval(Fi,si) for each @iEi and @iFi in KVi[i].values • [send m]: KVi[i].seq à KVi[i].seq + 1. Send KVi with m as KVm • [receive m]: for all j, if KVm[j].seq > KVi[j].seq then • KVi[j].seq à KVm[j].seq • KVi[j].values à KVm[j].value • store eval(Ei,si) and eval(Fi,si) for each @iEi and @iFi in KVi[i].values
0 2 2 2 2 0 2 0 1 1 2 0 6 9 6 6 6 6 5 6 5 6 5 9 Example p3 p2 Y=7 Y=3 violation p1 X=5 X=9 X=6 KV[1].seq ¡(Y¸ @1X) at p2 KV[1].values
DIANA Architecture pt-DTL Monitor
Conclusion • pt-DTL can express interesting and useful safety properties of distributed systems • Decentralized Technique to effectively verify Distributed Systems at runtime • No extra message over-head for monitoring • KnowledgeVector as monitors