Distributed Monitoring in P2P Systems Serge Abiteboul , Bogdan Marinoiu INRIA-Futurs and Univ. Paris 11

1. Distributed Monitoring in P2P SystemsSerge Abiteboul, Bogdan MarinoiuINRIA-Futurs and Univ. Paris 11 DocFlow - kick off - 2007 - Monitoring

2. Organization • Motivation • An algebra over XML streams • The P2PMonitor system • Optimization • Conclusion DocFlow - kick off - 2007 - Monitoring

3. Motivation

4. P2P systems • Peer-to-peer: A large and varying number of computers cooperate to solve some particular task without any centralized authority • seti@home; kazaa; cabal • Interactivity between the peers (Web 2.0) • Content sharing communities • Distributed applications (e.g., games) • New services DocFlow - kick off - 2007 - Monitoring

5. P2P systems • Benefit from technologies like XML and Web services • Complex and hard to control systems • Many peers • Peers are distributed • Peers are autonomous • Peers are often unreliable and selfish • Peers come and leave • Intense communications between peers • Content changes rapidly • Goal: monitor such systems DocFlow - kick off - 2007 - Monitoring

6. Why monitor P2P systems? • P2P Monitoring, e.g., supply chain or manufacturing system • Web Intelligence: e.g., business intelligence and surveillance of competing companies • P2P Security: e.g., against intrusion, spamming • P2P optimization and tuning: e.g., gather statistics, control indexing • P2P system administration: e.g., choose where to log or control replication • P2P error management: e.g., error detection, error, diagnosis • Business: e.g., billing for using resources, procurement DocFlow - kick off - 2007 - Monitoring

7. An algebra over XML streams

8. XML stream Detect events at the peer level Data changes Database updates Events in RSS feed Web page changes Web service calls Out call In call Each event is represented as an XML document XML Stream Monitoring one peer - Alerters DocFlow - kick off - 2007 - Monitoring

9. An algebra over streams  = outer join &  = s=“Lhasa” • Efficient Filter (can provide many selections simultaneously) • Binary operators: union, join, etc. • Operator with memory: duplicate elimination • Same plans meaningful for query processing or monitoring       s2 s3 s4 s1 s5 DocFlow - kick off - 2007 - Monitoring

10. ActiveXML algebra in short • Used both for query and monitoring plans • Local computation - Algebra for queries over trees: • Pick your favorite • Has to work on streams • Manage replication • Generic data and services • Glue • Send, receive, eval • Essential • Include service calls – typically query calls • Exchange streams of ActiveXML Algebraic Expressions DocFlow - kick off - 2007 - Monitoring

11. AXML stream operator • AXML stream Q a r1@p1 r Music@p2 Music@p3 r1 r1 r1 “Easy case” Monotone in-stream Monotone query Stream of ActiveXML s t s t s t Peer p1 DocFlow - kick off - 2007 - Monitoring

12. P2P Monitor system

13. Stream specification language • for $c on local: outCOM • let $timeCall := $c.call.time • and $duration := $c.response.time - $timeCall • where • $c.call.method = "GetTemp“and $duration > 300 • and $c.call.site = "http://meteofrance.fr" • return <longGetTemp> • <callTime>{$timeCall}</callTime> • <duration>{$duration}</duration> • </longGetTemp> DocFlow - kick off - 2007 - Monitoring

14. Compiled into a monitoring plan • Compile the specification into a monitoring plan • Some alerters produce input streams • Stream operators • Evaluate it • The monitoring plan is distributed DocFlow - kick off - 2007 - Monitoring

15. What to do with the result streams • Publish a stream • RSS, email, web site • Turn a stream into a channel • Users can subscribe to it • Store stream – distributed tracing • Provide traces of the computation • Basis for querying a run • Basis for analyzing a run: diagnosis DocFlow - kick off - 2007 - Monitoring

16. Example: monitoring involving many peers • For $com1 on http://peer.a.com :outCOM | http://peer.b.com :outCOM, • $com2 on http://meteo.fr : inCOM • Let $duration := $com1.response.timestamp - $com1.call.timestamp • Where • $duration > 10 and • $com1.call.method = « getTemp »  and Filter • $com1.call.site = http://meteo.fr and • $com1.call.id = $com2.call.id Join • Return Report($com1, $com2) • By email to service@meteo.fr; A M B DocFlow - kick off - 2007 - Monitoring

17. Monitoring Task Global Plan alerter@A alerter@B Filter Filter alerter@M ∪ Join Reporter Publisher DocFlow - kick off - 2007 - Monitoring

18. Distributed Monitoring Task Peer A: For $c on local:outCOM Let $duration = … Where . Return $c By publish as channel X alerter@A alerter@B Filter@A X ∪@B Publisher@A Filter@B Peer M: For $c1 on local:inCOM , $c2 on Y Where $c1.call.id = $c2.call.id Return Report($c1, $c2) By email to service@meteo.fr alerter@M Publisher@B Y Peer B: For $c on local:outCOM | X Let $duration = … Where …. Return $c By publish as channel Y Join@M Reporter@M Publisher@M DocFlow - kick off - 2007 - Monitoring

19. Optimization

20. Optimize the plan • Distribute the work among the peers • Try to place computation close to data if possible • Try to reduce redundancy DocFlow - kick off - 2007 - Monitoring

21.  = outer join &  = s=“Lhasa” Optimization based on ActiveXML algebra       monitoring plan (a) r2[s,at] r3[t,s] r4[t,s] r1[t,s] r5[t,s]         r1[t,s] r3[t,s] r4[t,s] r5[t,s] monitoring plan (b)         r5[t,s] r1[t,s] r3[t,s] r4[t,s] monitoring plan (c) DocFlow - kick off - 2007 - Monitoring

22. Conclusion

23. The plan Specify monitoring Generate data streams Store traces Query traces Mine traces (e.g., diagnosis) streams DocFlow - kick off - 2007 - Monitoring

24. Merci Merci DocFlow - kick off - 2007 - Monitoring