1 / 28

Dynamic Web Displays for Online Publishing

Implementing dynamic web displays to publish real-time online status information globally, featuring technologies like Comet to optimize data distribution and enhance user experience.

arlenel
Download Presentation

Dynamic Web Displays for Online Publishing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Online Displays for the World Publishing online status information to the web Functional requirements Implementation requirements Chosen technologies The publishing mechanism Gallery Problems JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb

  2. Functional Requirements • Publish data from PVSS to web pages • Allow users to peek on the experiment status from anywhere • Monitor subdetectors from home institutes without the need to login • Experiment status (DAQ) • Operations status • Subdetector status • … • Applications are all “nice to have”, but not critical • Necessary: Extract data from PVSS or any DIM service JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 2

  3. Implementation Requirements • Dynamic web pages • Online updating displays • No full page reloads (ugly ) • Was something new (hype factor ) • Architecture for the lazy developer • Minimize developments (minimize maintenance) • Limit developments to • Data extractor from PVSS or other DIM services • Web pages  Use comet (-like) technology • http://cometdaily.com/maturity.html (a bit old: 2009) JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 3

  4. Comet • Technology to feed data to dynamic web pages • Data fan-out: one producer, many listeners • Message queuing technology • Topic oriented subscription mechanism • Publishing triggered by first subscription • Data provided to web browsers by Javascript / HTML socket • The technology selection is determined by the provided • Client APIs • Provided protocols • Required API (main limits over technology choice)  C/C++ API at the data producer side (DIM/PVSS)  XmlHttpRequest objects at the client side (XHR/long poll) WebSocket not (yet) supported on all browsers JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 4

  5. Data Format • All data are strings • STOMP or Openwire will internally encode the data • base64 encoding: blobs etc.  become strings • Internal format (custom implementation) • timestamp # tag name # PVSS data-type id # value • timestamp currently unused • tag name used to dispatch at client side • PVSS data type used for display optimization (sprintf) • FSM special data type id • Float numbers limited to 4 significant digits Quickly changing numbers become more stable JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 5

  6. Comet • Server side architecture • Client architecture • Maximize the content of the black box JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 6

  7. Server Architecture DIM Producer Stomp DIM server(s) MorbitQ Stomp/HTTP “Out of the box” JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 7

  8. More Performing Architecture DIM Producer OpenWireStomp DIM server(s) Stomp Stomp/HTTP Out of the box. Needs config JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 8

  9. Web Publishing Mechanism • Any PVSS Data can be published: PVSS00dim • A small CTL scriptsadds PVSS data type before publish • FSM Nodes: • Special multiple item data type to support for more fancy rendering • Datapoints: • All Run Control Data • LHC data, etc. • Panel + Server: Clara JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 9

  10. WEB Client Architecture • No “HTML” pages • All pages are dynamic • HTML structure is programmed by directly manipulating the DOM tree in Javascript • My choice: all pages have the identical source • Instead of <TABLE> … </TABLE>:var table = document.createElement(‘table’);…. setup table ….parent.appendChild(table); • Not difficult, but different paradigm • Very flexible, rather complete and easy to use widget library JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 10

  11. WEB Client: First Value Problem • Comet only sends updates • No way to enforce sending first value at subscription time of a new client • Or me too stupid ? • Chosen solution • Client sends to special topic subscription request • Producer forces update of the newly subscribed topic • Disadvantage: All clients see the update Unnecessary load JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 11

  12. Published Pages • Web Pages Available • http://lbcomet.cern.ch/Online • Let’s walk through the galery • Page 1 • Run Control • Experiment Control • LHC/LHCb status • Fill Status • L0 Trigger status • Velo Motion display • Event Display • Browser based desktop JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 12

  13. Page1 JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 13

  14. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 14

  15. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 15

  16. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 16

  17. ORACLE  PVSS  WWW JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 17

  18. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 18

  19. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 19

  20. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 20

  21. Browser based desktop JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 21

  22. Monitoring Issues • Everything can fail • Monitoring the main components helps to investigate problems • AMQ • Comes with a rather comprehensive monitoring utility • Connection monitoring • Topic summary • Orbited • Rather rudimentary JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 22

  23. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 23

  24. JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 24

  25. Trouble and Problems • Normally things work pretty reliable • Limits seen with Orbited+MorbitQ at LHC startup • Too many clients • Better performance after move to ActiveMQ…but we have not seen these peaks anymore • Low number of hundreds of clients • Typically ~50 clients • Now during shutdown ~10 • No further optimization necessary so far JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 25

  26. Trouble and Problems • Web browsers going “ballistic” • About once/week during running • Less frequent during shutdown (request – related ?) • Single browser sends connection requests with tens of Hz • AMQ closes unused file descriptors after some timeout • Number of open file descriptors explode (>1000) • We investigated a lot (connection limits/client node etc.)but no real solution found  Open FD monitored by watch dogAutomated kill and restart if this happens JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 26

  27. Trouble and Problems • Never believe java/JavaScript has no memory leaks • This was a lie for Java and with web browsers and JavaScript it’s no better • References kept “somewhere” (i.e. DOM document) without remembered by user manifest the same way • Firefox exploded after some time • But happens on all pages being reloaded • Other browsers • IE: Leaks fixed with page reload every few hours • IE: No support for multiple connections from single through html iframes (dynamic popup windows) • Chrome: Very strict security model. But others catch up… JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 27

  28. Summary • Overall experience very good • Subdetector experts use the web pages quite a lot • Software maintenance overhead manageable • Binding to PVSS via DIM very simple • Performance sufficient for 100-200 clients • Mad browsers are an annoyance JCOP meeting 6-Dec-2011 M.Frank CERN/LHCb 28

More Related