Live Re-orderable Accordion Drawing (LiveRAC)

1. Live Re-orderable Accordion Drawing (LiveRAC) Peter McLachlan, Tamara Munzner Eleftherios Koutsofios, Stephen North AT&T Research Symposium August, 2007

2. Problem domain • Managed hosting services, network operations centre staff Motivation Problem Domain Monitored data Viz requirements SWIFT LiveRAC Key additions Field study Conclusion

3. Monitored data • Most data collected from monitored network devices is time-series data • any type of computer or appliance: servers, routers … • time stamp and value • Two types of time-series objects collected: • performance metrics • [ 10 AUG 2006 9:52:37, CPU, 95% ] • alarm data • [ 10 AUG 2006 9:52:37, MAJOR, “HIGH TEMP” ] • Key difference for visualization: • performance metrics quantitative • alarms categorical Motivation Problem Domain Monitored data Viz requirements SWIFT LiveRAC Key additions Field study Conclusion

4. Visualization solution requirements • Scale to large, dynamic datasets • thousands of devices • dozens of data channels • Interact with previously gathered data Motivation Problem Domain Monitored data Viz requirements SWIFT LiveRAC Key additions Field study Conclusion DB (SWIFT) LiveRAC

5. SWIFT • SWIFT is a set of data storage, aggregation and visualization tools that integrate multiple data sources [Koutsofios, 1999] • Developed at AT&T Labs, fully deployed in a production role • Data sources include SNMP, intrusion detection systems, Windows system monitors, and custom written daemons • Highly scalable • Optimized for streaming data Motivation SWIFT Introduction Visualizations LiveRAC Key additions Field study Conclusion

6. SWIFT visualizations • Existing views: • geographic views • node-link diagrams • line charts • text tables • Limitations: • difficult to compare between large numbers of time-series objects • LiveRAC: re-orderable matrix visualization for SWIFT Motivation SWIFT Introduction Visualizations LiveRAC Key additions Field study Conclusion Koutsofios, North, Truscott, and Keim. Visualizing large-scale telecommunication networks and services.IEEE Visualization 1999, 457-461

7. Our solution: LiveRAC Motivation SWIFT LiveRAC Demo Visual encoding Semantic zoom Key additions Field study Conclusion • Interactive user-directed exploration of overview & detail • live demo

8. LiveRAC visual encoding • High data density grid view • Rapidly explore time-series data • Context available at all times • Re-orderable matrix supports side by side data comparison Motivation SWIFT LiveRAC Demo Visual encoding Semantic zoom Key additions Field study Conclusion

9. Semantic zooming and aggregation • compact representations in reduced areas • large cells show time-series charts • aggregate spatial representation shown in highly compressed regions Motivation SWIFT LiveRAC Demo Visual encoding Semantic zoom Key additions Field study Conclusion

10. Project progress this year • Last year: hardwired prototype • This year: • robust, deployed cross-platform client • more scalable: • 4000+ devices tested • designed to scale past 10000 • new architecture allows full visual configuration interface • AT&T internal field study in progress Motivation SWIFT LiveRAC Key additions Progress callout Configuration New architecture Field study Conclusion

11. Configuration is critical in effective analytic tools • Exploratory data analysis requires looking at data in different ways • different tasks require different views • even the same task may require changes to the data representations • Provide end user full and flexible control through a GUI • not pre-compiled settings • Required complete data abstraction re-architecture Motivation SWIFT LiveRAC Key additions Progress callout Configuration New architecture Field study Conclusion

12. New architecture allows: • Data source groupings • by business, customer, logical group, geography … • Filter devices and channels of interest • pick ‘web’ devices and ‘cpu’ channels • Manage marker thresholds • at 85%, turn box red • Choose effective representations for data • use 2-line bar chart for 200x200 pixel cell • Select data aggregation strategies • aggregate to daily max when time window > 1 month Motivation SWIFT LiveRAC Key additions Progress callout Configuration New architecture Field study Conclusion

13. Field study in progress Motivation SWIFT LiveRAC Key additions Field study Conclusion • Deploying LiveRAC to AT&T internal users • tier 3 and tier 4 network professionals • more exploratory than front line tier 1 ops staff • Study objectives: • understand when and why users choose visualization in their workflow • specific benefits and limitations of LiveRAC in managed hosting environment • Gather data on LiveRAC usage • logs • structured interviews • questionnaires • Progress towards a system integrated into workflow of network operations analysts

14. Conclusion • Contributions: • robust system for interactive visualization of large scale network monitoring data • thousands of devices, hundreds of channels • months of data • highly configurable • deployed inside AT&T • field study in progress Motivation SWIFT LiveRAC Key additions Field study Conclusion

15. Information visualization • Human visual channel is highest-bandwidth perceptual system [Norretranders, 1999] • Information visualization: field of study whose object is to aid cognition through the graphic representation of abstract data • displays relevant information graphically to assist in memory tasks • supports data exploration through direct interaction • assists in pattern finding through the display of overview and detail, search, and user-directed reordering Motivation Related Work InfoVis Time-series Statistical Graphics Reorderable Vis Semantic Zoom Accordion Drawing SWIFT LiveRAC Discussion

16. Network devices • Network device: any electronic device that connects to a computer network • Most network devices can be monitored • Network Operation Centre (NOC): facility for monitoring large numbers of network devices Motivation Overview Problem Domain Devices & Data Domain Tools Requirements Our Solution Related Work LiveRAC Discussion

25. Data path Render path Motivation Related Work LiveRAC Architecture Semantic Zoom Reordering Playback Discussion