1 / 54

Itaipu A Business Activity Monitoring system designed for business users

Itaipu A Business Activity Monitoring system designed for business users. Prof. Jacob Slonim June 27, 2009 Student: Azza Abouzied, MCSc. Outline. Motivation Related Work Research Questions Our vision of BAM: Itaipu Multi-dimensional analysis using Stream Cubes Usability Future Work.

moanna
Download Presentation

Itaipu A Business Activity Monitoring system designed for business users

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ItaipuA Business Activity Monitoring system designed for business users Prof. Jacob Slonim June 27, 2009 Student: Azza Abouzied, MCSc

  2. Outline • Motivation • Related Work • Research Questions • Our vision of BAM: Itaipu • Multi-dimensional analysis using Stream Cubes • Usability • Future Work Largest hydroelectric dam in the world, provides 20% of Brazil’s and 94% of Paraguay’s power Motivation Related work Itaipu Stream Cubes Usability Future work

  3. Research Goal • Provide just-in-time analysis of business events • Business Intelligence (BI) • Business Objects • Initiated research • Different approaches Motivation Related work Itaipu Stream Cubes Usability Future work

  4. Motivation • Enterprise’s Global Responsiveness and Agility • Wal-Mart sales employees • Oil prices increase • Weather predictions • Canadian tire discounts garden products • Will we make the quarter? Motivation Related work Itaipu Stream Cubes Usability Future work

  5. Business Activity Monitoring (BAM) • Continuous queries over data streams • Continuous query • has implications on multiple query optimization • Infinite, unbounded, data streams • Single-pass processing, processing bounds, or approximation techniques required • Ad-hoc queries • Dynamic environment • Target business end-users NOT technical analysts Motivation Related work Itaipu Stream Cubes Usability Future work

  6. Related Work • Complex Event Processing Systems • Cayuga • Origin: Business Process Management • Business Objects Approach • Data Stream Management Systems • STREAM • Aurora, Borealis • TelegraphCQ • Origin: Monitoring sensor networks Motivation Related work Itaipu Stream Cubes Usability Future work

  7. Research Questions • Could we reuse DSMS for a BAM system? • Could we provide multi-dimensional analysis of data streams? • Could we simplify and expedite query write-up for business users? Motivation Related work Itaipu Stream Cubes Usability Future work

  8. Itaipu • Spring • Data Source • Dam • Query processing unit • Delta • Results display and storage Largest hydroelectric dam in the world, provides 20% of Brazil’s and 94% of Paraguay’s power Motivation Related work Itaipu Stream Cubes Usability Future work

  9. Itaipu’s big picture Motivation Related work Itaipu Stream Cubes Usability Future work

  10. Recap & What’s Next? • Brief design overview of Itaipu • Two more questions left: • Multidimensional data analysis of stream data • Enhancing user-experience or usability Motivation Related work Itaipu Stream Cubes Usability Future work

  11. Cube: a multi-dimensional data model • Dimensions • Region: (city < country < all) • Product: (sub-category < category < all) • Measures • Sum, count, average • Size Motivation Related work Itaipu Stream Cubes Usability Future work

  12. Stream Cubes • Han et. al introduced stream cubes • Minimally-interesting (m-layer) and observation layers (o-layer) • Partial Materialization • Popular drilling path Motivation Related work Itaipu Stream Cubes Usability Future work

  13. Stream Cubes • Tilted Time frame • Time dimension • Distant aggregates provided at larger granularities in comparison to more recent ones Conceptual representation of a tilted time frame Motivation Related work Itaipu Stream Cubes Usability Future work

  14. Difference in perspective • Popular drilling path • Users interested in o-layer and occasionally drill down • Path could be selected by an expert • Users drilling path is not likely to change • Dynamic materialization path • User requests are distributed at different abstraction layers • User requests are likely to change or evolve over the course of the cube’s lifetime in the system Motivation Related work Itaipu Stream Cubes Usability Future work

  15. Drivers for change • External factors • Oil prices increase • Export and oil-based products affected • Shift towards viewing more product information, less regional detail • Competition • Seasonal • Market or economic Motivation Related work Itaipu Stream Cubes Usability Future work

  16. Dynamic Materialization Path • Simple and efficient approach • Linear cost-effectiveness function • Dynamic programming approach • Cost-effectiveness function • Memory cost • Popularity effect • Update potential • Adapts to different operating environments • Weighted cost-effectiveness = - α(Memory cost) + β(Popularity) +γ(update potential) Motivation Related work Itaipu Stream Cubes Usability Future work

  17. Update potential? Motivation Related work Itaipu Stream Cubes Usability Future work

  18. Simulated example α = 0.5, β = 1.0 , γ = 0.75 Motivation Related work Itaipu Stream Cubes Usability Future work

  19. Resetting Path Motivation Related work Itaipu Stream Cubes Usability Future work

  20. Resetting paths on demand Motivation Related work Itaipu Stream Cubes Usability Future work

  21. Benefit of dynamic path materialization? • Flexible • Follows user requests • Low reliance on IT staff • Suitable for just-in-time operation • How does it affect data stream processing? • Queue Size measure • How does it affect memory consumption • Memory usage Motivation Related work Itaipu Stream Cubes Usability Future work

  22. Experimental Setup Data rate ≈ 12,000 Motivation Related work Itaipu Stream Cubes Usability Future work

  23. Experimental Setup Motivation Related work Itaipu Stream Cubes Usability Future work

  24. Results – Queue size Motivation Related work Itaipu Stream Cubes Usability Future work

  25. Results – Memory consumption Motivation Related work Itaipu Stream Cubes Usability Future work

  26. So, is it better? • Conclusion • Situation dependent: more flexible solution • Not suitable when fixed memory guarantees are required • Are their better dynamic path materialization algorithms? • Probably • Our aim: simple, efficient, something that works! • Future improvements? Motivation Related work Itaipu Stream Cubes Usability Future work

  27. Recap & What’s Next? • Brief design overview of Itaipu • Optimization work with stream cubes • Last question: • Enhancing user-experience or usability for Itaipu’s business end-users Motivation Related work Itaipu Stream Cubes Usability Future work

  28. BAM user experience • Bottleneck to the adoption of BAM • Operational decision making requires query enabling technology • The entire organization utilizes BAM: training effort needs to be minimal Motivation Related work Itaipu Stream Cubes Usability Future work

  29. A usability framework Jarke’s & Vassiliou’s model: • Training Effort • Repeated Efforts • Thinking • Self-construction • Assisted-construction • Input • Correction Motivation Related work Itaipu Stream Cubes Usability Future work

  30. Lessons Learned from database usability • The simpler the task, the less query formulation effort is required. • Corollary: Limiting the functionality of a query interface, makes it easier to use • Query by example • Query by forms • Collaboration reduces the training effort • C-TORI • SearchTogether Motivation Related work Itaipu Stream Cubes Usability Future work

  31. BAM’s user profile Redrawn from a table in Jarke et Vassiliou, 1985 Motivation Related work Itaipu Stream Cubes Usability Future work

  32. Our usability solution • Query Frames • Created by data analysts • Simplify query formulation to completing a query outline • Transformed easily to a form interface • Query search • Each query or query frame is associated with textual descriptions • Users and analysts could share queries and query frames • Passive collaboration • Keyword-based Google-like search interface Motivation Related work Itaipu Stream Cubes Usability Future work

  33. Query Frames Motivation Related work Itaipu Stream Cubes Usability Future work

  34. Dimension Query Frame Motivation Related work Itaipu Stream Cubes Usability Future work

  35. Dimension Query Frame Motivation Related work Itaipu Stream Cubes Usability Future work

  36. Itaipu’s Search Interface Motivation Related work Itaipu Stream Cubes Usability Future work

  37. Itaipu’s Search Interface Motivation Related work Itaipu Stream Cubes Usability Future work

  38. Analytical Evaluation Motivation Related work Itaipu Stream Cubes Usability Future work

  39. Final Recap Motivation Related work Itaipu Stream Cubes Usability Future work

  40. What’s in the future? • Further studies to validate some our hypotheses • A vision for BI Motivation Related work Itaipu Stream Cubes Usability Future work

  41. Conclusion • Business Objects was convinced with our solution: • Itaipu’s architecture • Stream Cube extension • Query frame and query search Motivation Related work Itaipu Stream Cubes Usability Future work

  42. Thank for the support of:NSERC, Business Objects, Business Intelligent Network

  43. Questions ?

  44. Spring • Framework, based on data sources, • define • Properties (allows drop) • Data rate • Push/pull • create: • Data Transformer • Data Streamer • Data specification Motivation Related work Itaipu Stream Cubes Usability Future work

  45. Dam: Execution Engine • Execution Engine • Variant of pipe & filter architecture • Similar to Fjords model • Works like an Operating system • Thread Manager • Queue Manager • Operator Builder Motivation Related work Itaipu Stream Cubes Usability Future work

  46. Execution Engine: Queue Manager Motivation Related work Itaipu Stream Cubes Usability Future work

  47. Execution Engine: Queue Manager Motivation Related work Itaipu Stream Cubes Usability Future work

  48. Execution Engine: Thread Manager Motivation Related work Itaipu Stream Cubes Usability Future work

  49. Execution Engine: Operator Builder • XML multiple-query plan • Decouple query planning/optimizing from execution engine • Java to XML Binding (JAXB) • Builds a content object model for each XML element representing an operator • Framework for building operators • Hierarchy of builders (Operator -> Aggregators -> SUM) • Update existing network of operators Motivation Related work Itaipu Stream Cubes Usability Future work

  50. Dam: Query Processing Unit • Query Language • SQL-like with windowing constructs • Industry standard • Current implementation • Extremely naïve • Open source tools Motivation Related work Itaipu Stream Cubes Usability Future work

More Related