Agenda

1. Iterative Waterfall Case Study: Network Information Data On-line AnalysisAlessandro Zoreralessandro.zorer@create-net.it

2. Agenda • Iterative Waterfall methodology (based on Sodalia SIMEP) • General approach • DWH ‘tailoring’ • Case Study: Network Information Data On-line Analysis • Needs • Approach • Focus on System Architecture • Functional View • Process View • Development View • Physical View • Fifth View • Summary • Q&A

3. Concept Exploration and Context Analysis Iterative Waterfall Methodology & Process • Iterative approach • Multilayer • Multiperspective System Development Strategy Methodology Iteration # 1 Iteration # 2 Iteration # n System Architecture. Component Design. Component Design. Component Design. Component Design. Component Design. Component Design. System Requirements System Test. Component Design. Component Development. Component Test.

4. Expressed as a UML activity diagram Use Case and Actor identification Use Case model Structuring Context Analyst Product impact Analysis Use Case Prioritization Integration Analysis Requirement Manager Data Warehouse design Architect DW Designer DW Analysis Interface Prototyping DW may be tailored to each specific project and domain Data Mart Designer Adaptation to DWH Methodology

5. Manage Dependencies Develop Business Model Elicit Stakeholder Needs Find Actors and Use Cases Structure the Use-Case Model Capture a Common Vocabulary Context Analyst Business Model Activity Diagrams Use-Case Model Requirement Attributes Stakeholder Identification Responsible for Glossary Context Analysis Methodology

6. Requirement Definition and System Architecture Analysis System Architecture Requirements Analysis Business Analysis Architectural Qualities System Architecture Design Methodology Tools Assessment & Evaluation • Use case identification • Data Sourcces identification • Data Consumers identification Capacity Performance Scalability UML DESMET Concept Exploration

7. Architectural Views • Logical View A logical abstract view of system elements and of the services to be provided to the end user • Process View Analysis the dynamic aspect of the system through scenarios and other diagrams (e.g., sequence, collaboration and activity diagrams). The elements focused are: tasks with their workflow, processes with their dependency, synchronization and concurrence aspects. • Development View Organization of the actual software modules in the software-development environment. The modules may be packaged in components or subsystems (component diagram) which may be organized in a hierarchy of layers, each layer providing a narrow well-defined interface to other layers. • Physical View Provide the deployment configurations in terms of Hardware and Software Components. This view shows the System Topology, a network of processing nodes with the software running on them. Capacity issues are addressed. • Fifth View Orthogonal view. Issues addressed are: Potential Software Reuse Analysis, Requirements allocation on Components, Performance Analysis, Functionality Categorization and Ranking.

8. DESMET Methodology for evaluating COTS based on: • Functional Qualities • Architectural qualities (i.e. adaptability, Scalability) • Performance analysis • Business aspects • Time to market • System lifecycle • Contractual constraints • Support organization Methodology

9. Physical Architecture Design Data Modeling DW Design Input Design Data Marts MD Schema Output Design Methodology Data Flow Design Metadata Management Design UML E-R System Architecture Design

10. Deployment Iteration DM Design Data Mart Construction Testing Training Methodology Customization Unit Testing Hardware Data Flow Database

11. Business Process Modeling Object / Component Modeling Logical / Physical Database Modeling Enterprise Integrated DW Modeling Support tools infrastructure Methodology

12. Case Study:Network Information Data On-line Analysis Business needs: • Definition and development of a DataWarehouse Framework for Multidimensional Analysis based on: • Call Data (Network Management) • Fault Data (Problem Management) • Performance Data (End-to-End Analysis) • Optimization of network performances through gathering and analysis • High integrability of new data sources • Optimization and extension of on-line analysis functionalities • Quick creation of reports and flexibility for the end user (through custom Data Marts) • Extension of capabilities in term of historical data management. Case Study Intro.

13. Solution • A specialized and adaptable Data Warehouse solution to support Network Traffic Management and Call Behavior Analysis through a smart data correlation among CDR and configuration, performance and trouble tickets • Highly scalable to adapt from small to large business needs • Based an a mix of COTS and developed components • Flexible to accomadate a variety of different sources and Call Data Record formats • Detailed data analysis capabilities to support different DSS customer organizations • Predefined “good example” analysis library to quikly develop and deliver QoS monitors and Service Level Management functions Case Study Intro.

14. System Framework Approach • Simplify the design, implementation, and management of data warehousing solutions • An open architecture that allows easy integration with and extended by third party vendors • Heterogeneous data import, export, validation and cleansing services with optional data lineage • Integrated metadata for warehouse design, data extraction/transformation, server management, and end-user analysis tools • Core management services for scheduling, storage management, performance monitoring, alerts/events, and notification Case Study Intro.

15. Logical View • UML Domain and System Modeling • describes system concepts in a formal way • drives data modeling • drives components design • drives dynamic modeling • Standard-based Object Information Model (OIM) from Microsoft and Metadata Coalition C.S.: Logical View

16. Layered Modeling Organization Data Analysis Layer C.S.: Logical View Data Warehouse Layer MetaData Management WorkFlow Management Data Transformation Layer

17. Generic Record-oriented Model Element SummaryInformation TransformableObject Column Classifier RecordItem C.S.: Logical View ModelElement Attribute 0..* 88Level +DeployedCatalogs Record GroupDef Group Field +Type RecordFormat DeployedRecord LogicalRecord DeployedGroup LogicalGroup DeployedField LogicalField

18. Generic Call Data Record Model DeployedRecord NEType C.S.: Logical View CallDataRecord ServiceType NetworkElement SourceID DestinationID Elapsed Measure

19. Generic OLAP Model Package Connection DataSource Catalog Store C.S.: Logical View +Data Sources OLAPDatabase Connection OLAP Server Cube +Cubes Dimension +Dimensions +DeployedCatalogs 0..* ModelElement DeployedOLAPDatabase LogicalOLAPDatabase +DimHierarchies 1..* DimHierarchy

20. OLAP and DSS • Fast • five seconds or less. • Analysis • Performs basic numerical and statistical analysis of the data, predefined or ad hoc • Shared • Implements the security requirements across a large user population • Multidimensional • Is the essential characteristic of OLAP • Information • Accesses all the data and information wherever it may reside and not limited by volume. C.S.: Logical View

21. METADATA Technical Users (Developers & Analysts) Business Users (Executives & Business Analysts) Data Administrator Metadata Management The link between the DSS system and the business analysts. Critical for maintaining, controlling, and expanding the DSS system. Reduces the cost and cycle time of problem resolution. C.S.: Logical View

22. Metadata Consumer • Business Users • Less technical • Use predefined queries & reports • DSS navigation and definition C.S.: Logical View • Power Business Users • More technical • Ad-hoc • Technical Users • Acquisition & access developers, analysts, data modelers, architects • Need users access patterns & frequency • Transformation rules

23. Metadata Management Business Meta Data Technical Meta Data Transformation Rules Attribute Names Domain Values Access Patterns Entity Relationships Attribute Business Definitions Entity Business Definitions Aggregation Rules Report Business Descriptions List of Available Reports C.S.: Logical View Technical Users (Developers & Analysts) Power Business Users Data Administrator Business Users (Executives & Business Analysts)

24. Data transformation • Finding the right data to satisfy end users needs • Moving the right data to the target • Scheduling and monitoring • Providing visual access • Linking transformations and movement metadata with all other metadata activity C.S.: Logical View

25. Process integration Data integration DW Metadata Operational Data Workflow Management C.S.: Logical View

26. Sequence Diagram C.S. : Process View

27. Metadata Repository Enterprise Reference Data Functional Architecture CASE & Modeling Tools Meta Data Management Meta Data Administration Utilities Meta Data Access Tools Meta Data Movement & Replication Tools Change Management Tools Project Deliverables Generator Operational Systems Data Data Mining & Simulation Tools C.S.: Development View OLAP Data Query, Reporting and Visualization Tools Query Data Quality Assessment Tools Source Data Extract Tools Database Utilities DW Data Marts Data Cleansing Tools Data Transformation Tools Load Validation Tools Operational DB Applications Meta Data Sources Warehouse Management Tools Data Warehouse Trasformation

28. Staging Area Layered Architecture Data Analyst Database Admin Operations Manager Network Admin Applic. Developer IT Users Data Capture Source Data (Internal and/or External) Data Transformation C.S.: Development View Enterprise Warehouse Data Management SupportInfrastructure Replication & Propagation Workflow Management Data Warehouse Middleware Network Management Database Management Systems Management Metadata Logical Data Model Physical Data base Design Data Dictionaries Dependent Data Mart Knowledge Discovery / Data Mining Information Access / Applications Data Analysis Business Users Power Analyst Knowledge Worker Executive/ Manager Customer Contact Application Server

29. Components Integration Data Management C.S.: Development View Integrated Support Infrastructure Data Capture Data Analysis

30. Transformation chain Asynchronous Data Cleaning acquisition & Maintenance Components Integration Data Management WEB Services Data Browser Schedule-driven Summarization acquisition C.S.: Development View Communication System Management Service Infrastructure Query Data capture Report Sched Workflow Change Management Integrated Support Infrastructure Data Analysis & DSS

31. Corporate Data Unix MVS Physical View Server Platform Directory Services DW OLAP C.S.: Physical View Windows NT Intranet Unix WS Windows 95/98/NT Client Platform

32. System Scalability System Sizing • Small Size ( <= 10 M CDR/day ) • Medium Size ( >= 10 M <= 50 M CDR /day ) • Large Size ( >= 50 M <= 200 M CDR / day ) Solutions: • Process distribution (divide et impera) • Different COTS choice (performance and TCO) • Hardware platform C.S.: Physical View

33. Architectural Qualities • Performance (Canned queries, MD Analysis, Ad hoc, Min. Impact on Operational System) • Flexibility (MD Flex, Ad hoc, Change data structure) • Scalability (No. of Users, Volume of Data) • Ease of Use (Location, Formulation, Navigation, Manipulation) • Data Quality (Consistent, Correct, Timely, Integrated) • Connection to the Detail Business Transactions C.S.: Fifth View

34. Summary • Iterative waterfall approach for large projects … • Architecture as a CENTRAL activity for the success of projects • Scalability as a driving factor in this case • Standard adoption (Metadata Coalition OIM Model) • COTS + developed components to meets Time to market and Best-in-class solution • Flexibility in data capturing and high modularity to improve the level of integration with already in place systems Q&A