170 likes | 180 Views
Data Warehousing. Naveed Iqbal, Assistant Professor NUCES, Islamabad (Lecture Slides Week # 16). DWH Lifecycle: Methodologies. DWH vs. Software Engineering. Compared to S/W Eng. It is quite a young discipline.
E N D
Data Warehousing Naveed Iqbal, Assistant Professor NUCES, Islamabad (Lecture Slides Week # 16)
DWH vs. Software Engineering • Compared to S/W Eng. It is quite a young discipline. • Does not yet have well-established strategies and techniques for the development process. • A lot of projects fail due to the complexity of the development process. • To date, there is no common strategy for the development of DWH, its more of an art than science.
Implementation Strategies • Top-Down Approach • When the technology is mature and well understood • Business problems that must be solved are clear and well understood • Bottom-Up Approach • Making technology assessments • Organizations that are NOT leading edge technology implementers • Business objectives to be met by DWH are unclear • Current or proposed business process will be affected by DWH
Development Methodologies • Waterfall Model • Linear sequence: Define, Design, Detailed Design, Integration & Testing, Operation & Maintenance • Spiral Model • Sequence of waterfall models • Iterative enhancements • Recognizes that requirements are not always available & clear • Model of choice for DWH Implementation • RAD Model • Scope, Analyze, Design, Construct, Test, Implement, & Review • Early prototypes drive future requirements as business users directly access and manipulate information
Development Methodologies (Contd.) • Data Driven • Bill Inmon’s Approach • Requirements are understood after DWH becomes operational • Development based on the analysis of the corporate data model and relevant transactions • Company goals and user requirements are integrated later on • Goal Driven • Ralph Kimball’s Approach • Determine goals and services the company provides to its customers • Analyze business process
Development Methodologies (Contd.) • User Driven • Wal-Mart Approach • Focus on implementing business strategy • Assumption: Company goals are same for everyone • First prototype: Based on the needs of the business • Business people define goals, set priorities and define business questions supporting these goals • Business questions prioritized, most important questions defined in terms of data element and definition of hierarchies
Where do you start from? • What specific problems the DWH will solve? • What criteria to be used to measure success? • How to manage time and money? • What skills are required?
DWH Life Cycle Model DESIGN ENHANCE PROTOTYPE OPERATE DEPLOY
5 Signs of Trouble • Project proceeded for two months BUT no body has touched the data • End users are not involved hands-on from day one throughout the project • IT team members doing data design have never used the access tools • Summary tables defined before raw atomic data is acquired and base tables have been built • Data design finished before participants have experienced with tools and live data
11 Possible Pitfalls • Weak business sponsor • Not having multiple servers / Very limited resources • Modeling without domain expert • Not enough time for ETL • Low priority for OLAP Cube construction • Fixation with technology • Wrong test bench / machines • QA people NOT DWH literate • Uneducated user • Improper documentation • Doing incremental enhancements (Dev -> QA -> Production)
Top 10-Common Mistakes to Avoid • Not interacting directly with the end users • Promising an ambitious data mart as the first deliverable • Never freezing the requirements i.e. being an accommodating person • Working without senior executives in loop, waiting to include them after a significant success • Doing a very comprehensive and detailed first analysis to do the DWH right the very first time • Assuming the business user will develop their own killer application on their own
Top 10-Common Mistakes to Avoid • Training users on the detailed features of the tool using dummy data and considering it a success • Isolating the IT support people from the end or business users • After DWH is finished, holding a planning and communications meeting with end users • Shying away from operational source systems people, assuming they are too busy.
Top 7-Key Steps for a smooth DWH Implementation • Assigning a full-time project manager, or doing it yourself full-time • Consider handing-off project management • During user interview, don’t go after answers, let the answers come to you • Assigning responsibilities to oversee and ensure continuity • Accept the “fact” that DWH will require many iterations before it is ready • Assign significant resources for ETL • Be a diplomat NOT a technologist
Conclusions • DWH is not simple • DWH is very expensive • DWH is not ONLY about technology • DWH designers must be capable of working across the organization • DWH team requires a combination of many experiences and expertise