1 / 28

Enhancing Data Warehouse ETL Systems Through Software Reuse

Learn how to boost productivity, quality, and cost-efficiency by applying software reuse concepts to Data Warehouse Loader implementation.

danielleb
Download Presentation

Enhancing Data Warehouse ETL Systems Through Software Reuse

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applying Concepts of Software Reuse to the Implementation of Data Warehouse ETL Systems Jiayang ZhouNovember 2001

  2. Applying concepts of software reuse to Data Warehouse Loader Fundamental of software reuse Description of Data Warehouse Loader Implementation of Data Warehouse Loader Reuse Analysis of Data warehouse Loader Summary Data Warehouse Loadersd&m

  3. Applying concepts of software reuse to Data Warehouse Loader Fundamental of software reuse Description of Data Warehouse Loader Implementation of Data Warehouse Loader Reuse Analysis of Data warehouse Loader Summary Data Warehouse Loadersd&m

  4. Definition of Software reuse Software reuse is a process of implementing or updating software systems using existing software assets. design model domain architecture database schema plan documentation test scenario manual standards code software components objects software requirements analysis Data Warehouse Loadersd&m

  5. Importance of Software reuse • Increases productivity • Enhances quality • Saves cost • Reduces software development schedules • Reduces maintenance • Enhances standardization • Increases portability Data Warehouse Loadersd&m

  6. Applying concepts of software reuse to Data Warehouse Loader Fundamental of software reuse Description of Data Warehouse Loader Implementation of Data Warehouse Loader Reuse Analysis of Data warehouse Loader Summary Data Warehouse Loadersd&m

  7. Architecture of data warehouse application Data Warehouse Loadersd&m

  8. Features of data warehouse • A data warehouse is a • Subject-oriented • Integrated • Time-variant • Non-volatile • collection of data for management’s decision making process. Data Warehouse Loadersd&m

  9. The role of Data Warehouse Loader • Extraction: extract raw data from different source system • Transformation: transform data in order to be fit into the architecture of data warehouse • Loading: import data into data warehouse

  10. The requirements of Data Warehouse Loader • Adaptability to any data source systems • Adaptability to any target data warehouse • High operation speed • Ability to deal with large amount of data • Flexibility for data transformation • Portability Data Warehouse Loadersd&m

  11. The reuse consideration of Data Warehouse Loader Data Warehouse Loadersd&m

  12. Applying concepts of software reuse to Data Warehouse Loader Fundamental of software reuse Description of Data Warehouse Loader Implementation of Data Warehouse Loader Reuse Analysis of Data warehouse Loader Summary Data Warehouse Loadersd&m

  13. Architecture of Data Warehouse Loader Data Warehouse Loadersd&m

  14. Loader-engine Data Warehouse Loadersd&m

  15. Loader-interface • Extraction-interface: access to data source • Transformation-interface: different transformation scheme • Database-interface: access to data warehouse for updating and retrieving • Record-interface: construct an object structure for each database record Data Warehouse Loadersd&m

  16. Data Warehouse Loadersd&m

  17. Format of intermediate file • INFO header • FORMAT header • DATA body Data Warehouse Loadersd&m

  18. Applying concepts of software reuse to Data Warehouse Loader Fundamental of software reuse Description of Data Warehouse Loader Implementation of Data Warehouse Loader Reuse Analysis of Data warehouse Loader Summary Data Warehouse Loadersd&m

  19. Reuse development of Data Warehouse Loader • Thorough examination of application requirements • Good documentation • Choosing of Java as programming language • Concrete separation of specification from a particular application Data Warehouse Loadersd&m

  20. Advantages of workflow architecture • Easy maintenance • Module reuse • Temporally decoupling • Manageable and controllable programming • Re-composition of operation sequence Data Warehouse Loadersd&m

  21. Re-composition of operation sequence Data Warehouse Loadersd&m

  22. Appling concepts of software reuse • Code reuse • Adaptability • Modularity • Interface Data Warehouse Loadersd&m

  23. Reuse architecture of Data Warehouse Loader Data Warehouse Loadersd&m

  24. Package overview of Data Warehouse Loader Data Warehouse Loadersd&m

  25. Applying concepts of software reuse to Data Warehouse Loader Fundamental of software reuse Description of Data Warehouse Loader Implementation of Data Warehouse Loader Reuse Analysis of Data warehouse Loader Summary Data Warehouse Loadersd&m

  26. START-MDB project Data Warehouse Loadersd&m

  27. Java and C • Java • slow and large RAM consumption • portability and platform-independence • easy for flexibility • object-oriented feature • C • fast • migration between different systems • difficulty for flexibility Data Warehouse Loadersd&m

  28. Thank you very much for your attention! meg.zhou@sdm.com Data Warehouse Loadersd&m

More Related