130 likes | 736 Views
Introduction to OWB(Oracle Warehouse Builder). 2009-04-01. Agenda. Data Warehouse Data Warehouse Concepts ETL Process Oracle Warehouse Builder(OWB) OWB Architecture Data Sources and Data Targets ETL: Mappings ETL: Process Flows Data Quality Management Demonstration
E N D
Agenda • Data Warehouse • Data Warehouse Concepts • ETL Process • Oracle Warehouse Builder(OWB) • OWB Architecture • Data Sources and Data Targets • ETL: Mappings • ETL: Process Flows • Data Quality Management • Demonstration • Extracting Data • Data Profiling and Cleansing • Transforming Data
Data Warehouse Oracle Warehouse Builder Oracle OLAP/ Data Miner • Find Pattern • Predict Behaviour or value • (Classification/ Regression) • Generate Report • ETL (Extract/ Transform/ Load) • Data Quality Control • Meta data Management “one of the major ETL tools in the market “
ETL Process • Extract: extract data from sources and put in a so-called Staging Area(SA), • usually with the same structure as the source. • Transform: join and union tables, filter and sort the calculations. In this • step, we can check on data quality and cleans the data if necessary. • Load: finally, data is loaded into a central warehouse, usually into fact • and dimension tables.
Data sources and Data Targets Targets Sources • Oracle • Tables, Views, MViews, Queues, External Tables, Sqlloader, Transportable Tablespaces, Data Pump… • DB2, Sybase, SQLServer, Informix, Mainframes, … (Oracle Transparent Gateways) • ODBC • Flat Files • XML • Applications • Oracle Ebusiness Suite • PeopleSoft • SAP • Siebel • Oracle • DB2, Sybase, SQLServer, Informix, Mainframes, … (Oracle Transparent Gateways) • ODBC • Flat Files • XML
ETL: Mappings • Declarative modeling of Data Flows • Map from Source to Target • Integrated Data Quality • N&A standardization • Match/Merge • Profiling • Generates SQL & PL/SQL • Merge, transportable tablespaces, data pump, sqlloader, xml data types, BLOBS/CLOBS, … • Leverage custom data transformations
ETL: Process flows • Declarative modeling of Process/work Flows • Co-ordinate execution of Maps and other activities • Create complex transitions • Send email, FTP source/target files, call any external process, SQL Plus, Notifications • Generates Oracle Workflow, Oracle Scheduler & XPDL
Data Quality Management • Data Profiling • Missing or invalid values • Distributions of the values in a specific column • Data Rule for Cleansing
Metadata Management • Dependency Management • Data Lineage at attribute level • Impact Analysis at attribute level • Metadata Snapshots • Change Management (diff, merge and reconcile) • Reporting (browser) • APIs (Scripting, SQL, PL/SQL) • Exchange (import/export)
Demonstration Define Sources & Targets Extract Data Profiling 1. Identifying data sources/ targets and importing metadata 2. Import data and design and execute mappings (Extract) 3. Data profiling and decide data cleansing strategy “Derived Data Rule” “Generated Code” Transform Load 4. Design and execute mappings (Merging) and cleansing 5. Design dimension tables “Generated Code”