1 / 32

Oracle Data Integration Strategy and Roadmap Oracle Fusion Middleware Product Management

Oracle Data Integration Strategy and Roadmap Oracle Fusion Middleware Product Management. Agenda. Introduction to Oracle Data Integration Business Drivers for Data Integration Benefits from a Modern Data Integration Platform Key Oracle Data Integration Products

Mia_John
Download Presentation

Oracle Data Integration Strategy and Roadmap Oracle Fusion Middleware Product Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Oracle Data Integration Strategy and RoadmapOracle Fusion Middleware Product Management

  2. Agenda • Introduction to Oracle Data Integration • Business Drivers for Data Integration • Benefits from a Modern Data Integration Platform • Key Oracle Data Integration Products • Oracle Data Integration Solution • Oracle GoldenGate Overview • Data Integrator Overview • ODI & GG Together • Best of Breed Integration for Batch and Realtime Data Integration • Support any Type of Data Integration Use Case • Implementing Best-Practice Technical Pattern for Data Warehousing • Technical Details – How it Works • Demonstration and Q&A (if available)

  3. Data Integration PortfolioI N T R O D U C T I O N

  4. Business Drivers for Data IntegrationEssential Ingredient for Information Agility Strategic Value of Data Integration • Consistency for major enterprise initiatives like BI, DW, & MDM • Common technical foundation platform across data silos • Central point for data governance, availability and controls Key Data Integration Use Cases • BI, DW, and OLTP Data Integration & Replication • SOA, Enterprise Integration & Modernization • Migrations and Master Data Management

  5. Benefits from a Modern DI PlatformData Integration is Infrastructure that enables Business Value Do More with Less Design metadata-driven integration Leverage skills & dictate patterns Compete Globally 24X7 Ensure continuous uptime Access data in real time Use Data for Competitive Advantage Ensure the quality of your data Actively govern most valuable asset Automate and Adapt Business Processes Expose data services for reuse Orchestrate processes using SOA

  6. Key Data Integration Products • Heterogeneous E-LT & ETL • High-speed Transformations • OLAP Data Loading • Data Warehouse Loading • Comprehensive Integration • ELT/ETL for Bulk Data • Service Bus • Process Orchestration • Human Workflow • Data Grid • Real Time Data Replication • Changed Data Capture • DBMS High Availability • Disaster Tolerance • Data Service Modeling • XQuery Data Federation • Data Security/Redaction • XA Compliance • Business Data / Metadata • Statistical Analysis • Time Series Reporting • Integrated Data Quality • Cleansing & Parsing • De-duplication • High Performance • Integrated w/ODI

  7. Oracle Data Integration SolutionBest-in-class Heterogeneous Platform for Data Integration Custom Applications MDMApplications BusinessIntelligence Activity Monitoring SOAPlatforms OracleApplications Comprehensive Data Integration Solution SOA Abstraction Layer Process Manager Service Bus Data Services Data Federation Oracle Data Integrator Oracle GoldenGate Oracle Data Quality ELT/ETL Real-time Data Data Profiling Data Transformation Log-based CDC Data Parsing Bulk Data Movement Bi-directional Replication Data Cleansing Data Lineage Data Verification Match and Merge Storage Data Warehouse/Data Mart OLTPSystem OLAP Cube Flat Files Web 2.0 Web and Event Services, SOA

  8. Data Integration PortfolioG O L D E N G A T E

  9. Oracle GoldenGate OverviewEnterprise-wide Solution for Real Time Data Needs • Standardize on Single Technology for Multiple Needs • Deploy for Continuous Availability and Real-time Data Access for Reporting / BI • Highly Flexible • Fast Deployments • Lower TCO & Improved ROI Disaster Recovery, Data Protection Standby (Open & Active) Zero Downtime Migration and Upgrades Log Based, Real-Time Change Data Capture Operational Reporting Reporting Database OGG ETL EDW ODS ETL Heterogeneous Source Systems EDW Real-time BI Query Offloading Data Distribution

  10. How Oracle GoldenGate Works Modular De-Coupled Architecture Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to target(s). Route: data is compressed, encrypted for routing to target(s). Delivery: applies data with transaction integrity, transforming the data as required. Trail Trail Capture Pump Delivery LAN/WANInternet TCP/IP TargetDatabase(s) SourceDatabase(s) Bi-directional

  11. Data Integration PortfolioD A T A I N T E G R A T O R

  12. Oracle Data Integrator Enterprise EditionOptimized E-LT for High Performance, Productivity and Low TCO Legacy Sources E-LT Transformation vs. E-T-L Any Data Warehouse Declarative Set-based design Application Sources Change Data Capture Any Planning System Hot-pluggable Architecture OLTP DB Sources Pluggable Knowledge Modules 12

  13. How ODI Works: E-LT ArchitectureHigh Performance, Flexible, Lightweight Architecture • Key Architecture Benefits: 100% Java, Open APIs, fast E-LT PackagedApplication Business Intelligence& Data Warehouse ODI Agent ODI Agent may be deployed in any part of the architecture LKM IKM JKM A D B C$_0 I$ E$ (Errors) LKM C$_1 File C IKM CKM RKM Check-Load Transform Extract-Load

  14. Data Integration PortfolioO D I & G G T O G E T H E R

  15. Oracle Data Integration SolutionBest-in-class Heterogeneous Platform for Data Integration OracleApplications Custom Applications MDMApplications BusinessIntelligence Activity Monitoring SOAPlatforms Comprehensive Data Integration Solution SOA Abstraction Layer Process Manager Service Bus Data Services Data Federation Oracle Data Integrator Oracle GoldenGate Oracle Data Quality ELT/ETL Real-time Data Data Profiling Data Transformation Log-based CDC Data Parsing Bulk Data Movement Bi-directional Replication Data Cleansing Data Lineage Data Verification Match and Merge Storage Data Warehouse/Data Mart OLTPSystem OLAP Cube Flat Files Web 2.0 Web and Event Services, SOA

  16. Traditional ETL + CDC Invasive Capture on OLTP systems using complex Adapters Transformations in ETL engine on expensive middle tier servers Bulk load to the data warehouse with large nightly/daily batch Best-of-Breed Data IntegrationHeterogeneous, Real-time, Non-Invasive, High Performance E-LT, and Low Hardware Costs Oracle E-LT + Real-time Extract Trickle Xform Xform Bulk GG+ ODI GG+ ODI LookupData LookupData Staging Load Heterogeneous Continuous feeds from operational systems Non-invasive data capture Thin middle tier with transformations on the database platform (target) Mini-batches throughout the day or bulk processing nightly

  17. Support Any Type of Data IntegrationBest of Breed means using the Right Tools for the Job! Analytical OLTP Query / Report EDW ODS ODS OLTP OLTP Operational OLTP Old OLTP New OLTP OLTP OLTP OLTP OLTP Heterogeneous Heterogeneous Heterogeneous

  18. ODI-EE Integration with GoldenGateNon-invasive Data Capture combined with ODI ELT strengths • Key Benefits: • Eliminate Overhead  no need for DB API overhead on the Source, or the invasiveness of the ODI J$ objects on the Source system, • Automate GoldenGate  automation of GG deployment directly from ODI GUI • Provide Common DW Pattern  supplies a common pattern for mini-batch style (non-real-time) DW aggregate loads Source Systems ODI Staging & Target ODI J$ Tables Replicated Source Tables Source DB’s Target EDW Transactional RDBMS Replicated Source Tables ODI CDC Framework Target Tables Source Tables J$ Execute end-to-end CDC Generate all ODI CDC infrastructure ODI Generate all GG deployment files

  19. Oracle’s Data Integration Joint SolutionBest-of-Breed and Proven Oracle GoldenGate Oracle Data IntegratorEnterprise Edition Technology Differentiators: • E-LT architecture for best performance of high data volume transformations • Lowest latency and highest throughput; non-invasive, low overhead Performance • Knowledge Module architecture for extensibility and flexible connectivity • De-coupled architecture; multiple deployment styles; open and extensible Extensible & Flexible • SOA-native, integrated with Fusion MW to fit future enterprise architectures • Maintain transactional integrity; resilient against interruptions and failures Enterprise

  20. ODI & GG TogetherT E C H N I C A L D E T A I L S

  21. Leverage Database Optimizations: Native SQL; Native Functions; Native Loads; Native Journaling / CDC Tailor to an organization’s existing best practices Ease administration work Reduce cost of ownership Overview of the ODI KM Framework Pluggable Knowledge Module Architecture Reverse Engineer Metadata Journalize Read from CDC Source Load From Sources to Staging Check Constraints before Load Integrate Transform and Move to Targets Service Expose Data and Transformation Services Reverse Integration w/GoldenGate is here! WS WS WS Staging Tables Load Integrate Services CDC Check Journalize Target Tables Sources Error Tables Benefits 21

  22. ODI CDC in a NutshellA General Framework for Change Capture on Source DBs Automatic w/JKMs Journal Tables Capture Services Create Capture Process Start/Stop Capture Process Subscription Services Manage Consistency Sets Register/Un-register Subscriber Consumption Services Consumption Views Consumption Operations Extend Window Lock/Unlock Subscriber Purge Operations Capture Services Capture Process Capture Process Capture Process Consumers Consumption Services ExtendWindow() PurgeJournals() LockSubscriber() UnLockSubscriber() Subscription Services Journal Tables RegisterSubscriber() AddTableToConsistencySet() Source Data

  23. Overview of the IntegrationUsing ODI & OGG Together Staging DB Transactional RDBMS Source Tables Target DB Target Tables Replicated Source Tables ODI CDC Framework ODI Interfaces Datapump Extract WAN Replicat Source trail files Staging trail files

  24. Scenario: Analytics & Reporting Staging DB1 Replicated Source Tables ODI CDC Framework Target DB Target Tables Transactional RDBMS Source Tables Datapump Staging DB2 WAN Replicat Staging trail files Extract HistoricAnalytics/ Reporting Datapump Replicat Source trail files WAN Replicated Source Tables Staging trail files Realtime Reporting

  25. Step-by-step How It Works Staging DB Transactional RDBMS Source Tables Target DB Target Tables

  26. 1. Replicated tables created with Common Format Designer (using ODI) Staging DB Transactional RDBMS Source Tables Target DB Target Tables Replicated Source Tables

  27. 2. Start Capturing Changed Data in Source (OGG Extract process) Staging DB Transactional RDBMS Source Tables Target DB Target Tables Replicated Source Tables ODI CDC Framework Extract Source trail files

  28. 3. Initialize Staging and Target Data (with ODI or optionally, OGG) Staging DB Transactional RDBMS Source Tables Target DB Target Tables Replicated Source Tables ODI CDC Framework ODI Interfaces ODI Interfaces Extract Source trail files

  29. 4. Start Replication / Propagate Changes to Target DB (OGG and ODI) Staging DB Transactional RDBMS Source Tables Target DB Target Tables Replicated Source Tables ODI CDC Framework ODI Interfaces Datapump Extract WAN Replicat Source trail files Staging trail files

  30. Simplified & Productive Development D E M O N S T R A T I O N

More Related