1 / 58

IBM Mainframe-Integration Mainframe Change Data Capture

IBM Mainframe-Integration Mainframe Change Data Capture. Presented by. Mainframe Applications + Databases. Workstation/ Client-Server Applications + Databases. Internet/Web Applications + Databases. Mobile Applications + Databases. The current situation:. The current situation:.

meredith-anchoret
Download Presentation

IBM Mainframe-Integration Mainframe Change Data Capture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IBM Mainframe-Integration Mainframe Change Data Capture Presented by

  2. Mainframe Applications + Databases Workstation/Client-Server Applications + Databases Internet/Web Applications + Databases Mobile Applications + Databases The current situation:

  3. The current situation: MainframeApplications+Databases • ADABAS • IDMS/DB • DB/2 • IMS and DL/I • VSAM • DATACOM/DB • other…

  4. The current situation: WorkstationClient/ServerApplications+Databases • SQL Server • Oracle • DB/2 UDB • MS Access • other…

  5. The current situation: Internet/Web-Applications + Databases • Sybase • MySQL • DB/2 UDB • ORACLE • other…

  6. The current situation: MobileApplications+Databases • MS SQL • MS Access • DB/2 UDB • ORACLE • other…

  7. The current situation – Data exchange: Data exchange problems • Different data formats • Different data models • Large data volumes • Limited batch window • Requirement for up-to-dateness of information

  8. tcVISION – Mainframe Change Data Capture The Solution Moving data... → as much as needed → as little... → as transparent... → as flexible... → as secure... AS POSSIBLE

  9. tcVISION – Mainframe Change Data Capture The new data exchange generation

  10. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Basic questions: 1. How current must the information be? Data exchange between source- and target-system should be ... • continuous / in realtime • cyclic / interval based • event based

  11. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Basic questions: 2. What are the attributes of the datasource? The datasource attributes influence the choice for the data capture method ... • Size of datasource • Format of datasource (Structure, DBMS) • Change frequency • Amount of changes and • available resources (CPU-power, network bandwidth)

  12. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Usage of the DBMS logging capabilities • IMS/DB, VSAM, DB/2, DL/I, ADABAS, IDMS, DATACOM • Transfer of changed data in scheduled time frame • Best for batch window • Best for processing right after logfile creation

  13. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Efficient transfer of entire databases • Analyzis for data consistancy • Best for „Initial Load“ prior to log processing • Best for periodic mass data transfer • One step data transfer

  14. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Comparison of data snapshots • Efficient transfer of changed data since last processing • IMS/DB, DL/I, VSAM, DB/2, ADABAS, IDMS, DATACOM, Sequential files • Flexible processing options (SORT etc.) • Automatic creation of deltas by tcVISION

  15. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Realtime capture of changed data • Changes directly obtained from DBMS • IMS/DB, VSAM, DB/2, ADABAS, IDMS, DATACOM • Secure data storage even across DBMS restart • Flexible propagation methods

  16. Up-To-Dateness of changed data tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods

  17. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Data volume

  18. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Steps: • Extraction of database logs • Process meta information • Process changes of selected tables • Propagate changes • Implementation into target database

  19. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Steps: • Read datasource • Process meta information • Selective and structured transfer • Implementation into target databse

  20. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Steps: • Read datasource • Read corresponding snapshot • Determine changes • Process meta information • Propagate changed data • Implementation into target database

  21. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods • Steps: • Capture changes in DBMS • Transfer changes into „Pool storage“ • Parallel processing of changed data from the pool • Propagation of data to the target system • Implementation into target database

  22. tcVISION – Mainframe Change Data Capture The tcVISION Data Capture Methods Summary Every method has been designed to meet special needs and requirements. An efficient and secure data synchronization is guaranteed with all methods.

  23. tcVISION – Mainframe Change Data Capture Transfer and Propagation Propagation to other mainframe systems Propagation to Internet-databases Propagation to mobile systems Propagation to client server databases

  24. tcVISION – Mainframe Change Data Capture Transfer and Propagation • Propagation independent of data origin • Propagation to different target systems (even for the same data)

  25. tcVISION – Mainframe Change Data Capture Examples Case 1: DB/2 Log processing and implementation into DB/2 UDB

  26. tcVISION – Mainframe Change Data Capture Examples Case 2: DL/I Real-Time Capturing and implementation into a DB/2 UDB and VSAM

  27. tcVISION – Mainframe Change Data Capture Examples Case 3: IDMS- Realtime Capturing and bi-directional synchronization with ORACLE, DB2/UDB and MS-SQL

  28. tcVISION – Mainframe Change Data Capture Examples DL/I IDMS SQLServer All cases are Customer implementations, that are currently in production.

  29. tcVISION – Mainframe Change Data Capture Examples • Steps: • Import of the metadata • Assignment of target tables • Definition of processing script • Test and implementation

  30. tcVISION – Mainframe Change Data Capture Examples Import of meta data The tcVISION Manager publishes the layout of the log-record for every table  for later use by the processing scripts

  31. tcVISION – Mainframe Change Data Capture Examples Assignment of target tables Creation of target tables or Assignment of existing tables in the target system.

  32. tcVISION – Mainframe Change Data Capture Examples Definition of a processing script Use the tcVISION script wizard to create the script Definition of all required parameter: Input and output, LUW processing etc.

  33. tcVISION – Mainframe Change Data Capture EXAMPLES Test and implementation Start and test the script Implement script execution into scheduler and automatic processing

  34. tcVISION – Mainframe Change Data Capture Example Technical aspects: Standard data exchange was not possible due to extremely large amount of data. • 760 tables • 200.000 changes per day are processed • Propagation to DB2 UDB under SUN Solaris • Processing time with tcVISION per day less than 30 minutes

  35. tcVISION – Mainframe Change Data Capture Examples Summary Just a few steps to implement a data synchronization process Solid monitoring and logging High stability Changes applied to structures of source databases are automatically processed

  36. tcVISION – Mainframe Change Data Capture Examples Summary Just a few steps to implement a data synchronization process Solid monitoring and logging High stability Changes applied to structures of source databases are automatically processed

  37. tcVISION – Mainframe Change Data Capture Examples • Steps: • Define a collector and pool storage +Definition of a processing script • Creation of target table • Test and implementation DL/I

  38. tcVISION – Mainframe Change Data Capture Examples Creation of a collector/pool combination for the DL/I database Definition of a processing script

  39. tcVISION – Mainframe Change Data Capture Examples Define Collector as DBMS Extension Define Pool to buffer the changes Define processing script to implement into the target database Use project wizard to specify the work-flow

  40. tcVISION – Mainframe Change Data Capture Examples Test and implementation Start the project using the tcVISION Control Board

  41. tcVISION – Mainframe Change Data Capture Examples Technical aspects: Standard data exchange was not possible due to extremely large amount of data. • 40 DLI segments • 200.000.000+ records • 150.000+ changes per day • Implementation: Realtime using DBMS Extension Capturing of changes performed by Online- and Batch-programs.

  42. tcVISION – Mainframe Change Data Capture Examples Additional Implementation: If the target DB/2 is not available, changes are automatically saved into a VSAM KSDS  Highest possible recovery Customer performs amainframe to mainframesynchronization

  43. SQLServer tcVISION – Mainframe Change Data Capture Examples • Steps: • Step 1: • Create a collector/pool combination +Definition of a processing script • Creation of target table • Test and implementation IDMS

  44. SQLServer tcVISION – Mainframe Change Data Capture Examples Import of the SCHEMA definitions for the IDMS database

  45. SQLServer tcVISION – Mainframe Change Data Capture Examples Define Collector as DBMS Extension Define Pool to buffer the changes Define processing scripts to implement the changes Use the project wizard to define the necessary steps

  46. tcVISION – Mainframe Change Data Capture Examples Test and implementation Start the project using the tcVISION Control Board

  47. SQLServer tcVISION – Mainframe Change Data Capture Examples Technical aspects: Conventional synchronization not possible • Bi-directional updates must be supported • Updates in DB2/UDB, ORACLE and MS-SQL are captured by TRIGGERS • TRIGGERS invoke tcACCESS/ODBC to perform IDMS updates • tcVISION recognizes updates from tcACCESS and prevents a loop-back

  48. Mainframe Applications + Databases Workstation/Client-Server Applications + Databases Internet/Web Applications + Databases Mobile Applications + Databases tcVISION – Mainframe Change Data Capture

  49. Mainframe Applications + Databases Workstation/Client-Server Applications + Databases Internet/Web Applications + Databases Mobile Applications + Databases tcVISION – Mainframe Change Data Capture

  50. tcVISION – Mainframe Change Data Capture Data exchange challenges: • Different data formats • Different data models • Large data volumes • Limited batch window • Requirement for up-to-dateness of information

More Related