Data Virtualization & Information As A Service ( IaaS )

Data Virtualization&Information As A Service (IaaS) By Anil Allewar Senior Solutions Architect - Synerzip

About Me!! Anil Allewar Senior Solutions Architect @ Synerzip Technology Evangelist & speaker Core interests: JEE, EAI, EII Confidential

Agenda • Use cases • What does it mean? • Architecture explained • Implementation Frameworks • Demo • Questions?

Why it makes sense? Confidential

Use Cases Confidential

Traditional Data Integration Confidential

Problems with ETL More than 1 copy of data for staging Intermediate data => Errors Lead time to add new source Domain knowledge for mapping Batch Process => No real time data Confidential

Problems with DBMS consolidation Alternate approach => Single EIS (say RDBMS) Extensive changes to existing apps Might not satisfy everyone’s requiremets Confidential

Data Virtualization & Federation Single API to access data Only metadata stored at virtualization layer Real time access without copying/moving data Federate data across hetero/homogenous sources Confidential

Data Virtualization Confidential

Architecture Common Access API Translator 1 Virtual Database Connector 1 Translator 2 User Application Connector 2 RUNTIME & QUERY ENGINE Confidential

Vendors • Commercial Products • Composite Software • http://www.compositesw.com/data-virtualization/ • Denodo • http://www.denodo.com/en/product/overview.php?n=h • IBM • http://www-03.ibm.com/software/products/en/ibminfofedeserv • Informatica • http://www.informatica.com/us/data-virtualization/ • Red Hat • http://www.redhat.com/products/jbossenterprisemiddleware/data-virtualization/ • Open Source • Jboss Teiid • http://teiid.jboss.org/ Confidential

Selected Platform – JBoss Teiid Open Source JEE standards Number of relational/NoSQL/ERP/CRM data stores Add custom EIS support using JEE components Active & responsive community Synerzip contribution: Defect discovery, root cause analysis, feature verification Confidential

Teiid Components • Virtual Database • container for components used to integrate data from multiple data sources • Source Models • structure and characteristics of physical data sources • View Models • structure and characteristics of abstract structures you want to expose to your applications • Teiid Designer • Eclipse based UI to dynamically discover data source objects and apply data federation • Generate virtual database from 1 or more sources Confidential

Teiid Components • Translator • Provides abstraction later between Teiid Query Engine and source system • Convert Teiid SQL commands to source specific execution commands • Convert result data from source system to Teiid specific format • Resource Adapter • Provides connectivity to the physical data source • Integration provided through Java Connector Architecture (JCA) API Confidential

Teiid – Supported EIS • Amazon SimpleDB • Apache Accumulo • Apache SOLR • Cassandra • File • Google Spreadsheet • JPA • LDAP • Excel – as file • SalesForce • JDBC • MS access, DB2, derby, excel-odbc, greenplum, h2 , hive(for accessing Hadoop), oracle, teradata and most RDBMS • MongoDB • Object • OData • OLAP • Web Services • SAP Netweaver Gateway Confidential

Performance Characteristics • Access same data using Oracle and Teiid drivers • Retrieval times comparable when accessing tables having no Blobs ms Confidential

Performance Characteristics • Teiid slower when accessing Blob data • Can be tuned Confidential

Demo JDBC API mySQL Translator RDBMS Resource Adapter Federated VDB MongoDB Translator JDBC Client mySQL MongoDB Resource Adapter TEIID RUNTIME & QUERY ENGINE Confidential

Demo-Steps • Pre-requisites • mySQLserver 5.5+ installed • MongoDB2.4.x+ installed • Steps • Load the mySql and MongoDB database with sample data • Setup environment – JBoss, Eclipse • Create Teiid project in Eclipse using Teiid designer • Import source model using JDBC • Create the virtual model and federate data from the source model • Create a virtual database (VDB) and deploy to JBoss • Access data using JDBC client or through browser using OData Confidential

Demo – Scenario Federated Data Confidential

Demo – Connection Profile Confidential

Demo – Source Model Confidential

Demo - Source Model Generation Confidential

Demo – Map Source To View Confidential

Demo - Association Confidential

Demo – Data Federation Confidential

Demo – Source Code • Source code • https://github.com/Synerzip/JBoss-Teiid • Contains • Configuration files • Instructions • “How-to” videos • VDBs, source models and view models Confidential

Conclusion • Data Virtualization and Federation is a rapidly emerging technology that solves traditional BI/ETL problems. • It provides lower time to market, distributes data across the enterprise as a service and provides real time access to enterprise data. Confidential

Contact Me • anil.allewar@synerzip.com Confidential

www.synerzip.com Hemant Elhence hemant@synerzip.com 469.322.0349 Questions? • 36 84

Synerzip in a Nutshell • Software product development partner for small/mid-sized technology companies • Exclusive focus on small/mid-sized technology companies, typically venture-backed companies in growth phase • By definition, all Synerzip work is the IP of its respective clients • Deep experience in full SDLC – design, dev, QA/testing, deployment • Dedicated team of high caliber software professionals for each client • Seamlessly extends client’s local team, offering full transparency • Stable teams with very low turn-over • NOT just “staff augmentation”, but provide full mgmt support • Actually reduces risk of development/delivery • Experienced team - uses appropriate level of engineering discipline • Practices Agile development – responsive, yet disciplined • Reduces cost – dual-shore team, 50% cost advantage • Offers long term flexibility – allows (facilitates) taking offshore team captive – aka “BOT” option

Our Clients

Thanks! Call Us for a Free Consultation! Hemant Elhence hemant@synerzip.com 469.322.0349

Data Virtualization & Information As A Service ( IaaS )