90 likes | 223 Views
Future of Data Warehousing Sudin Baraokar Business Intelligence Conference, Pune January 29 th 2011. Future of Data Warehousing from ETL to Information Unification, Integration & Insights.
E N D
Future of Data WarehousingSudin BaraokarBusiness Intelligence Conference, PuneJanuary 29th 2011
Future of Data Warehousing from ETL to Information Unification, Integration & Insights Sudin predicts the future of Data Warehousing by shifting focus from operational issues to informational transformation that will be engineered by need for faster reporting, more complex analytics and better understanding of the Warehouse ecosystem including the important aspect of Information velocity and variety….with the help of some Golfing Strategy! Sudin Baraokar Head – Center of Innovation, Barclays Technology Center India
Current State of the Union - DW-BI • Operational issues – Where do we start and how do we end??? • Silos • Manpower • Traceability, Audit, Lineage • Version Control – of what? • Release management • Data and User governance • Data inflation • Information Paralysis Analysis • Tool Proliferation • Where is the ROI? • Limited Self Service • Query performance - over the weekend! • Data ineQuality - Open a Problem Ticket! • Documentation • Upgrade costs - $$$ • S2M – Source to Market • Ownership Golf course is of 18 holes (72 Strokes) and the idea is to win with the lowest number of strokes. If you hit 72 shots then you are “Par” for the course. If you take more then 72 then you have a handicap!
DW Platform Processing Engines • DW Platform Processing Engines…..need to understand how they work • Understanding of the Processor architecture and component throughput • Multiple cores • Multiple nodes • Resource pooling • Virtualization Support – Internal or External • Linkage to the Application development Teams • Availability of a simulator • E2E SWOT • Impact on Query Development • Increase in Horsepower increases the skew • Platform upgrade • DW Lifecycle focus – Extract, Load, Transform You are allowed max 14 Clubs in your Golf set. Drivers, woods, Irons, Wedges and a putter. Driver is for over 200 Yards, Woods 150-200, Irons 100 – 150, Wedges 20-100 and putter for a few yards.
Information Dynamics…….Velocity and Variety • Frequency of Information Assimilation vs the Amount of Information dissemination • More information variability increases the complexity of Processing • Information Velocity • Cadence based • Market based • Process based • Rule based • Workload based • Information Variety • Report based • Device based • Standards based • Governance based
Research on Information Patterns…….Data Tectonics can lead to an Information Meltdown! • Type of industry • Financials • High Velocity related to Higher Risk • Industrial • High Variety due to process Complexities • Services • Mid Velocity due to Lower Risk • Technology • High Variety due to higher level of componentization
Data Extortion....Fastest way to Information Insights! • Power shift to Business users expecting relevant data in Real Time • Technology will have to focus on faster through put processing - Workloads & Transformations • Increased usage of mobile devices will further push data on demand • Integrated Self Service It is mandatory to shout “Fore” if you stroke a ball and think it may hit another living being on the Golf Course!
Open Research on Information Management • Work with Academia • Collaborate with Solution Providers • Get involved in Forums • Formulate Beta’s • Q & A Panel • Sudin Baraokarsudin.baraokar@barclays.com • Mukund Wadekarmukund.wadekar@barclays.com • Aditya KumarMishraaditya.m@barclays.com
Sudin is second from Left – Aircel PGTI Pro-Am Player Championship 2010, Pune, Oxford Club