640 likes | 709 Views
InfoSphere Metadata Comprehensive Deck. Last Updated 9-8-08. Agenda. 2008 Metadata Market Survey InfoSphere Information Server Metadata Approach InfoSphere Business Glossary InfoSphere Metadata Workbench Third Party Metadata Cognos Integration Case Studies. 2008 Metadata Market Survey.
E N D
InfoSphere Metadata Comprehensive Deck Last Updated 9-8-08
Agenda • 2008 Metadata Market Survey • InfoSphere Information Server Metadata Approach • InfoSphere Business Glossary • InfoSphere Metadata Workbench • Third Party Metadata • Cognos Integration • Case Studies
2008 Metadata Market Survey IBM and Gavilan Research Associates (GRA)
IBM and Gavilan 2008 Survey Highlights: Turning your Data into Information with Metadata Top 5 areas companies said they need to improve their metadata: • Documenting business metadata (business rules & element descriptions) - 80.8 % of respondents • Support for Data Governance and Data Stewardship (67.5% of respondents) • Documenting a Data Warehouse, Data Mart, or Analytical Application (64.8% of respondents) • Support for Data Lineage reporting and/or Data Mapping analysis (64.9%) • Implementing a corporate Data Dictionary or Data Glossary (64.7%) Of those surveyed: Less than 7% said their implementation was highly successful ! IBM & Gavilan Research Associates 2008 Metadata Survey – 287 respondents
What Are Your Top Metadata Management Issues ? Respondents could choose multiple answers, which created a broad spectrum. The “top 5” metadata management issues in 2008 are: • documenting business metadata (business rules & element descriptions) - 80.8 % of respondents • support for Data Governance and Data Stewardship (67.5% of respondents) • documenting a Data Warehouse, Data Mart, or Analytical Application (64.8% of respondents) • support for Data Lineage reporting and/or Data Mapping analysis (64.9%) • implementing a corporate Data Dictionary or Data Glossary (64.7%) Respondent Comments: • “My top issue is getting metadata out of existing toolsets into a repository, where it can be accessed and reported on.” • “Change Management is our biggest issue! How could you miss this point?” Source: IBM / Gavilan Research Associates 2008 MetaData Market Survey
With a trademark label Stored in a can Made by Since Type of food With a special ingredient With many varieties So what exactly is Metadata? Metadata enables you to put context and meaning to things. It is generated and consumed by every organization and software product.
Business Metadata Business rules, Stewardship, Business Definitions, Auditing Terminology, Glossaries, Algorithms and Lineage using business language. Audience: Business users. Technical Metadata Defines Source and Target systems, their Table and Fields structures and attributes, Documentation for Auditing Derivations and Dependencies. Audience: Specific Tool Users – BI, ETL, Profiling, Modeling. Operational Metadata Information about application runs: their frequency, record counts, component by component analysis and other statistics for auditing purposes. Audience: Operations, Management and Business Users. Metadata Primer for Business B T O Literally, “data about data” that describesyour company’s information from both a business and a technical perspective
MetaData is what describes, defines, & inter-relates your key information assets “Social Security Number: a unique identification number issued to citizens, permanent residents, and temporary (working) residents by the US Government ...” pic XXX-XX-XXXX “Profit Amount: a currency value that is calculated by combining data from the Customer Master database and Wholesale Inventory applications . . .” Calculation included on Monthly Report Let’s put Metadata to an Example Suppose you have these items of “data” 552-90-3456 $233,567.00
What data or information exists ? Where is it being used ? What is its business definition ? What other names has it been called or is being called ? How is it inter-related to other information ? Who is using it ? Why do we need it ? When was it last updated ? Metadata helps answer important questions such as: What is “Profit Amount” ? How is it defined ? How is it calculated ? Where is it stored or used ? Is it reliable ? Accurate ? If I make a change to “Profit Amount” – what will be impacted ?
Metadata Business DriversWhat is happening in the Data Integration Market that requires Metadata? • Governance and Compliance Regulations are increasing • How do organizations comply and meet documentation requirements? • How can organizations ensure accountability and responsibility? • Business Competition continues to grow • How do organizations individualize their customer experience? • How can organizations get access to information to make correct decisions? • Costs and system complexities are expanding • How can organizations drive optimization with integration? • How do organizations manage complex software environments?
IBM Data Governance Maturity Model - measurable results Business Transformation
A Brief History of Metadata – Where did we come from? Information Server Launched IBM buys Unicorn Community Focused Repositories … 2008 IBM buys Ascential $50 million to $100 million mainly maintenance and most organizations decided live with metadata definitions in multiple technologies, and manually synchronize 2007 2002 Single Enterprise Repository IBM led AD Cycle Federated Repositories And Registries e.g. Unicorn Understand how software and data services related to business processes workflows, and the requirement to publish, reuse and govern these services. 2000 1990 DataWarehouse Repositories ETL, DQ, DP e.g. Ascential Metastage Steve Born 1980 Metadata Consolidation Solutions less-ambitious more-scoped metadata management solutions 1970 Consolidation to central repository using parsers, bridges and generators e.g. Dovetail 1/3 failed miserably, 1/3 succeeded marginally, 1/3 demonstrated good ROI First Metadata Repositories IBM Data Dictionary 13
InfoSphere Information Server Metadata IBM Clean, controlled metadata supply Enterprise Repositories Where IBM Fits in the Metadata Life Cycle
Metadata Business Challenges Metadata is naturally a very complex subject which virtually all organizations address at some point and time. Some successfully and some unsuccessfully. Key challenges: • Obtaining agreement on what metadata means • What does metadata mean to a specific organization or division? • What metadata is important to track and manage? • How does each group use metadata for their particular job? • Selecting the correct metadata strategy for particular business requirements • How many and what kinds of silos of metadata exist today in organization? • Are there revenue $ at risk, compliance issues, regulatory rules which must be addressed? • Do we have the flexibility to assess the impact of changes with our current architecture? • Promoting adoption of a metadata strategy and associated technology • How does our approach address our different user’s needs? • Is our approach one that easy to use and facilitates adoption rather than hinders it? • Do we have business and technical sponsorship?
The IBM Solution: InfoSphere Information Server IBM InfoSphere Information Server Unified Deployment Transform Deliver Understand Cleanse Discover, model, define, and govern information structure and content Standardize, merge, and correct information Combine and restructure information for new uses Synchronize, virtualize and move information for in-line delivery Unified Metadata Management Parallel Processing Rich Connectivity to Applications, Data, and Content Delivering information you can trust
Unified Metadata Management Operational | Technical Business | Exchange Store Enhance Any User Define relationships, control extensibility, and link 3rd party metadata Active, centrally managed repository with secure access via services layer Share and deliver relevant information across the organization InfoSphere Information Server: Unified Metadata Management • Data integration metadata repository – core to InfoSphere Information Server • Facilitates change management & reuse • Simplifies integration • Increases trust and confidence in information • Enable data governance initiatives
Business Users Subject Matter Experts Architects DataAnalysts Developers DBAs InfoSphere Information Server: Unified Metadata Management Unified Metadata Management Operational | Technical Business | Exchange Store Enhance Define relationships, control extensibility, and link 3rd party metadata Active, centrally managed repository with secure access via services layer Share and deliver relevant information across the organization Role-based Tools with Integrated Metadata Enhance Collaboration & Alignment
Business Glossary Overview • Applications • Users • Business Glossary • Business Glossary Anywhere
Applications of a Business Glossary Simply put, a Business Glossary is created to represent the language of the business, independent of technology • Three Primary Applications: • Ownership • Identifying stewards • Managing content • Collaboration • Common, approved vocabulary • Sharing domain expertise - Business & IT • Auditability • Evolution of language • Centralized management All key enablers to regulatory compliance and support the IBM Data Governance Maturity Model
Business Glossary: For Every Enterprise User Examples of most common users include: • General Business Users • Business Analysts • Data Analysts • BI and ETL Developers • Data Stewards • Subject Matter Experts • Business Glossary Authors • Business Glossary Administrators
Understanding the value of Business Metadata • In the language of the business, independent of technology • Documents the business meaning of data & related technology assets • Used to • define a shared meaning • standardize names • establish responsibility, accountability, and traceability • govern access • share insights & experiences among users • represent business hierarchies • document business descriptions, examples, abbreviations and synonyms • Must be managed bythose that understand the meaning and importance of the information assets to the business • Better aligns the efforts of IT with the goals of the business
InfoSphere Business Glossary Business Users Steward Create and manage business vocabulary and relationships Steward Console Features • Facilitate business & IT communications by creating & managing a common business vocabulary • Web based interface shared across enterprise business teams • Allows creation of stewards & assignmentof their responsibilities for terms & assets. • Link business terms to information assets Benefits • Aligns the efforts of IT with the goals of the business • Provides business context to information technology assets • Establishes responsibility and accountability in accordance with data governance policies Business Interface
Industrial Telco Retail Populate: Import IBM Industry Models Jump-start your Business Glossary with thousands of industry standard terms from the IBM’s Industry Models. Refined by real customer usage, leverage years of experience and expertise to provide the best foundation for your glossary. Financial Services Insurance Healthcare Available now!
Populate: Import InfoSphere MDM for PIM Share your product category information structures from Master Data Management with Business Glossary enabling access by all enterprise users.
Populate: Create and Assign Stewards • Create and Assign Stewards to Terms, Categories or Assets • Who is responsible for this Term? • Who is responsible for this Category? • Who is responsible for this Asset? • View Contact Information for Steward
Access: Simple to use – business browser I didn’t need any special training to use the business glossary. It’s laid-out exactly as I expected. I can set this up so that my people immediately see the most used business hierarchies
InfoSphere Business Glossary Anywhere ANY User Real-time access to business glossary from any desktop application Features • Click on a term and view definition in a pop-up window without loss of context or focus • Intelligent matching returns best candidates in a single search • Search engine for terms and categories • Access steward contact information directly • Security enforced via the Information Server common security layer From Any Application... Benefits • Increased trust and acceptance of information by delivering definitions in context • Expanded adoption of enterprise glossaryoutside of Information Platform technologies • Improved information availability with multiple access mechanisms for electronically stored information (ESI) Pop the Definition!
3. Select Term to view full details. 2. Candidate Terms returned. Business Glossary Anywhere with Cognos 1. Highlight Term Active and click to look up in Business Glossary
Manage: Business and IT understanding Category: Costs Term: Tax Expense Full Name: Tax to be paid on Gross Income “The expense due to taxes …..” (John Walsh is responsible for updates. 90% reliable source) Status: CURRENT Database = DB2 Schema = NAACCT Table = DLYTRANS Column = TAXVL data type = Decimal (14,2) Derivation: SUM(TRNTXAMT) Achieve a common vocabulary between business & technical users! Shared Information Server metadata repository InfoSphere DataStage InfoSphere Business Glossary
Business Glossary Business Benefits • Enables data governance • Common language supports compliance regulations such as Basel II • Represent and expose business relationships • Accountability and responsibility • Assign stewards as single point of contact for control • Improved productivity • Administrators can tailor the tool to the needs of their business users • Access information you need when you need it • Increased collaboration • Capture and share annotations between team members • Greater understanding of the context of information. • Greater trust in information • Immediate, in-context access to terms and definitions, improves decision making
Metadata Workbench Overview • Applications • Users • Metadata Workbench
Applications of Metadata Workbench Simply put, Metadata Workbench allows you to understand where information came from and where it is used. • Three Primary Applications: • Trust • Data lineage for traceability • Understand relationships • Productivity • Assess impact of change • Maximize re-use • Governance • Visibility and reporting • Centralized management All key enablers to regulatory compliance and the IBM Data Governance Maturity Model
Metadata Workbench: For Projects leads & Architects • Project Leaders • Data Architects and Modelers • DataStage Developers • BI and OLAP Developers • Business Subject Matter Experts • Business Users and Compliance Officers Examples of most common users include:
IT Developers Administrators Project Managers & DBAs InfoSphere Metadata Workbench Exploration of assets generated and used by Information Server Features • Cross-suite view and administration of Information Server assets • Explore, analyze and manage metadata graphically and textually • Comprehensive reporting on data lineage • Assess dependencies across Information Server & 3rd party tools • Perform searches and build customized queries Benefits • Assess and mitigate change management risk • Support compliance and governance initiatives requiring data lineage (eg Sarbanes-Oxley, Basel II) • Proactively manage and administer complex data integration environments
Information Server: Optimizing Application Development Import Industry Data Models Exchange Data Structures Services Oriented Architecture Rational Data Architect Information Services Director Link Populate Deploy Transform and Cleanse Common Enterprise Vocabulary Map Sources to Target Model Search and Profile Source Data Link Define Create DataStage and QualityStage Business Glossary FastTrack Information Analyzer Share Share Share Share Unified metadata for active administration, management and reporting Metadata Workbench offers complete visibility and control of metadata
ADMINISTER EXPLORE ANALYZE Manage Integration Assets to enable in-depth analysis Explore key Integration Assets: Jobs, Reports, Databases, Models, Terms, Stewards, Systems, Specifications, Quality Rules Analyze dependencies and relationships between key Integration assets, Business Intelligence Reports and data models • Assign security roles • Link together multiple viewpoints of design assets from ETL, business, BI and modeling with operational metadata • Edit names and descriptions of Integration Assets • Access runtime information to enrich reporting • Import export manager for 3rd party integration • Trace data movement to and from databases, jobs and reports for full lineage • Understand business meaning of columns, tables, and other assets • Assess the impact of change across Integration assets • Robust query builder • Easy navigation of key Integration Assets • Simple and advanced search • Integrated cross-view of Information Server and 3rd party assets • Graphical view of Asset Relationships Metadata Workbench Feature Overview
Explore: Homepage The homepage offers direct access to Discover key Information Assets, Find any Information Asset or execute Queries.
Explore: Graphically Browse Browse Database, Files, DataStage/QualityStage Jobs, Table Definitions or other objects within their Folder and Project placement. Flexible browsing capabilities enable users to find information easier and better manage complex environments
Explore: Textually Search Where and how do I find out details about assets? • Full search and explore capabilities • View in textual list or graphical format • Search multiple ways including by asset class, name or by asset property
IBM Information Server Analyze: Data Lineage Where does a Field of Data in this Report Come From? • Import & Browse Full BI Report Metadata • Navigate through report attributes • Visually navigate through data lineage across tools • Combines operational & design viewpoint Source Tables
Analyze: Cross-Tool Impact Analysis Where Happens if I Change this Column? • Show complete change impact in graphical or list form • Includes impact on reports in BI tools • Visually navigate through impacted objects across tools • Allows impact analysis on any object type
Analyze: Business MeaningWhat does this Field Mean? • Show relationships between business terms, data model entities, and technical and report fields • Provides cross-tool mapping of business terms • Allows field meaning to be understood • Allows business term relationships to be understood
Metadata Workbench Business Benefit • Good exploitation and coordination of metadata across tools is very compelling • Tool Integration & metadata collaboration saves time and money, and improves Quality of results. • An organization’s ability to Govern Data is significantly improved. • A organization is better able to Manage Change, more Agile. • A Shared, Common, Vocabulary saves time and effort, helps development, and makes data more accessible & more understandable. The key is making the metadata capture and share process effortless
IT Developers IT Administrators Import Export Manager for Information Server Expand visibility of metadata touch-points in support of data integration projects Features • Security enforced via Information Server common security layer as well as the 3rd party application security layer • Metadata Bridges interchange metadata with each specific application a consist of a model, a decoder, and an encoder which require no coding. • Import capabilities for 3rd party BI tools (Cognos, Business Objects, MicroStrategy), data modeling tools (ERwin, RDA) and databases (ODBC connections to all major RDBMS) • Support a variety of import formats including XMI, XML, UML, CWM and CSV metadata exchange formats Benefits • No manual interface coding required for 3rd party metadata visibility • Visibility of data modeling to ETL to report layer minimizes risks of overlooking critical dependencies • Leverage common metadata exchange environment for application development consistency