1 / 13

Data Quality that’s par for the course

Data Quality that’s par for the course. Quality Assurance Methodologies and the Data Quality Golf Card. Introduction. Our “UDW” Product Our ETL Process Creation of a Quality Assurance Environment. The “Up” Methodology. Data Quality as a Percentage

yuri-thompson
Download Presentation

Data Quality that’s par for the course

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Quality that’s par for the course Quality Assurance Methodologies and the Data Quality Golf Card

  2. Introduction • Our “UDW” Product • Our ETL Process • Creation of a Quality Assurance Environment

  3. The “Up” Methodology • Data Quality as a Percentage • Data Analytics with the concept of improving • Scores and numbers that make sense to executives • Works well in a completely defined problem space

  4. The “Down” Methodology • Data Quality score that relates to the number of errors • Data Analytics with the concept of lowering the score • Relates better for Data Sets without completely defined errors

  5. Screens • Screening for Data • Filtering out the “Dirt” Leaving the “Gold” • Our Methodology and Language

  6. Orphaned Data • Orphaned Data is an artifact of building a Data Store or Data Warehouse • Managing Orphaned Data • Testing for Orphaned Data issues

  7. What we’re doing • The Data Quality Golf Card • Using Severity Score, once aggregated called “Data Quality Index” • Meeting with Units, Leaders, and Front-Line staff to continue to add new tests and define a workflow process for fixing them

  8. Types of Tests • Tests for our office, and tests for our clients • Data Integrity (our office) • Workflow (our clients) • Missing Values (both) • Entity Resolution (both)

  9. Getting Buy-in • Using the Score • Showing “Unknowns” on reports • Describing the impact on institutional reporting as it relates to the errors being seen

  10. The Data Quality Golf Card • All tests are organized by the office responsible for resolving the issue • Currently achieved using SQL Queries output into an Excel pivot table • Each score has associated with it a number of test results, resulting in an index • Drilling into the index gives the office what’s needed to solve the errors

  11. Golf Card Demo

  12. The Future of the Golf Card • Implemented in SAS EBI • More Workflow Options • Data Quality Dashboard

  13. Recommended Reading • The Kimball Group Reader • ISBN: 978-0-470-56310-6 • Chapter 11.12, Data Quality Screens • MDM in Practice • ISBN: 978-0-470-91055-9 • Customer Data Integration • ISBN: 978-0-471-91697-0

More Related