You Can ’ t Manage What You Don ’ t Measure

You Can’t Manage What You Don’t Measure FedFleet 2012 Ron Stewart – INL FAST Web Team Fleet Data Management Best Practices FAST Data Quality Issues: Your VMIS Can Help!

FAST data quality: a real problem • DOE, GSA, FAST team • always talking about it… • adding tools to find and fix issues • looking for good ideas on how solve this challenge • What we see in FAST: • systems + tools + processes + policies + procedures + people • Your VMIS may be part of the problem… • … it definitely can be a big part of the solution!

Today’s take-aways… • 2 capabilities your VMIS should provide • 2 strategies for minimizing FAST data quality problems

Real examples… • Reviewed 45 agency FY2011 submissions • Agency “Data Quality & Consistency Report” • Prior-year OMB A-11 AMVFR • FAST’s “Data Call Summary” report • Basic aggregate fleet metrics: • Average gross vehicle cost/mile • Average gross vehicle GGE/mile • Agency use of DQR late in the data call

Real examples… • Every agency reviewed had at least one issue identified • 33 of 45 agencies flagged on AMVFR • 33 of 45 agencies flagged on DQR • 15 agencies flagged on Data Call Summary • 14 agencies had no/minimal use of DQR

Real examples… • OMB A-11 AMVFR: • Monthly operating costs out of normal range • Acquisition costs out of range • Planned acquisitions w/o costs • Armored or LE vehicles w/o costs • Negative out-year inventory projections • Average age of owned vehicles out of range/negative • Missing one or more narrative sections

Real examples… • Data Quality & Consistency Report • Significant swings in one metric w/o corresponding supporting increases • Mismatches in data • Inventory w/o costs • Inventory w/o fuel • Inventory w/o mileage • Operational data w/o inventory

Real examples… • Data Call Summary report • Significant agency swings relative to prior year • Fuel cost & consumption • Mileage • Overall costs • Inventory • EISA Section 141 designation of acquisitions

Gross errors • “Gross” … as in visible in agency aggregates • … but traceable back to • specific groups of vehicles or • specific slices of agency-submitted data

How do we solve this? • Attack the problems where it makes the most sense… VMIS(s) Reporting FAST

VMIS Capability #1 • VMIS should characterize data similar to FAST • Vehicle types & classes • Fleet costs • Difficult to compare VMIS to FAST w/o similar groupings

Strategy #1: Compare, compare, … • Compare VMIS to FAST • along every major data axis • inventory, acquisitions, … • fuel cost & consumption • mileage • fleet costs • variety of reports and tools on the FAST side • DQR, AMVFR, query tool • Investigate, understand, resolve major discrepancies

VMIS Capability #2 • Robust report generation & data querying • Ability to group data similar to FAST • Ability to identify vehicles with anomalies • stale, incomplete, or inconsistent data? • compare vehicles to their own history? • compare vehicles to their peers?

Strategy #2: Attack the problem early • … early in time: • Use VMIS through the year to make sure vehicle-level data quality issues are identified and resolved as early as possible • Dashboards • Data quality & consistency threshold • Monthly or quarterly data reviews • … early in the process: • Fix the problems in the reporting process • Fix the problems in the VMIS(s) • FAST reporting • … complete as early as possible • … review as early as possible

Summary • Key VMIS Capabilities: • Characterize data the same way FAST does • Robust report generation and querying • Strategies to improve data quality: • Compare, compare, compare • Attack the problem early

You Can ’ t Manage What You Don ’ t Measure