Evolution of Metadata Management at ABS: Towards the 2020 Vision

1. Case Study Australian Bureau of Statistics A brief history of metadata at the ABS Simon Wall Metadata Team Leader simon.wall@abs.gov.au

2. Premetazoic era • 1905-1973 • before the term 'metadata' existed • some metadata management anyway • but no metadata management strategy.

3. Protometazoic era • 1973-1990 • created data dictionaries • limited reuse of common data elements • not fully corporate in scope • driven by technology, not by strategy

4. Mesometazoic era • 1991-2000 • focus on data warehousing for output • beginning of a strategic view • Towards a Unified Data and Metadata System at the ABS (1991) • developed core metadata repositories • statistical activities (ABS: collections), datasets, data items (data elements/variables), classifications, terms • seen as overhead by subject matter areas • limited return for effort • quality questionable, not actively maintained

5. Neometazoic Era • 2001-2007 • reorganised as Corporate Metadata Repository • adding services interfaces • legacy repositories both asset and liability • vision greater than implementation • Formal strategy adopted • Strategy for end-to-end management of ABS metadata (2003) • Some new development work • 11179-based Data Element Registry • Quality Declarations

6. Holocentric epoch • 2008+ • ABS executive seek clear and compelling picture of long term metadata aspirations. • not provided by existing strategy. • How would it improve the way the organisation operates? • What new capabilities would it deliver? • Are they the capabilities we need most? • Is the strategy actually achievable? • Is it the most appropriate future for us to be investing in? • now developing "2020 Vision" • encapsulating longer term ABS aspirations for metadata

7. Drivers for 2020 vision • Uptake of Corporate Metadata Respository services very slow • existing processing systems "monolithic" • not easy to modify • existing processing systems have "special needs" • customisations required • lack of funds and business drivers • disconnect between user view and description for systematic manipulation. • processing systems unable to provide metadata to describe transformations

8. Drivers for 2020 vision • Frame of reference for ABS metadata management requirements is no longer defined by the boundaries of the organisation • Developing infrastructure that can be shared with other producers of statistical data within Australia • Other government agencies now have their own focus on interoperability, including metadata. • Software collaboration / open source software • Interoperate with agencies with a focus on administrative, geospatial, or research-oriented data content.

9. Where are we going? • OneThe practical complexities and difficulties of using metadata in an "end to end" context to actually drive actual ABS processes is becoming clearer. • Two There is pressure to make our metadata capabilities less ABS specific • more flexible, interoperable and generic - . • These factors suggest that charting a successful way forward in regard to metadata management for the ABS will require a "paradigm shift", a new era.