Who Owns It & Taking Care of It: Governance & Maintenance Issues

1. Who Owns It & Taking Care of It: Governance & Maintenance Issues Ron Daniel, Jr.

2. Agenda Fundamental processes and outlooks Team structures at different organizations Governance processes Tools

3. Fundamental processes & outlooks • What are the two fundamental processes every organization should implement to maintain its metadata and taxonomies? • Query log / Click trail examination • Error Correction • What are the key outlooks a taxonomist should try to instill in their organization? • Measure & Improve Mindset • Integrated Taxonomy, Metadata, and UI

4. Fundamental process #1 – Query log examination • How can we characterize users and what they are looking for? • Query Log & Click Trail Examination • Only 30-40% of organizations interested in Taxonomy Governance examine query logs* • Basic reports provide plenty of real value • Greatest value comes from: • Identifying a person as responsible for search quality • Starting a “Measure & Improve” mindset • Greatest challenge: • Getting a person assigned (≥ 10%) • Getting logs turned back on • UltraSeek Reporting • Top queries • Queries with no results • Queries with no click-through • Most requested documents • Query trend analysis • Complete server usage summary Click Trail Packages iWebTrack NetTracker OptimalIQ SiteCatalyst Visitorville WebTrends Source: Metadata Maturity Model Presentation, Ron Daniel, ESS’05

5. Fundamental process #2 – Error correction • Errors will happen, and some will be found. What are you going to do about them? • Tagging errors, content errors, taxonomy errors, … • Define an error correction process. • Process will accommodate questions like: • Who looks at it? Is it an error? What are the costs to correct vs. not correct? Does the correction need to be scheduled? etc. • Once a tagging error is corrected, NEVER lose that fact. • Manually reviewed pages are vital for training automatic classifiers • Has implications for metadata specification and review procedures • Over time, multiple error detection methods will be defined • e.g. Statistical sampling of newly added pages • Gradually, additional error correction processes may be defined to deal with particular types of errors You have an error correction process. Would you hate to see it on paper?

6. Fundamental outlooks • Measure & Improve Mindset • Query logs and click trails are prime example • Next place to instrument: Error correction and error detection processes • Integrated handling of Taxonomy, Metadata, UI, & Search • To be most effective, these must work together • Governance structure must help that happen

7. Agenda Fundamental processes and outlooks Team structures at different organizations Governance processes Tools

8. Taxonomy governance as a standards process Closely linked to organizational metadata standard Taxonomy must evolve, but in predictable way Take tips from other standards efforts Team structure, with an appeals process Taxonomy stewardship is part-time role at most organizations Team needs to make decisions based on costs and benefits Documentation and educational material on Taxonomy and Metadata Announcements Comment-handling responsibilities (part of error-correction process) Issue Logs Release Schedule Taxonomy governance overview These practices are in rough order of implementation.

9. Organization 1 – Internal portal for Fortune 50 Diversified Multinational. Executive Sponsor Advocate for the taxonomy team Business Lead Keeps team on track with larger business objectives Balances cost/benefit issues to decide appropriate levels of effort Specialists help in estimating costs Obtains needed resources if those in team can’t accomplish a particular task Technical Specialist Estimates costs of proposed changes in terms of amount of data to be retagged, additional storage and processing burden, software changes, etc. Helps obtain data from various systems Content Specialist Team’s liaison to content creators Estimates costs of proposed changes in terms of editorial process changes, additional or reduced workload, etc. Small-scale Metadata QA Responsibility Taxonomy Specialist Suggests potential taxonomy changes based on analysis of query logs, indexer feedback Makes edits to taxonomy, installs into system with aid of IT specialist Content Owner Reality check on process change suggestions Organization 1: Taxonomy Governance Team Changes Taxonomy Strategist Taxonomist Information Architect 2 Communications Specialist*

10. Organization 2 – A non-profit international organization. Improving information management practices to reduce overlap between many similar vocabularies across many systems. Even when number of vocabularies reduced, some must still have very close links. Business Lead Chairs group. Assures CVs fit with organization’s larger information management effort. Small group management experience, Information management background. Vocabulary Custodians (3) Responsible for content in a specific CV, typically based on organizational lines. Team lead experience, detail-oriented. Familiar with databases and organization processes IT Representative Backups, admin of CV Tool IT administration experience IT Steering Group Oversees Vocabulary Policy Committee Stakeholders Managers of systems using the vocabularies, thus affected by changes. They have a lot of visibility into the process. Control over CV changes is limited, but they schedule their system’s adoption of changes. Additional Roles – available during startup of team, and on an as-needed basis later Training Representative Develops communications plan, training materials Work Practices Representative Develops processes, monitors adherence Organization 2: Vocabulary Policy Committee Other Relevant Staff

11. Organization 3 – Public catalog site for Fortune 50 Retailer. Data for products provided by manufacturers. Business Lead Chairs committee, resolves disputes Marketing Representatives Provide product marketing expertise Advocate for product manufacturers Represent data entry concerns Website Representative Provides input on search and navigation impacts Advocate for customers and other website users Provides search log and click trail analysis Taxonomy Specialist Maintains taxonomy and product catalog Provides data feeds to drive site Organization 3: Taxonomy team Larger team than many retailers, where a single person is responsible. A single person still makes the changes here, but there is some oversight.

12. Realize: Its not totally solo – IT help, Graphics & UI help, Business Goals help, Funding help, Review & QA help… It needs to be part of your objectives Limit the objectives to what can be achieved by you, and by your organization Concentrate: Resource allocation (i.e. Manage your time) Fundamental processes Communications!!! Move towards fundamental outlooks Cherry-pick from Roles Business Lead – align with organization goals, get needed resources, make cost/benefit decisions IT Liaison – Work with IT specialists to get software installed, logs gathered, content harvested, etc. Consider impact of changes on tools and data Taxonomy / Search Specialist – analyze behavior and suggest changes. Implement changes which pass cost/benefit muster Website/User Representative – consider impact of changes on users and job performance What if I have to do it solo?

13. Agenda Fundamental processes and outlooks Team structures at different organizations Governance processes Tools

14. Web CMS Archives Intranet Search ERMS ’ ’ Other Controlled Items CVs Taxonomy governance environment Change Requests & Responses Published Facets Consuming Applications 1: External vocabularies change on their own schedule 2: Team decides when to update facets within Taxonomy ISO 3166-1 Vocabulary Management System Other External Notifications 3: Team adds value via mappings, translations, synonyms, training materials, etc. Intranet Nav. ERP DAM Custodians … Other Internal … ’ ’ 4: Updated versions of facets published to consuming applications Taxonomy Governance Environment

15. Other controlled items • Taxonomy Team will have additional items to manage: • Charter, Goals, Performance Measures • Editorial rules • Team processes • Tagger training materials (manual and automatic) • Outreach & ROI • Communication plan • Website • Presentations • Announcements • “Roadmap” • Advanced practice, requires long planning horizon for organization's IT projects

16. Taxonomy governance: Generic team charter • Taxonomy Team is responsible for maintaining: • The Taxonomy, a multi-faceted classification scheme • Associated materials, such as: • Corporate Metadata Standard • Editorial Style Guide • Taxonomy Training Materials • Team rules and procedures (subject to CIO review) • Team evaluates costs and benefits of suggested changes. • Taxonomy Team will: • Manage relationship between providers of source vocabularies and consumers of the Taxonomy • Identify new opportunities for use of the Taxonomy across the Enterprise to improve information management practices • Promote awareness and use of the Taxonomy

17. Controlled item: Communications plan • Stakeholders: Who are they and what do they need to know? • Channels: Methods available to send messages to stakeholders. • Need a mix of narrow vs. broad, formal vs. informal, interactive vs. archival, … • Messages: Communications to be sent at various stages of project. • Bulk of the plan is here

18. Controlled item: Editorial rules • To ensure consistent style, rules are needed • Issues commonly addressed in the rules: • Sources of Terms • Abbreviations • Ampersands • Capitalization • Continuations (More… or Other…) • Duplicate Terms • Hierarchy and Polyhierarchy • Languages and Character Sets • Length Limits • “Other” – Allowed or Forbidden? • Plural vs. Singular Forms • Relation Types and Limits • Scope Notes • Serial Comma • Spaces • Synonyms and Acronyms • Term Order (Alphabetic or …) • Term Label Order (Direct vs. Inverted) • Must also address issue of what to do when rules conflict – which are more important?

19. Firewall Application Tagging UI UI Tagging Logic Taxonomy governance: Where changes come from Firewall Firewall Application Application Tagging Tagging UI UI UI UI Application Logic Content Content Tagging Tagging Logic Logic Taxonomy Taxonomy Staff Staff Query log Query log notes notes analysis analysis ‘ ‘ missing missing ’ ’ concepts concepts End User End User Tagging Staff Tagging Staff • Recommendations by Editor • Small taxonomy changes (labels, synonyms) • Large taxonomy changes (retagging, application changes) • New “best bets” content • Team considerations • Business goals • Changes in user experience • Retagging cost Taxonomy Editor Taxonomy Editor experience experience Taxonomy Team Requests from other Requests from other parts of the organization parts of NASA

20. Different organizations will need to consider their own change processes. Organization 1: A custodian is responsible for the content, but checks facts with department heads before making changes Organization 2: Analysts suggest changes, editors approve, copyeditors verify consistency Change process MUST also consider cost of implementing the change Retagging data Reconfiguring auto-classifier Retraining staff Changes in user expectations Taxonomy Change Cases Case 1. Renaming a term Case 2. Adding a new leaf term Case 3. Inserting a new term Case 4. Splitting a term Case 5. Deleting a leaf term or subtree Case 6. Deleting a term Case 7. Moving a subtree Case 8. Merging terms Case 9. Adding a CV Case 10. Deleting a CV Processes

21. Taxonomy governance: Taxonomy maintenance workflow Problem? Yes No Add to enterprise Taxonomy Suggest new name/category Review new name Copy edit new name Problem? Taxon-omy No Yes Analyst Taxonomy Tool Editor Copywriter Sys Admin

22. Agenda Fundamental processes and outlooks Team structures at different organizations Governance processes Tools

23. Sample taxonomy editor functionality • Standard and Custom Fields • Standard and Custom Relations • Data Typing, Restrictions, and Consistency • Flexible Reporting • Flexible Importing? • UNICODE • Multiple Vocabulary Support • Inter-Vocabulary Relations • Unique IDs • ISO Codes not sufficient • Workflow • Voting • Change Request Management • Stylistic rules enforcement • Programmability Term Editing Hierarchy Browser

24. Taxonomy editing tools Most popular taxonomy editor? MS Excel Immature industry – no vendors in upper-right quadrant! high Ability to Execute High functionality, high cost low Widely used, cheap, good reporting, bad IDs Niche Players Visionaries Completeness of Vision