Planning Made All The Difference: City of St. Louis Case Study

1. Planning Made All The Difference: City of St. Louis Case Study 2008 Taxonomy Boot Camp Organizing Information for Search & Discovery San Jose McEnery Convention Center San Jose CA City of St. Louis, Missouri

2. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

3. Background • Started as a Community Information Network not a municipal website in 1995. • Focus was on general useful content (history, Neighborhoods, Education, Housing, Services, and Government). Municipal information came later. • By 2006 site had over 10,000 pages of content, 120+ stand-alone dynamic applications and no overall way to stitch everything together • Organizing and finding information was an exercise in faith and frustration. Complaints became increasingly common. City of St. Louis, Missouri

4. Site Consolidation Goals Decided to overhaul website and consolidate web presence using a content management solution. Goals: • Classification and organization of content • Integration of 120+ dynamic applications • Use standard metadata and create a Standardized Municipal Topic Classification Challenge: • How? City of St. Louis, Missouri

5. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

6. Research & Review of Practices • No standardized US model for government available • International models reviewed: • Australia: Thesaurus of Australian Government Subjects http://www.naa.gov.au/records-management/create-capture-describe/describe/agls/index.aspx • Canada: Government of Canada Core Subject Thesaurus http://en.thesaurus.gc.ca/intro_e.html • United Kingdom: Integrated Public Service Vocabulary (IPSV) http://www.esd.org.uk/standards/ipsv/ City of St. Louis, Missouri

7. Top Terms Deep Structure British IPSV as model The British central government developed and promotes the Integrated Public Services Vocabulary as the standard for all British Governmental bodies. http://www.esd.org.uk/standards/ipsv/viewer/ City of St. Louis, Missouri

8. How to Integrate Our Early Classification Effort • User Survey • Services A to Z • Random Site Topics • Category Agreement Assessment City of St. Louis, Missouri

9. Survey: Category Agreement Assessment (CAA) • 400+ terms on site • Random presentation of 20 terms • User asked to type primary and secondary category • Results generated list used for our A to Z pages City of St. Louis, Missouri

10. Category Agreement Assessment: A to Z • Resulted in 45 categories • Created A-Z list • Topics • Categories • Beginning of a Folksonomy City of St. Louis, Missouri

11. Tagged sample contents with IPSV • Mapped our A to Z topics and categories against British IPSV City of St. Louis, Missouri

12. Findings • Our Content topics mapped well to IPSV • IPSV hierarchy was too deep and complex • Content classification needed to expand beyond topics • Effort was beyond our team’s expertise • Outside Information Architecture services was needed to create a robust but simpler classification City of St. Louis, Missouri

13. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

14. Retaining IA Expertise • Identified possible resources mentioned in reputable resources with a mandate toward standards and Dublin Core • Developed list of questions • Interviewed contacts to identify good match and clarify scope • Selected Taxonomy Strategies to filter all information and devise a well structured Information Architecture document to guide our CMS implementation City of St. Louis, Missouri

15. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

16. Materials Used to Create Classification Schema • IPSV • List and frequency of Search Terms • User Survey Results: • Usability Assessment Results • List of information people could not find on website • Category Agreement Assessment (folksonomy) • A to Z content items and categories • Audience/user scenarios • CMS Application requirements list • Site Architecture and Navigation Requirements City of St. Louis, Missouri

17. Summary of St. Louis Metadata Standard Text Fields: • Title, Description, Date, Headline, Subhead, … • Useful for search, and for linking from one page to another. Typical Vocabulary-based Fields: • Access, Audience, Content Type, Organization, Topic • Content Type provides the basis for page templates. • Organization and Topic provide basis for dynamic navigation. Special Vocabulary-based Fields: • Event Type, Neighborhood, Procurement Type, Ward • These are key to improving the dynamic navigation. City of St. Louis, Missouri

18. Summary of St. Louis Content Types.CV City of St. Louis, Missouri

19. St. Louis Topics.CV Before Usability Testing Business & Industry Business Assistance, Incentives and Promotion Business Regulation and Licensing Business Taxation Labor Market Facts & Figures Local Economy Small Business Workforce Training and Development Community Births, Deaths, Weddings, and other Life Events Beautification and Nuisance Reduction Community Facilities Facts, Figures & Community Spirit History Neighborhoods Neighborhood Associations Neighborhood Programs & Events Social and Support Groups Volunteering Non-Governmental Organizations Immigration and Nationality Education & Skills Training Adult and Community Education Further and Higher education Homeschooling Public and Private Schools Special Educational Needs and Additional Support Employment, Jobs & Careers Employment Rights and Regulations Local Government Employment Opportunities Self employment Unemployment and jobseeking Youth Employment Environment Environmental Protection Hazardous Materials Waste Management & Recycling Climate Government Local Government Records Local Governmental Units Complaints and Service Requests Elections & Voting Elected Officials Federal and State Government Services Fees & Payment Information Local Government Services Permits, Inspections & Certifications Policy Making Procurement Taxation & Revenue Wards and Neighborhoods Account & Annual Budget & Operating Plan Employees Health Animal Control Immunizations and Public Health Food Safety Nutrition and Preventative Care Housing & Community Development Home Ownership Home Rental Housing Financial Programs Home Repairs and Renovation Relocation Temporary Housing Homelessness Leisure & Culture Arts and Entertainment Events Children's and Teens Activities Libraries, Museums & Cultural Venues Gambling and Lotteries Parks and Gardens Pets and Animals Shopping & Dining Sports and Recreation Tourism Planning & Urban Development Land Acquisition Preservation Planning Property Assessment Zoning Public Safety Police Fire Dept. Ambulance and Emergency Services Building Inspections & Permits Courts and Justice System Legal Assistance Reporting & Operation Planning Transport & infrastructure Airports Construction, Maintenance & Improvements Electric, Gas, Sewer & Water Utilities Telephone, Cable TV & Internet Utilities Motor Vehicles Public Transportation Parking Rail and Water Transportation Streets, Sidewalks & Streetlighting City of St. Louis, Missouri

20. Dublin Core Metadata Standardhttp://www.dublincore.org • Followed the Dublin Core Metadata Element, a vocabulary of fifteen properties for use in resource description • Mapped Dublin Core properties to corresponding metadata fields in the proprietary CMS • Coded to output DC metadata in source code • Examples: City of St. Louis, Missouri

21. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

22. Taxonomy Testing 3 testing approaches used: • Tagging representative content • Card-sorting and use-based scenario testing with focus groups • Benchmarks and metrics to evaluate usability test results, identify coverage gaps, and provide guidance for changes. City of St. Louis, Missouri

23. Tagging Representative Content Test Cases: • A to Z topics • Random URLs • Human Services content pages City of St. Louis, Missouri

24. Card Sorting and Use-based Scenarios with Focus Groups • 22 representative questions were selected • 42 participants were tested • Testers were asked “where would you look for …” • Testers were asked to “think aloud” • Pre- and post questions were asked • Initial findings indicated that 8 of the 22 scenarios may require revisions • Governance Team agreed on revisions • 8 of the original scenarios were re-tested using 8 new participants • Modifications to the proposed taxonomy were based on the findings and approved by Governance Group. City of St. Louis, Missouri

25. Topics • Content Types • Government Local Government Records Local Governmental Units Complaints and Service Requests Elections & Voting Elected Officials Federal and State Government Services Fees & Payment Information Local Government Services Permits, Inspections & Certifications Policy Making Procurement Taxation & Revenue Wards and Neighborhoods Account & Annual Budget Employees Topics • Business & Industry • Community • Education & Skills Training • Employment, Jobs & Careers • Environment • Government • Health • Housing & Community Dev • Leisure & Culture • Planning & Urban Development • Public Safety • Transport & infrastructure Task-based Card Sort Representative Question Example: You are planning to build a new deck on your house.  You want to know if you need a permit for this. Which topic would you select to get started? City of St. Louis, Missouri

26. Analysis of Task-based Card Sort City of St. Louis, Missouri

27. Summary Usability Testing Process • Card Sorting and Use-based Scenarios • Governance Team Members Involvement • Re-testing and Validation • Changes to Taxonomy and Metadata based on results For Example: • ‘Public Safety’ became ‘Law, Safety, and Justice’ • Words like “Development” and “Community” repeated in different terms were eliminated • Identified additional ‘See Also’ association City of St. Louis, Missouri

28. Topics.CV Before Usability Testing Business & Industry Business Assistance, Incentives and Promotion Business Regulation and Licensing Business Taxation Labor Market Facts & Figures Local Economy Small Business Workforce Training and Development Community Births, Deaths, Weddings, and other Life Events Beautification and Nuisance Reduction Community Facilities Facts, Figures & Community Spirit History Neighborhoods Neighborhood Associations Neighborhood Programs & Events Social and Support Groups Volunteering Non-Governmental Organizations Immigration and Nationality Education & Skills Training Adult and Community Education Further and Higher education Homeschooling Public and Private Schools Special Educational Needs and Additional Support Employment, Jobs & Careers Employment Rights and Regulations Local Government Employment Opportunities Self employment Unemployment and jobseeking Youth Employment Environment Environmental Protection Hazardous Materials Waste Management & Recycling Climate Government Local Government Records Local Governmental Units Complaints and Service Requests Elections & Voting Elected Officials Federal and State Government Services Fees & Payment Information Local Government Services Permits, Inspections & Certifications Policy Making Procurement Taxation & Revenue Wards and Neighborhoods Account & Annual Budget & Operating Plan Employees Health Animal Control Immunizations and Public Health Food Safety Nutrition and Preventative Care Housing & Community Development Home Ownership Home Rental Housing Financial Programs Home Repairs and Renovation Relocation Temporary Housing Homelessness Leisure & Culture Arts and Entertainment Events Children's and Teens Activities Libraries, Museums & Cultural Venues Gambling and Lotteries Parks and Gardens Pets and Animals Shopping & Dining Sports and Recreation Tourism Planning & Urban Development Land Acquisition Preservation Planning Property Assessment Zoning Public Safety Police Fire Dept. Ambulance and Emergency Services Building Inspections & Permits Courts and Justice System Legal Assistance Reporting & Operation Planning Transport & infrastructure Airports Construction, Maintenance & Improvements Electric, Gas, Sewer & Water Utilities Telephone, Cable TV & Internet Utilities Motor Vehicles Public Transportation Parking Rail and Water Transportation Streets, Sidewalks & Streetlighting City of St. Louis, Missouri

29. Finalized Topics.CV After Usability Testing Business and Industry Business Assistance, Incentives, and Promotion Business Regulation and Licensing Business Taxation Labor Market Facts and Figures Local Economy Small Business Workforce Training and Development Community Births, Deaths, Weddings, and other Life Events Beautification and Nuisance Reduction Community Facilities Facts and Figures History Neighborhoods Neighborhood Associations Neighborhood Programs and Events Social and Support Groups Volunteering Non-Governmental Organizations Immigration and Nationality Education and Skills Training Adult and Community Education Further and Higher education Homeschooling Public and Private Schools Special Educational Needs and Additional Support Employment, Jobs, and Careers Employment Rights and Regulations Local Government Employment Opportunities Self employment Unemployment and Jobseeking Youth Employment Environment Environmental Protection Hazardous Materials Waste Management and Recycling Climate Government Local Government Records Local Government Offices, Agencies, and Departments Complaints and Service Requests Elections and Voting Elected Officials Federal and State Government Services Fees and Payment Information Local Government Services Permits, Inspections, and Certifications Policy Making Procurement Taxation and Revenue Wards and Neighborhoods Account and Annual Budget and Operating Plan Employees Health Animal Control Immunizations and Public Health Food Safety Nutrition and Preventative Care Housing Home Ownership Home Rental Housing Financial Programs Home Repairs and Renovation Relocation Temporary Housing Homelessness Law, Safety, and Justice Police Fire Department Ambulance and Emergency Services Courts and Justice System Legal Assistance Tourism Leisure and Culture Arts and Entertainment Events Children and Teens Activities Libraries, Museums, and Cultural Venues Gambling and Lotteries Parks and Gardens Pets and Animals Shopping and Dining Sports and Recreation Transportation, Infrastructure, and Utilities Airports Construction, Maintenance, and Improvements Electric, Gas, Sewer, and Water Utilities Telephone, Cable TV, and Internet Utilities Motor Vehicles Public Transportation Parking Rail and Water Transportation Streets, Sidewalks, and Streetlighting Urban Development and Planning Land Acquisition, Reutilization, and Development Preservation Planning Zoning City of St. Louis, Missouri

30. Process Summary Review and filtering of all relevant documentation (A to Z classification, International Best Practices, search terms and frequency, use scenarios, requirements, etc.) = Draft Information Architecture Document + Sample Tagging of pages + Wireframes =Proposed Information Architecture Volume 1.0 + Taxonomy Testing =Adopted Information Architecture Volume 1.0 City of St. Louis, Missouri

31. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

32. Wireframe Tagging of Page Spotlights and other sidebars can be manually filled at page creation/edit time. Alternatively, they can be dynamically filled with queries like: SELECT (Title, URL, Description, Thumbnail) WHERE Topic = AROUND (”Planning & Urban Development/Preservation”, “Community/History) AND Thumbnail not null. Page Index with Metadata for content placement Tagging of Page: Type=Group Home Pages Org=Planning & Urban Design Agency / Cultural Resources Office Topic=“Planning & Urban Development/Preservation”, “Community/History” Coverage.Wards=“MANY_WARDS” Coverage.Neighborhoods=“City Wide” Populated by querying calendar system with Org =“Cultural Resources Office” OR Topic= ”Planning & Urban Development/Preservation” OR Topic=“Community/History” WHERE Date > Today ASCENDING ORDER BY Date Office contact info dynamically populated from contacts database WHERE Org = Planning&UrbanDesign Agency/CulturalResourcesOffice Related folders box dynamically populated from query AROUND(“Planning & Urban Development/Preservation”, “Community/History”). Same as See also references City of St. Louis, Missouri

33. Taxonomy & Applications Integration Challenge: How to leverage Information Architecture for outside applications (120+)? Solution: • Create homepage in CMS • Login Permissions managed by CMS • Re-coded to have one single page output • Select Metadata and taxonomy during page creation Example:Gatewood Cemetery • Application data outside of CMS tables • Permissions set inside CMS • Display page is tagged using CMS metadata. • Topic relationships visible in “Browse by Topic” and “ See Also…” City of St. Louis, Missouri

34. External Application Integrated in CMS using Metadata and Taxonomy Tagged Content Page Topic association with other facets Browse by, See Also… City of St. Louis, Missouri

35. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

36. Governance • Information Architecture Governance Team (IAGT) created to institute a process for changes. • 9 members, representatives of major sections of government (Public Safety, Community Services, Parks, Health) and Web Team Manager as Lead • IAGT Responsible for Policies Governing Process Change. Policy Documents to-date: • Process Guide • Editorial Rules • IAGT Responsible for Testing and Validation City of St. Louis, Missouri

37. Agenda • Background • Research and Review of practices • Information Architecture Expertise • Metadata and Taxonomy Development Process • Taxonomy Testing • Implementation • Governance • Conclusions City of St. Louis, Missouri

38. Conclusions • Goal to standardize Metadata and Taxonomy for Municipal Government proved difficult. • Taxonomy Testing was critical • Testing with user groups forced customization of taxonomy that reflected local flavor • Process helped identified areas of missing content and classification. (i.e. Included audience facet for human services type classifications) • Benefits Created: • Strengthened Relationships between content • Extensible to other section of city government • Improved ability to find information and search • More discreet filtering • Established process for future site/content growth. City of St. Louis, Missouri

39. Thank you! For more information contact: Sonya Pelli Internet Services Manager Information Technology Services Agency City of St. Louis, Missouri pellis@stlouiscity.com http://stlouis.missouri.org 314 622-3400 x258 City of St. Louis, Missouri