340 likes | 496 Views
Quote. Success is getting what you want. Happiness is wanting what you get. Dale Carnegie. http://www.quotationspage.com/. Who are we?. Matt Sharpe (Document Manager) Peter Centgraf (Technical Lead) Lisa Edelman (Client Liaison) Lorrianne Nault (Project Manager)
E N D
Quote Success is getting what you want. Happiness is wanting what you get. Dale Carnegie http://www.quotationspage.com/
Who are we? Matt Sharpe (Document Manager) Peter Centgraf (Technical Lead) Lisa Edelman (Client Liaison) Lorrianne Nault (Project Manager) Adrian Tang (User Testing Lead)
What is CitiStat? • CitiStat Goals: • Ensure that City departments have the resources and infrastructure to deliver services • Provide operating savings • Set Benchmarks for comparison to industry standards • Focus resources on priority initiatives, such as Pittsburgh Clean Neighborhoods • CitiStat is a management data reporting process • Provides data to Mayor and committee for review • Charts, graphs, and maps are used for data analysis • Added importance due to 5-year budget plan, Act 47 compliance
How do we fit in? • Goal: To create a system that supports visualization and analysis of performance data, promotes efficient data entry, and enables more effective use of city resources. • The MHCI challenge • Create a fully-functional project that can be put into practice, even if in limited form • Give the City’s technology group something they can support, but not have to implement • Special Concerns • Ease of learning • Ease of use and efficiency • Sustainability of technology • Scalability • Delivery of fully-functional prototype A B C Raw Data Manage data, where it comes from Dynamic Reporting Visualization of mgt needs Final Reporting Presentation of value-add data Departments Management Public / Regulators
Where did we begin? • Determined a scope of work • Focused on Department of Public Works (DPW), Bureau of Operations • Investigated CitiStat process • Worked backwards along flow of information • Office of Management and Budget (OMB) to DPW • Contextual Inquiry and Interview techniques • Analyzed the most effective ways to improve it • Constructed diagrams to describe and confirm our understanding • Recorded key observations and their links to CitiStat goals
Work Department of Public Works Enter Data Foreman System Copy Data Excel template worksheets Update Copy to History File Print Look at data set, search for trends Analyze Changes, problems, useful findings Alert Tell OMB Staff History complete Graph Create graphs of relevant data Compose Briefing books to share data Report CitiStat meetings Improve Assist DPW with resources, etc How does the CitiStat process work?
What challenges did we find? • Redundancy (6) • Data points: CI1-14, CI1-2, CI2-12, CI9-5, CI8-1, CI4-14 • Shared Understanding (4) • Data points: CI6-1, CI1-1, CI6-2, CI7-25 • Effort (5) • Data points: CI1-13, CI9-2, CI8-2, CI9-6, CI4-14 • Coordination (7) • Data points: CI7-12, CI4-24, CI4-28, CI4-18, CI9-3, CI4-19, CI7-21 • Accuracy (4) • Data points: CI9-7, CI6-13, CI6-8, CI7-17 • Also identified main themes in the data and other key observations Breakdowns
Work Department of Public Works Enter Data Foreman System Copy Data Excel template worksheets Update Copy to History File Print Look at data set, search for trends Analyze Changes, problems, useful findings Alert Tell OMB Staff History complete Graph Create graphs of relevant data Compose Briefing books to share data Report CitiStat meetings Improve Assist DPW with resources, etc Work Department of Public Works Enter Data Foreman System & Input Interface Analyze Quick graphs and charts Create Briefing books to share data Report CitiStat meetings How are we changing it? • Input interface allows DPW employees to input the data directly into live database • Immediate access to the data by both DPW and OMB Analysts • Export Excel report, data tables, and graphs • Ability to quickly change the order of metrics, add new metrics, and remove metrics no longer needed • Comments and/or description for metrics
How does it support CitiStat? • Office of Management and Budget (OMB) • Eliminated manual conversion to different forms and formats • Visual data analysis based on automatic graphs and data tables • Flexibility to explore different cross-sections of data by year or division • Documentation for the meaning behind the numbers • Department of Public Works (DPW) • Some data could be imported from current tracking system • Ability to view data to prepare for CitiStat meetings • Can use data to inform internal decisions
How does it work? • Input interface allows DPW to input CitiStat data directly into the database • Change interface provides flexibility for the order of metrics and allows a user to add additional or remove pre-existing metrics • Analysis interface allows OMB and DPW staff to view overall trends and analyze the data Input Interface Access Database Analysis Interface Foreman System Change Interface
How does the change interface work? • Add or delete categories and metrics • Specify metric description, type, labeling style, and unit of measure • Manipulate the ‘metric tree’ as needed Add new items Search Name, Description and type Metric Tree Units Labels Links
How does the input interface work? • Find section to input in metric tree • Input data according to month and budget vs. actual data • Correct previous data as needed Attributes Search Value History Metric Tree Navigation
How does the metric tree work? • View graphs and data tables through navigation of the data metric tree • Search for metrics • Description of the metric provided (as entered in the change interface)
How does the graph area work? • View or Hide the data table • Copy data table or graph and paste in Word, Excel, etc • Adjust the X-axis and Y-axis • Customize the graph title
How do the graph option controls work? • Graph Data: Actual Value, Planned Value, % of Plan, and % Change by Month and Year • Graph Type: Line or Bar Graph • Compare by Years or Divisions • Exponentially expands the number of possible graphs and comparisons
How do the summary tables work? • When a category is selected, a summary data table is shown • Can copy and paste summary table
What other features does it support? • Export data table or graphs • Print graphs • Load / Save graphs as favorites • Export Excel report (comparable to current template)
What can be done in the future? • Graphs • Add trend-line to bar charts • Add a pie chart or other graph types • More flexibility on editing the graph and changing the display • Communication features • Add contact information in metric description • Provide functionality to email metric contacts about discrepancies or misunderstandings • Add notes section for specific metrics • General Improvements • User-test and improve input and change interface • Contextual help • Save screen real-estate by using collapsible pallets
Design Develop Test How did we get here? Research Contextual Inquiry Design Paper Prototypes Documentation & Deployment User Testing Heuristic Evaluation Development
Research Contextual Inquiry Design Paper Prototypes Documentation & Deployment User Testing Heuristic Evaluation Development How did we uncover context? • Learned more about city management • Investigated Baltimore, MD CitiStat program • Interviewed public works department of Wheeling, WV • Purchased CompStat Paradigm by Vincent Henry • Familiarized ourselves further with Pittsburgh • On-going review of Pittsburgh Post-Gazette • Explored more design-centered visualization techniques • Examined Enterprise Dashboard technology, persistent, on-going, interactive data displays • Samples from the gaming world • Data Visualization instructors from HCII at CMU
Research Contextual Inquiry Design Paper Prototypes Documentation & Deployment User Testing Heuristic Evaluation Development How did we discover and define problems? • Observation of Pittsburgh’s CitiStat program • Exposure and understanding of Pittsburgh’s culture • Hands-on experience with how the system currently worked • Contextual Inquiries and Interviews • Contextual Inquiry: An approach to answering the question, “What should we build to help CitiStat employees do their work even better?” • Gained a better understanding of tasks and data flow • Areas for improvement in current processes • Environmental and technology constraints • Having conversations with users in the context of their work creates richer data
Category Speed Flexibility Simplicity Pivot Mac Tree Tabs Excel + Average Not Strength Strength Research Contextual Inquiry Design Paper Prototypes Documentation & Deployment User Testing Heuristic Evaluation Development What did we do with that data?
Research Contextual Inquiry Design Paper Prototypes Documentation & Deployment User Testing Heuristic Evaluation Development How did we evaluate our designs? • Heuristic Evaluation: Small team of evaluators to test an interface based on recognized usability principles including • Visibility of system status 6. Recognition vs. recall • Match between system 7. Flexibility and efficiency of use and the real world 3. User control and freedom 8. Aesthetic and minimalist design 4. Consistency and standards 9. Error recovery 5. Error prevention 10. Help and documentation • Think-Aloud Usability Testing: Asks users to “think aloud” as they work • Discover specific problems with the interface from actual DPW or OMB users • Understand the way the users think about the problem in order to present information appropriately
Prototype Iterations Design Develop Test How did we utilize User Testing? Testing Goals Scenarios Results Sheet and UARs Consolidation Solution Brainstorming
What did it take to get here? • 12 contextual inquiries / interviews • 29 user tests • 10,782 lines of code • 461 presentation slides • 363 emails • 11 computer failures (Matt) • 2 lost or injured USB drives • 500+ cups of coffee • 1,960 crackers
What are some future directions? • Useful improvements • Automatic unit conversions • System Integration • Pull data directly from other data sources • History • Login functionality to keep track of individual’s changes • Apply this tool for the other departments within Pittsburgh
Who has supported us along the way? • We would like to thank: • Joanne Foerster • Pittsburgh Department of Public Works • Pittsburgh City Information Systems • Pittsburgh Office of Management and Budget • Bob Kraut • Carolyn Rose • Bonnie John • MHCI class, especially our critique team • And all of you for attending our presentation