TECHNOLOGY ADVANCEMENT AND OUTREACH

TECHNOLOGY ADVANCEMENT AND OUTREACH Hosted byDrew Bittner August 19, 2010 Room 5E-081 Call In # 301-903-6183 Web Coordinators Meeting

Around the Room (Drew) – 15 min. ITP User-Centered Design (Wendy) – 30 min. Refacing Update – 10 min Progress Alert Subscribers (Chris) – 5 min. Cookies (Chris) – 5 min. Recovery Act (Drew) – 5 min. Communications Standards (Elizabeth) – 5 min. Agenda

Agenda ITP Usability Study • Project background • Program goals • User tasks • Usability study results • Next steps

Usable Web sites are: ITP Usability Study Effective Easy to Learn Efficient Error Tolerant Engaging

A successful site helps ITP accomplish its program goals by helping users accomplish their personal goals. ITP Usability Study ITP Web Site Users “ITP's mission is to have U.S. industry lead the world in energy efficiency and productivity.” We really needed to understand how much energy our plant is using; I wonder if ITP can help us with an energy audit? Oh terrific – they do offer assessments for plants our size – all I have to do is fill out this application. And it looks like they offer some trainings as well – I’ll definitely check that out… • Increase number of assessment applications • Increase participation in training events Accomplish program goals Positive userexperience Users’ needs and goals

Establishing who is visiting the site and their tasks ITP Usability Study Stakeholder input Listserv / Webmaster feedback Previous usability studies Phone survey User research Web survey Web statistics/logs Interviews with EERE Info Center

ITP Usability Study Manufacturers Consultants Researchers User profiles

ITP Usability Study Purpose of the usability study • Identify how well the current site meets the needs of manufacturers, consultants, and researchers. • Identify major problems users may experience when visiting the site. • Establish baseline data for measuring progress in making the site easier to use. • Provide the basis for redesigning and improving the site.

ITP Usability Study A usability study is a research activity in which: • Plant/energy mgrs 6 • Energy consultants 5 • Researchers 6 • Total 17 • Variety of industry sectors • Newbies to veterans • Range of age • Both genders • Range of familiarity with ITP • BCS • Energetics • EES • Anthro-Tech • Program reviewers for study materials • Such as: • Time per task • Pathways • Requests for help or hints from the facilitator • Users’ verbal comments • Task completion • Collected with: • Session recordings • Qualitative/quantitative notes • The study was performed remotely, using UserVue and GoTo meeting. • Advantages: • Users work in their own environment • Interact with a national audience • Based on user research: • Statistics and Web logs • Survey • Information Center interviews • Listserv feedback • Such as: • Find out if a plant qualifies for an energy audit • Find ideas for improving steam efficiency • Find out about training opportunities • Find funding opportunities Representative users Perform realistic tasks A research team Collects empirical data In users’ own environments

ITP Usability Study Key study findings 1. Users see great value in ITP and its Web site  2. Users struggle to successfully complete key tasks on the site  3. The home page does not help users discover the breadth of information available   4. The organization of the site does not match user expectations  5. Content and terminology of the site does not match user expectations  6. Search fails to provide users with helpful or relevant information

ITP Usability Study  Finding 1: Users see great value in ITP and its Web site I didn't know…that there were webinars and trainings…I didn't know about all of the tools that were available either. I'll definitely be going to the site in the future.” —PM4 “There's a lot of information on there. It seems to cover lots of topics .” —PC5 You could effectively set up a program within your facility and use the website as a very good guideline to implement the program.” —PM5 • Users are surprised and pleased by the information they find on the site. • Participants view ITP as trustworthy, comprehensive, and reliable.

Finding 2: Users struggle to successfully complete key tasks on the site ITP Usability Study  • Overall success is defined by a user’s ability to successfully find information: • within one or two attempts • with no hints from the study facilitator • in under five minutes.

ITP Usability Study Finding 2: Users struggle to successfully complete key tasks on the site (video 1:28 min)  PM-4

ITP Usability Study Finding 3: The home page does not help users discover the breadth of information available  • Almost all users returned home at least once. • Users do not understand the 3 major categories. • Users missed helpful task-oriented links. • Users ignored or didn’t like the rotating feature. visual fold “I keep going back to the three at the top: Industrial Technologies, Save Energy Now, and Technology Delivery. But I don't think they are what I'm supposed to be looking at.”—PM2

ITP Usability Study Finding 4: The organization of the site does not match user expectations  • The structure of the site does not map to users’ mental models: • The place users look is not where the information lives. • Users employ a trial and error approach to navigation. • Users navigate in circles. • Users are confused by too many paths to the same information. • Users lose confidence when the same content is posted in multiple locations.

ITP Usability Study When looking for: Funding & incentives • Users were drawn to: • Financial Opportunities • But the info is located: • Program Areas (State • Incentives & Resources • subsite) Contact information • Program Areas > Steel • Information Resources • EERE Information Center • About the Program • About Steel IOF Energy-Saving Assessments • Industrial Assessment Ctrs • Program Areas • Financial Opportunities • Deployment (Save • Energy Now subsite) Info on the LEADER program • Financial Opportunities • Energy-saving assessments • Deployment (Save • Energy Now subsite)

ITP Usability Study Finding 4: The organization of the site does not match user expectations (video 1:33 min)  PM-6

Finding 5: Content and terminology of the site does not match user expectations ITP Usability Study  “I wouldn't know what to find under certain headers or certain links; I'd click something with one thing in the mind, and it would turn out to be something else.” —PM6 “Looking at Technology Delivery, I’m not exactly sure what that means.” —PM1 Users are confused by labels that are: • Program-centered or jargony • Overly general Deployment Technology Delivery Distributed Energy Combined Heat & Power Industrial Technology R&D Information Resources

Finding 5: Content and terminology of the site does not match user expectations ITP Usability Study  • Users are overwhelmed by the amount of content. • Content for a particular audience isn’t marked – this confuses other audiences. • Participants incorrectly assume they completed tasks correctly. • Some users will search through PDFs, but many won’t. • Users react negatively to content that is old. “I mean, I feel like there's a lot of stuff in here. It just seems kind of hard to find because there‘s so much.” —PM6 “But all these documents, fact sheets about these technologies, these are old documents, and we need the LATEST information about these technologies.” —PR4

ITP Usability Study  Finding 5: Content and terminology of the site does not match user expectations (video 1:25 min) PR-6

ITP Usability Study  Finding 6: Search fails to provide users with helpful or relevant information • Users turn to search when navigation efforts fail. • Number of results returned is overwhelming. • Few visit the second page of search results. • The search returns primarily PDFs instead of HTML pages. • “At this point I would start right at the Search and type in ‘Steel’ and see what happens. I ended up with far more information than I need.” —PC2 • “Maybe what I'll do is get rid of ‘funding opportunities’, because I’m getting a lot of clutter.” —PR1

ITP Usability Study  Finding 6: Search fails to provide users with helpful or relevant information (video 1:56 min) PR-4

Conclusion ITP Usability Study • The ITP site is not meeting the needs of the manufacturing, consulting, and research audiences. • As a result, users: • Do not find the services or information they are looking for • State they would need to get the information through other channels • Draw incorrect conclusions about the information or services they qualify for. • ITP is losing a valuable opportunity to help users discover valuable information and services.

Recommendations ITP Usability Study • Redesign the Web site so users can accomplish their goals and tasks efficiently and easily. • Provide global navigation and content that matches users’ language; avoid jargon. • Organize the site according to users’ mental models, not ITP’s organizational structure. • Strive to make all content on the site useful, clear and easy to understand; layer content from general to specific. • Get rid of content that is old / seldom visited, or that doesn’t support user or program goals. • Improve search results by optimizing content for searchability, and have the search feature return Web content before PDFs.

ITP Usability Study Where we go from here Where we go from here Business analysisDefine goals and metrics Prioritize usersUCD plan and scheduleUser research Search logsWeb analytics Info Center Interviews Online survey Usability analysis Benchmark usability study User modelingUser profiles Task matrices Scenarios Content Audit Strategy Information architecture Card sort Build Test Revise DesignNavigation model Layout Interactions EvaluateScenario based walkthrough Prototype usability study Produce site Content creation and revisions Migrate content Functional testing Ongoing customer feedback / user research Measure success Benchmark usability study Complete In progress

ITP Usability Study Questions and Contact Information Wendy Littman wendy.littman@hq.doe.gov 301-525-7521

Areas of Interest (users can choose more than one) Progress Alert Subscribers As of July 28, 2010

Affiliation Progress Alert Subscribers As of July 28, 2010

WebContent.gov and Federal Web Metrics Group drafting boilerplate policy and implementation recommendations. Cookies

Recovery Act Funded Projects Recovery Act

Native file formats and PDFs Communications Standards

Wrap Up • Next Meeting Sept. 16

TECHNOLOGY ADVANCEMENT AND OUTREACH