PROTEA: A Process for procuring accessible software

1. PROTEA: A Process for procuring accessible software Dr Richard Walker E-Learning Development Team Manager & VLE Service Group Leader University of York Professor Helen Petrie Professor of Human Computer Interaction University of York

2. About THE University of York • Ranked #7 in The Times Higher Education 100 Under 50 world ranked universities (2013) • Home to more than 15,000 students and 3,000 staff • Blackboard users since 2005 • Currently on Blackboard Learn v9.1 Service Pack 12 (licensing community, content and learning management system and mobile learning building block)

3. About THE HCI Research Group at York • One of the oldest research groups studying human-computer interaction in the UK, if not the world • Founded in 1984 – centred in Computer Science, but with members in Archaeology, Electronics, Health Sciences, History, Management, Psychology, Sociology and Theatre Film TV • Key values – putting humans at the centre of human-computer interaction • Particular interest in people with disabilities, older people, • e-learning

4. Background – from the e-Learning DEVELOPMENT TEAM • Have a strong commitment to accessibility and usability • Institutional staff and student VLE surveys have highlighted usability issues, for example: • In designing modules • Uploading content • EARL reading list application • Tools felt to be “clunky”

5. Background – from the e-Learning DEVELOPMENT TEAM • Have worked with the HCI Group on two occasions: • On upgrade from v8 of Blackboard Academic Suite to the next generation release 9.1 Service Pack 5  (in 2011) • To review version 9.1 Service Pack 12 – as an additional action to the VLE retender (see below) • We report our findings to the University’s E-Accessibility Forum and use the information to inform our training and support to system users

6. BACKGROUND – FROM THE HCI RESEARCH GROUP • We have done many evaluations of accessibility and usability of websites and web-based systems, in particular: • A formal investigation of websites for the Disability Rights Commission of Great Britain • We evaluated 100 websites with 50 people with disabilities, 10 people per website • Asked them to conduct typical tasks on each websites • We found that if we evaluated with blind, partially sighted and dyslexic participants we found 80% of accessibility problems

7. OUR CHALLENGE • We wanted to support the university in providing more accessible software, for both students and staff • One of the difficulties is in procuring new software – how do we know it will be accessible, most software does not come with any details of accessibility • If it does, it is usually a statement of conformance to guidelines, perhaps Web Content Accessibility Guidelines (WCAG) – is this sufficient?

8. OUR CHALLENGE • Two problems: • WCAG guidelines are not relevant to all software and quite hard to apply to very complex systems • Guidelines such as WCAG do not tell you definitely that students/staff with disabilities will be able to use the software, they are not precise enough, you need testing with users

9. Our solution:PROTEA: Protocol for testing e-aCcessibility • So we created a protocol for user testing of software: • Can be applied to any kind of software, to be used by students or staff • Can be conducted by our group, other groups in the university or by the software provider (who would provide evidence that it has been followed)

10. Our solution:PROTEA: Protocol for testing e-accessibility • Need to evaluate the software with 3 blind people, 3 partially sighted people and 3 people with dyslexia, as close as possible to the real users of the software being evaluated • The blind and partially sighted people should use a mix of different assistive technologies to access the software • Blind participants – JAWS, WindowEyes, VoiceOver, NVDA • Partially sighted participants – SuperNOVA, ZoomText • Dyslexia participants – may change screen configuration

11. Our solution:PROTEA: Protocol for testing e-acCessibility • Need to pick a series of 3 - 5 “typical tasks” for the participants to do: • “typical task” means something quite “meaty” • Send an email to your tutor asking for a meeting • Upload an essay • Find the reference list for your archaeology course next term • Start with easy tasks, work up to harder ones, perhaps weave into a story

12. Our solution:PROTEA: Protocol for testing e-accessibility • A facilitator sits with the participant as they go through the tasks, to assist if needed • If possible, sessions should be recorded for later analysis and discussion • Participant talks through what they are doing (“think aloud” or “verbal protocol”)

13. Our solution:PROTEA: Protocol for testing e-accessibility • If the participant finds a problem, they pause and rate it on a scale 1 (= cosmetic, small annoyance) to 4 (= major, I’m really stuck) • If the participant gets really stuck (e.g. the screen reader cannot interpret the page), the facilitator helps move the participant on • BUT participant should not be “lead” through the task – not told where to click or the particular names of button, links etc

14. Our solution:PROTEA: Protocol for testing e-accessibility • Occasionally might have some of the development team sit in • BUT only if the participant is comfortable with that and will not feel intimidated • Gather all the problems encountered, analyse for which groups of users/assistive technologies are being affected

15. Our Results • We applied PROTEA as part of the recent procurement process for the VLE at York • We did find some unexpected accessibility problems that we fed back to Blackboard • We also highlighted some issues where we need to educate our staff members who develop content to ensure accessibility

16. Our Results • Issue 1: Assistive Technology unable to give access to functionality • Problem: screen reader users could not interact fully with the Content Editor • Consequences: could not send emails, contribute to fora, answer quiz questions

17. Our Results • Issue 2: Navigation within software • Problem: Moving between frames was difficult and inconsistent for screen reader users • Consequences: users could not navigate to where they wanted to be, became “trapped” in frames

18. Our Results • Issue 3: Auditory information needs different handling to visual information • Problem: In character limited text boxes, screen readers announced the character count after every keypress

19. OUTCOMES AND RECOMMENDATIONS • Blackboard have been very responsive in investigating these issues • They are the kinds of problems which will not necessarily be picked up by testing to guidelines • Need really realistic evaluation with users and a range of assistive technologies

20. OUTCOMES AND RECOMMENDATIONS • Other useful outcomes of using this kind of evaluation: • VLE support staff become much more aware of problems that students with disabilities may face • Can create better support materials, training for students/staff who are creating content

21. OUTCOMES AND RECOMMENDATIONS • We have used the PROTEA method not only with Blackboard, but with several systems being procured by the university and created in-house • Both systems for students and for staff • We have also worked with the UK government (on DirectGov) and a major bank

22. Thank you! • If you would like more information, we will be setting up a website in the next weeks: • www.yorkhci.org/protea • We are happy to provide general advice for free and of course can undertake evaluations • Helen.Petrie@york.ac.uk