USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS USING ANALYTICAL HIERARCHY PROCESS

1. USABILITY ASSESSMENT FRAMEWORK FOR CODE GENERATORS USING ANALYTICAL HIERARCHY PROCESS Joshua R. Dolecal Dahai Liu Remzi Seker Andrew Kornecki Department of Human Factors and Systems Embry-Riddle Aeronautical University Daytona Beach, Florida

2. Outline • Usability • Code Generation • Rationale • Framework • AHP • Results

3. Usability • Also called: Usability Engineering, Human Factors, Ergonomics, and User Centered Design. • UCD is defined as “the practice of designing products so that users can perform required use, operation, service, and supportive tasks with a minimum of stress and maximum of efficiency. (Woodson,1981) • Nielson (1993) defines Usability as the overall acceptability of a system.

4. Usability Criteria • Learnability • How easy something is to learn • Efficiency • Of the product (how well it aids users in performing tasks) • Memorability • How easy it is to remember skills learned/used • Errors • Error prevention, correction, and identification • Satisfaction • How well users are satisfied with the product

5. Process of Generating Code • Select template • Create a model/chart/diagram • Select output language (C/C++, Java, Ada…) • Generate Code (Stephens, 2002)

6. Rationale • ACG is relatively new • ACGs will change the way companies work and SW is developed • Usability need to be addressed, for special users • Experts (Computer Programmers) • Little research done

7. Framework

8. Framework • General Usability Criteria • (Nielson, 1993) Basic well accepted usability • Specific Usability Criteria • (Stephens, 2002 & Whalen, and Heimdahl, 1998 & Maclay, 2000, etc.) • Elements that are unique to ACGs according to the current literature

9. Method • AHP framework constructed • Usability criteria and • ACG functional analysis • 8 Graduate students gave pairwise comparison. (~6 months exposure with ACGs) • AHP weight • Consistency

10. Analytical Hierarchy Process • 3 Main steps • Establish a hierarchy • Calculate pair-wise comparison weights • Check for consistency • Apply weights to gathered data (objective/subjective) • Process can be Automated using Expert Choice • http://www.expertchoice.com/

12. AHP Values Thomas Saaty (1982)

13. Thomas Saaty (1982)

14. 4 Step Consistency Process • Thomas Saaty (1982)

15. Consistency • When dealing with many criteria/variables, people are not going to be perfectly consistent. It is up to you to review their decisions with them such that the “decision maker” understand transitivity. • E.g. If A > B and B > C then A > C

16. Results

17. Results

18. Results

19. Results • Semantic consistency and User satisfaction received the highest weights, but traversing up the hierarchy we see the Translator and then Specific Usability Criteria received the highest weight • A customized answer was produced • Best product for your situation • Does not label products as better than others • Allows a combination of subjective & objective data

20. Questions, Comments?