Automated Evaluation of WordPress Theme Design

1. Automated Evaluation of WordPress Theme Design Derek Ohanesian CSC 499 Advisor: Prof. Chris Fernandes November 7, 2014

2. Research Question In what ways can an automated system provide web design advice? Can an automated system analyze a blog’s WordPress theme and provide valuable feedback about design improvements?

3. Motivation The World Wide Web democratizes publishing A well designed websiteis lessaccessible.

4. What is design? By design, we mean the user interface, encompassing the presentation, aesthetics, and accessibility of the website. Design is separate from the content of the website.

5. Why Blogs? • Limits scope • Similar content, differentiated by design. • Focus on evaluating the design, while the content type remains constant • Limited data set the two-column style blogs only

6. What is WordPress? • An open-source content management system (CMS) for blogging. The most popular CMS for websites. • Separates the content from the design. Different WordPress “themes” can be applied to the blog content to create the design of the website. • Thousands of freely available and open-source themes available.

10. Research Question How can we use data collected from WordPress themes to provide valuable feedback about design improvements?

11. Data Collection • WordPressthemes include a set of functions that the theme designer can call to access the blog’s content and present it in the design of the page • I intercept these function calls and inject labels to each section of the website, so I know what content is being displayed

13. Title Tagline Post titles Post Title Post

14. Data Collection • By injecting labels into the WordPress theme, I was able to collect more precise data about the content being displayed. • In addition, general data about… • position of elements • fonts • colors • sizes • visual style

15. Procedure • Previous research shows that certain quantitative heuristics correlate with good design • For this project, I tested similar heuristics on a data set of over 200 two-column WordPress themes

16. Heuristics • Quantity of unique colors used • Text contrast on background • Left/right balance of major elements • Density of content elements (percent of area) • Font size for some elements • Quantity of fonts used

17. Number of Colors Number Of Themes Number of Colors Used

18. Number of Colors Number Of Themes One theme with 38 colors Number of Colors Used

19. Color Scheme

20. 5 colors

21. 17 Colors

22. Number of Colors • “Sweet spot” at 5 colors? • Greater than 10-15 colors indicative of a problem?

23. Layout Heuristics • Area of page used (density) • Balance (left/right)

24. Area of Page (~2% content)

25. Area of Page (~47%)

26. Area of Page (~47%)

27. Balance

28. Balance 100% (of content on left side)

29. Balance 50% (of content on left side)

30. Text Contrast • WC3 recommendation for making web more accessible • Web Content Accessibility Guidelines (WCAG) • Recommended color contrast algorithm must produce contrast within a certain range to be considered “good visibility”

31. Text Contrast 100% of color combinations pass WCAG test

32. Text Contrast 0% of color combinations pass WCAG test

33. Text Contrast • Inconsistent data • Some color combinations are not included in my analysis (hard to parse CSS, or alpha channels) • WCAG 2.0 has an improved color-contrasted algorithm

34. Evaluating Heuristics • Heuristic provide a set of scores for each theme • A test set of unevaluated themes has been set aside • A human evaluation of the themes in the test set can be compared to the automated evaluation to evaluate its effectiveness

35. Future work • Evaluating automated scores against human heuristic evaluation (this term) • Assigning weights to heuristics • Scoring themes with additional heuristics • Adding additional themes to data set

36. Q & A