Rethinking the Competency Portfolio System

1. Will: • Lord of PHP • Supreme Chancellor of SQL • Frances • “The Hammer” • Goddess of CSS • Prescriber of Will’s Paxil • Jake: • Ambassador of Human Empathy • Technophobia Minister Rethinking the Competency Portfolio System

2. Olin Competency Portfolios(or how good intentions went astray) • Demonstrate student competencies • Allow students to showcase their work to employers and graduate schools • Help intra-Olin communication of “cool stuff” • Knowledge management Rethinking Competency Portfolios

3. The reality • Students don’t like it. • Must enter each entry 3 times • All entries look the same, ugly • Students don’t contribute to it. • Enforcement impossible. Rethinking Competency Portfolios

4. Our Goals • Empower students. • To make the portfolio system easy to use. • Motivating students is essential to make the system easy to use. • Assign grades. (boo) • Make it fun enough for students to do it voluntary/Make it a procrastination diversion. • Make it valuable to the students. Rethinking Competency Portfolios

5. Surprisingly, we turned to personasfor guidance • Jenny Shannon –bread and butter Olinite • Apathetic about her portfolio • No HTML skills, no desire to learn • bioE Often she’s sober Rethinking Competency Portfolios

6. 2nd Persona • Liam Doyle – Power User • Wants to use portfolio to display l337 H4X0R skills • WoW fiend, level 52 warlock Rethinking Competency Portfolios

7. 3rd Persona • Michael Huffman –junior professor • Wants to view advisee’s portfolios • Familiar with HTML and web • Not interested in taking time to explain to less e-competent colleagues • Single and searching Rethinking Competency Portfolios

8. 4th persona • Frank Lin – backbone of Olin’s administration • Wants to view and present aggregate portfolio data • Wants to create user accounts and access for ABET, others • Nickname in college lacrosse: ‘whack-a-mole’ Rethinking Competency Portfolios

9. Personas do things in ‘scenarios’ • Jenny • Login, create an entry, attach a file, make it public • Liam • Editing existing entries, more complex formatting and privacy options • Michael • Searching the portfolio library by class, subject, advisees, etc. • Frank • Viewing aggregate data, creating accounts Rethinking Competency Portfolios

10. What we learned:It’s the wrong interaction Old Portfolio System New Portfolio System Rethinking Competency Portfolios

11. The Manila Folder • Bureaucratic • Minimalism conveyed a lack of concern for quality • Either the same as everyone else, or too much effort • Dump your files here. Rethinking Competency Portfolios

12. The book model(It’s all about ownership) • Students are the authors of their portfolios. • They can change content and the layout. • They pick the cover. • They set the tone. Rethinking Competency Portfolios

13. How can portfolios be integrated into the lives of Olin Students? • Provide varying levels of commitmentand effort(should take as little or as much time as you want) • As little as possible for some • Quick and easy to begin • Use tools students already know • Take as little time as possible • Powerful and flexible for others • Customizable visual appearance, layout • Make it a valuable as a marketing tool Rethinking Competency Portfolios

14. Implementing low activation energy • One click to make a new entry. • Default full layouts provided.(Pick one of five flavors to start from) • Fill out as much or as little you want. Rethinking Competency Portfolios

15. Implementing Customization • Easy initial customization – pick favorite default layout template • CSS/HTML controlled • First dabbling is changing colors • Change layout properties • Write your own tags. • Hack the code with <!-- --> marks. Rethinking Competency Portfolios

16. Implementing Marketing Value • Make portfolios polished enough that students will want to show them employers. • Default templates will be professional and functional. • Give portfolios personality (not clones) Rethinking Competency Portfolios

17. Implementing More Value - Managing Access • Externally accessible! • Manage access • Students don’t want employers to see everything. • Make it easy for external people to use • Individual Generic access accounts -> access one portfolio (put on resume) • Specific access account -> access multiple portfolios Rethinking Competency Portfolios

18. Unimplemented Features • Searching • Faculty Pages • Basic Layout Designed • Staff Pages • Identical to Faculty Layout but no advisee links • Guest Users • Annual Reports • CV and Resume Uploading Rethinking Competency Portfolios

19. Implemented Interaction Flow Rethinking Competency Portfolios

20. Responding to Lo-Fi prototype • Novel browsing technique: • Scenarios didn’t match personas well • Profs care about advisees and aggregate data, not own courses • Manually filling out CV a bad idea • Preview pane unclear • Search functionality unclear Rethinking Competency Portfolios

21. Responding To Heuristic • Created separate Profile Manager • Added Google-style quick searching • Feedback Added When • Updating or Deleting Entries • Uploading Files • Warning for unsaved changes to entries • Awkward Phrasing/Order for Mass-Editing • Previous/Next Entry Links include the title of the entry • Consistent Use of Buttons for Style Sheet controls Rethinking Competency Portfolios

22. Responding To Usability Study • Result of Tests: • Mass-editing organized by functionality and creation of a “style manager” page to add symmetry • Related Corrections: • Lack of User Feedback  Added Messages • Highlight entries that were changed in mass-edits + display message on success • Index Page Had Too Much Content at the Top • Added “Welcome” on first login, but removed all large-font static messages • Users attempt to use the links in the preview plane and navigated away from page • Removed links from toolbar shown in preview plane Rethinking Competency Portfolios

23. DEMO!

24. Lessons Learned • Limit Information Overload • When interacting with users, be prepared for anything. • Don’t be ego involved in the design – controlled apathy is your friend (see Jake) • The system should provide more feedback to the user than you think it should Rethinking Competency Portfolios

25. Questions? Rethinking Competency Portfolios