Download
1 / 38
380 likes | 558 Views
Usability Findings: JIRA Recommendations. September 4, 2009 Author: Ashutosh Kaushik. Presentation Overview. Study Overview Reporting Usability Test Methods Reporting Tasks Executive Summary Priority Scale Findings/Recommendations. Reporting Usability Test : Study Overview.
E N D
Usability Findings: JIRA Recommendations September 4, 2009 Author: Ashutosh Kaushik
Presentation Overview • Study Overview • Reporting Usability Test Methods • Reporting Tasks • Executive Summary • Priority Scale • Findings/Recommendations
Study Overview - Testing Methods • Testing Tools • Production Version of Jira Software • Imported AM issues from Bugzilla • Goals • To assess the usability of the defect management user interface of JIRA and to identify any issues that may impact the user experience of the application. • To provide actionable recommendations that can be implemented prior to the organization-wide rollout of JIRA. • Methodology • Aggressive 1 week timeframe • In-person cube study, an informal user test performed at the participant’s desk rather than in a usability lab. • Testing session length = approximately 30 minutes • 10 internal Participants • Cross-functional Bugzilla users
Reporting Functionality Tasks • Background: During a recent review of the UI of the latest code drop of Europa, you noticed certain discrepancies between Europa and Ganymede Group 1 • Task 1: Create a new issue in JIRA • You’ve determined that the Ganymede implementation is correct. You regard this as UXD issue and want to assign the issue to Brian Manning. You also think it would be beneficial for Brian to look at issue and thus you want to attach one of the screen shots. Assuming you had the files on your desktop; show me how you’d do this. Expectations: Determine if labeling and values make sense. Watch for any validation issues (e.g. missing required fields). Do the participants explore the other tabs in JIRA? This would also help us determine whether the user makes use of the numerous tab functionalities while creating issues or makes edits after he has created the issue.
Reporting Functionality Tasks • Task 2: Edit an existing bug • Task 2.1: You’ve been asked to provide a better screenshot of the issues and remove this one as it was confusing for the assignee. Go ahead and remove the previous screen shot and add the new from the files you were sent in the meeting invite. • Task 2.2: If you wanted to view the modifications to this defect since it was opened, how would you do that? Expectations: Determine how the user goes ahead and makes the edits. With the file attachments does the user get confused by the two options of screenshots or file attachment. Does the user understand the procedure for removing an attachment? Can the user easily find common tasks like “Change History”? Group 2 • Task 1: Search for an issue in JIRA • After some investigation, you’ve determined that the Europa implementation is correct, although someone believes there is a bug logged against this stating the opposite. Show me how you would find this issue in JIRA. Expectations: Determine the search functionality and ease for the user to find related issues
Reporting Functionality Tasks • Task 2: Close an issue in JIRA • Now that you’ve found the bug, you’d like to add a comment and close the issue. Additionally, you’d like to be notified should anyone else make changes to this issue. Show me how you would do this. Expectations: Determine the functionality of editing an already existing issue, test features like Watch It and tabs functionality like comments.
Executive Summary • Strengths • Most of the participants had a positive perception of the application, although many noted the differences vs. Bugzilla • Most of the participants were able to easily create new issues • The thumbnail of the screenshot was appreciated • Many users found the dashboard shortcuts useful • Areas of Improvements • The process of searching an issue proved difficult for some users as many of them required prompting to complete the task • Many Participants did not expect to find actions (workflow and operations) in the left navigation menu • Most participants had trouble finding the top menu due to color contrast issues • Most participants expressed frustration in these areas: - Attaching screenshot image file using the Attach Screenshot operation - Narrowing down the search results - Managing file and image attachments - Using the “View” and “View & Hide” buttons on the Filters screen
Severity Scale Finding Severity Scale • Major = Blocking, results in inability to complete tasks and user frustration – addressing may be necessary to avoid user dissatisfaction. • Moderate = Major Irritation; results in extra clicks, long pauses and uncertainty – should be addressed; addressing would greatly improve user experience. • Minor = Minor irritation, results in short pauses and momentary uncertainty – nice to have; addressing would improve user experience.
Findings/Recommendations Finding: • Most participants while creating an issue expected a separate field for Steps to Reproduce. • Some were afraid that by having steps to reproduce in the description field it might get lost as that field can get verbose Severity: Moderate Recommendation: • Add a separate Steps to Reproduce field while creating an issue. This will help the users to reproduce the bug more easily
Findings/Recommendations Finding: • Many participants were confused about what to put in Found in Build field • Some assumed it was required and ended up putting in wrong information Severity: Minor Recommendation: • Specify/clarify during training what is expected to be entered for this field. • Provide a note below the field stating: (if applicable).
Findings/Recommendations Finding: • When asked to assign the issue many participants looked for the functionality under Edit. • Some users expected to assign the issue on the creation screen Severity: Minor Recommendation: • Place the Assignee option on the create issue page (Pending final decision on bug triage workflow)
Findings/Recommendations Finding: The tabs while creating an issue were not noticed by many of the participants. Severity: Minor Recommendation: • Make these links look more like tabs • Consider duplicating tabs at bottom of form • A reminder at the bottom stating to use the tabs to add additional information. Also we expect the users to learn about the tabs with more use.
Findings/Recommendations Finding: While creating an Issue, under Customer & Environment tab, the OS field drop down is not grouped logically. Severity: Minor Recommendation: • Grouping the OS better (e.g. all Windows, Linux Os grouped together) would help.
Findings/Recommendations Issue: Markup language capability interferes with special characters Severity: Minor Recommendation: • Remove the capability to use markup language, if not possible the user should be informed as to how it works.
Findings/Recommendations Finding: • Some users expected to see the key or bug number in the header of the issue. • Some users like to copy and paste the bug number and the title for reference Severity: Minor Recommendation: • Have the Key/Bug number in the header.
Findings/Recommendations Finding: • A lot of users had trouble finding the link File Attachment to get them to the page where they could manage their files and screen shots. • User were confused about the value None Severity: Major Recommendation: • A more explicit link to manage attachments • More descriptive value than None • Make Image Attachments a link as well
Findings/Recommendations Finding: Users expected a reference to the file name as part of comments for an attachment Severity: Moderate Recommendation: • Comments associated with attachments should reference the file name, preferably as a link • If recommendation is not feasible then make it clear to users to reference the file name in comments
Findings/Recommendations Finding: A lot of participants were confused by the Attach screenshot and Attach file operations while attaching a screenshot from a file. Severity: Moderate Recommendation: • Renaming the attach screenshot operation as Screenshot from Clipboard or Copy screenshot would help in making the distinction between these operations clearer. • If possible remove the Attach Screenshot link altogether
Findings/Recommendations Finding: Some users clicked on the close button after opening the image attachment (via thumbnail link) expecting to come back to the view issue screen, but ended up closing the application Severity: Minor Recommendation: • Open the image attachment in a separate tab or window to prevent users from accidentally closing the JIRA.
Findings/Recommendations Finding: Many participants had difficulty finding the Find Issues link on the top menu to search for the issue. Severity: Minor Recommendation: • Improve color contrast between the top menu and the page header
Findings/Recommendations Finding: • Most of the participants were confused by “View” and “View & Hide” buttons. • Some who used “View & Hide” did not know how to get the search menu back again. Severity: Major Recommendation: • Rename the “View” button as “Go” or “Search” to make the action more clear. • Rename the “View & Hide” to “Hide Sidebar”. • Make it easier for the user to get the search menu back after performing the View and Hide operation.
Findings/Recommendations Finding: Some of the users had trouble remembering what they selected as search criteria. Severity: Moderate Recommendation: • Show the search criteria that was selected at the top of the search results.
Findings/Recommendations Finding: Most of the participants did not notice the Blue Box pop up while searching for issues after choosing a project. Severity:Major Recommendation: • Additional filter fields should dynamically update based on user selections in the Project and Issue Type fields • If recommendation is not feasible, make this step clearer in the UI and training.
Findings/Recommendations Finding: The required fields were not marked while resolving and closing an issue, leading to frustrated users. Severity: Major Recommendation: • Indicate all the fields that are required
Findings/Recommendations Finding: Many participants chose None for certain required fields which led to an error message for not entering data. Severity: Moderate Recommendation: • Make all the defaults “Select One” instead of “None”.
Findings/Recommendations Finding: Many users expected to find an option for adding comments in the Comments tab Severity: Moderate Recommendation: • Have the ability to Add comment under the Comments tab
Findings/Recommendations Finding: Organization and layout of left-hand menu made it difficult for users to find the links they were looking for Severity: Moderate Recommendation: • Apply more distinction (e.g. color, spacing, grouping of links etc.) to the various components of the left-hand menu
Additional Recommendations • The bug triage workflow process for JIRA differs from the process that is currently in place, which caused confusion for users around assigning, fixing, resolving, and closing issues. These differences need to be made explicit in training and post-training materials. • Provide Help as either a demo or slideshow presentation that can be accessed any time, as new hires join RSA and will have to learn JIRA. • Use real estate on the Home page to provide hints for getting started with Jira. • This could cover 2-3 commonly occurring issues that users might face (e.g. using the blue box for search, tips for using quick search, info about triage/workflow steps, etc.) • These could be brief descriptions or links to the Help
Next Steps • Determine which recommendations can be implemented prior to JIRA rollout • UXD to work with JIRA project team to provide design direction for implementing agreed upon recommendations, where needed.
