User-Test Report Memo: Setup

1. User-Test Report Memo: Setup

2. Step 1: Memo format To: [Instructor] From: [Your name] Date: [Due date] Subject: User Test Report of [name of your instructions]

3. Step 1: Sample Memo Format To: Kim Lacey From: A. Student Date: March 2, 2010 Subject: User Test Report of the ‘How to Model a Bracket in NX 5’ instructions.

4. Step 2: Introduction (New paragraph: skip line, block left) Introduction: Description of your report and “why” you are testing it. Answer the question: Why am I testing my instructions? What’s the purpose? *Your introduction should be about 1/2 a page, possibly less.

5. Step 2: Sample Introduction Introduction: Engineers must use their creative ability along with their acquired skills to produce various engineering appliances. Designing and modeling plays a major part of the process that leads to the culmination of creative ideas which produce engineering marvels. Engineers can use computer aided design (CAD) programs to aid them in this designing process. Thus, a set of instructions were written and tested for the Novice and/or Experienced users, for how to model a bracket using Unigraphics NX 5.

6. Step 3: Objectives (pg. 423-4) (New paragraph: skip line, block left) Objectives: Answer the questions: What’s the idea/motivation behind my instructions? What specific questions do I have about my test? You can write two, small paragraphs answering each one of these questions. *Your objectives section should be about 1/2 a page.

7. Step 3: Sample Objectives Objectives: The idea behind writing the instructions was to familiarize engineering students and potential users with modeling, using CAD. In this case the Unigraphics program is used as the medium to accomplish the intended modeling of the bracket. The purpose of using the bracket is to serve as an example for the users to understand the functioning in a CAD environment using the instructions. At the same time, the instruction serves as a means to introduce the users to NX 5. The testing of the draft instructions will, presumably, provide feedback for improving the instruction. Any form of information which is either missing or unnecessary can be added or deleted according to the test results. The test will also illustrate the usage and aid of the images used in the instructions. The test should also help to demonstrate the clarity of each step and organization of the material.

8. Step 4: Methods (New paragraph: skip line, block left) Methods: Describe your test readers and why you selected them. Describe what kind of test you designed: understandability, location, or performance. Why did you select that specific test? How did you perform the test?

9. Step 4: Sample Methods Methods: Three individuals were asked to test the instructions. All the three testers were from the engineering background. Testers 1 and 2 were asked to read and follow each step of the instructions and complete the modeling of the bracket using NX 5. Testers were taken to the computer labs at Wayne State University, and were monitored while they performed each step. Tester 3 was asked to review the instructions. A questionnaire was given to each Tester to assess the effectiveness of the instructions. A sample of the questionnaire is attached on page 6. Notes were taken when the testers were struggling with a step. They were assisted only when a problem arose that was out of context of the instruction or which would affect their ability to perform other steps. It was also noted if the testers were able to complete the model. Testers 1 and 2 are the ideal target users for testing the instructions as the instructions were written for the novice and partially experienced users. Following is a brief background of each tester. Tester 1: A junior engineering student currently attending Wayne State University. The tester had no prior experience using the NX 5 software. It was the tester’s first time using a CAD program, while testing the instructions. Tester 1 was termed as a Novice user. Tester 2: A junior engineering student currently attending Wayne State University. The tester had prior knowledge and experience of NX 5. Tester 2 had taken the CAD course at Wayne State. Before beginning the user test, Tester 2 was asked to follow the instructions only as stated in the document, and not to regard any prior knowledge of using NX 5. Tester 2 was termed as a partially experienced user. Tester 3: The instructor of the CAD class of the Mechanical Engineering Department at Wayne State University. The instructor was asked to review the instructions, complete the questionnaire and comment or suggest for any improvements. Tester 3 was termed as an experienced user.

10. Step 5: Results and Discussion (New paragraph: skip line, block left) Results and Discussion: What happened during the test? What responses did you get from your testers? Was there something that surprised you? Did everything go as planned? Were there any similarities between the testers? (e.g., did everyone get confused by the same image?) Also, note specific comments from the testers here.

11. Step 5: Sample Results and Discussion Results and Discussion: The testing yielded a few discrepancies in the draft instructions. A few of the images of the draft instructions were unclear to the testers. Minor changes were prescribed for specific steps because the testers were seen to have trouble continuing to the following steps. Testers 1 and 2 gave major feedback for improving the instructions. The following 1 – 5 are common problems the Testers were found to have while following the instructions on the computer. 1. Each user had a different level of difficulty understanding a few steps. There were only a few instances in the instructions which were found to be a hindrance to all testers. Figure 1 of the instructions under Conventions, showing dimensions of the bracket, was found difficult to read by all three testers. Therefore, Figure 1 will definitely be considered as part of the improvement for the revised instructions. 2. During testing, Testers 1 and 2 accidentally performed commands in NX 5 which were not mentioned in the instructions. Tester 2, who is partially experienced, managed to find the Undo tab to rectify the problem. For the novice Tester 1, however, this was a problematic situation that led to a dead end, from where the Tester did not know what to do next. In this scenario Tester 1 was helped in undoing the mistake. As Tester 1 is a target user, it could be a problem for novice users to continue with the rest of the instructions if they accidentally perform an unsubscribed step or make a mistake. Therefore, it is necessary to consider this fault in the instructions for improvement. 3. Tester 1 and 2 had common problems in the Hole section. Both Testers had difficulty identifying the Y-Z plane of Step ii in the Hole section. The reason for this may be due to the positioning of the Notes for this step, which was kept under the Figure for that step. The Testers kept observing the Figure to understand the step, instead of moving ahead to see the Note before they performed the step.

12. Step 5: Sample, cont. 4. While performing the test, it was also found that in the Hole section a few intermediate steps were missing. These were the ‘Click Ok’ steps which were missing. This was found to confuse the Testers while following the instructions. The Testers were helped in this case due to the incompleteness of the instructions. 5. Another common trouble for the Testers was found related to Figure 1, the Hole and Slot section. The Testers had troubles locating the dimensions for the second hole and slot. Throughout the instructions all the dimensions are stated in each step. But for the last step of the Hole and the Slot section the user has to refer to Figure 1 for the dimensions. The Testers took a while before they could understand where to look for the dimensions. This poses a concern for the target users and should be dealt with. Comments which were stated by the Tester or observed: Tester 1: Tester had trouble in Step ii of the Line Command section and Step i of Constraints section. The Tester was continuously holding the mouse button to create the Line and Constraint. The Tester was supposed to click mouse and let go to create the Line/Constraint. Failed to complete the model. Tester 2: Did not understand term ‘drag down arrow’ for Step i of Constraints section and Step ii of the Slot section. Had trouble locating the ‘drag down arrow’. Suggested few of the Notes were not important, specifically point to the note under Figure 25 in the instruction. Finished the model. Tester 3: Suggested, to explain why inverted L was used as starting point for the model because there are alternative ways to begin the model. Also, suggested to reduce the gaps and large white spaces between steps or sections. This was observed to be a minor problem, since not all the Figures in the instructions of the same size. Each Figure had to be accommodated, which distorted the page layout because sometimes a figure would carry to the next page leaving the step on the page above. This created a void, sometimes half a page long, immediately after the step.

13. Step 6: Conclusions (New paragraph: skip line, block left) Conclusions: If you were going to revise your instructions, based on the feedback from your testers, what would you change? *Number these conclusions

14. Step 6: Sample Conclusions Conclusions: The user test for the instructions illustrated partial deficiency in the material. Mostly, users had common difficulties which help locate and rectify those problems. The specific difficulties faced by each user will be considered in a lighter note than the common ones because these seem to arise from each Tester’s experience of operating a computer, not NX 5. Using the result of the user test the following changes were made to the instructions. 1. Figure 1 was enlarged and dimensioned again to make it clear and all dimensions visible. As far as understanding Figure 1 goes, it is clearly state in the Intended Users part in the introduction that the user must have a basic understanding of engineering drawing to read the instructions. 2. Since it was found that novice users were prone to making mistakes, it was necessary to introduce the Undo command in the instructions. The Undo command was introduced before the beginning of the actual instructions, under Directions along with other hints which would be useful to the users. 3. Testers 1 and 2 had trouble locating the plane in the Hole section. Figure 6 for that step was enlarged and an arrow was put into the figure to point out the plane. The Notes of that step, along with those of other steps, were placed above the figure for those respective steps. This was done to avoid confusion to the users, so that they could read the Note before viewing the figure and performing then step. 4. The minor incompleteness of the Hole section was rectified. The intermediate steps of ‘Click Ok’ were added wherever appropriate to remain consistent and clear to the users.

15. Step 6: Sample, cont. 5. For the trouble Testers had regarding finding the dimensions for the last step of the Hole and Slot section, there will be no changes to the step. This is because it is clearly stated in the introduction section under Usage that the users are expected to complete the model with the given information. This part is intended to serve as a practice exercise for the user to strengthen their skill, as stated in the introduction. All the information is given, and it can only be speculated that the Testers failed to understand the Usage of the instructions. 6. Tester 1 had trouble with Step ii of the Line section and Step i of Constraints Section, with using the mouse button. As stated above, this is a minor fault, which can is attributed to the Tester’s weak computer skills, not necessarily the instructions. It is clearly mentioned in the Conventions, in the instructions, the definition of a ‘Click’. 7. Tester 2 did not understand the ‘drag down arrow’. There is no mention of the term in the Conventions or anywhere in the instructions. Therefore, the drag down arrow is be labeled in the respective figure in Step i of Constraints section and Step ii of the Slot section. 8. Tester 3 suggested pointing out why the inverted L was used. This was a important point, as it will inform the users that there are alternative methods to model the bracket. Therefore, a brief note was added to the Usage in the Introduction section, of the instructions, to point out that an alternative method is possible to model the bracket. Tester 3 also suggested reducing the gaps and white space between sections to make the instructions layout user friendly. For this purpose, a few of the images were cropped to fit in exactly below the respective steps on one page. On other instances it was difficult to clear the gap between the figures due to the default formatting done by Microsoft Word. In this case a few extra Notes or Hints were added to accommodate the white space. Lastly, a note was added to the conventions in the instructions, to begin every sub-section e.g. Hole on a new page.

16. Step 7 (last one!): Appendix Skip page Appendix: Include any questionnaires you distributed to your testers.

17. Step 7: Sample Appendix Appendix: Questionnaire 1. Did you have any problems navigating through the instructions? 2. Did you have any difficulty in understanding or performing any step? if so, which step and what was the difficulty? 3. Were the images and figures clear, understandable and helpful? 4. Did you find any information or step which was unnecessary or feel that it should not have been included in the instructions? 5. Did you have any difficulty understanding the dimensions in Figure 1? Yes/No 6. Did you find the Hints and Notes useful? Yes/No 7. Did you understand the subject/scope, aim, intended user and usage of the instructions? Yes/No 8. Please suggest or comment on any other issue that wasn’t covered in this questionnaire.

18. Step 7: Sample, cont. For Novice/Partially Experienced Users: 9. Were there any terms in the instructions which you did not know of? If so what were they? 10. Where you able to complete the bracket model following the instructions? Yes/No For Experienced Users: 11. Did you feel that there were steps missing? Which steps? 12. Do you think any information or step, in the instructions should be added, changed or deleted?