Assessing Quality In Student Work Products: The Role of Customers and Expectations

1. Assessing Quality In Student Work Products: The Role of Customers and Expectations by Barry McNeill & Lynn Bellamy Arizona State University

2. Today’s Goals • Have audience achieve Knowledge Level of Learning for: • ASU Freshperson course structure • quality in a student work product • the work product assessment process • the role of the customer in assessment • Have audience achieve Knowledge and Comprehension Level of Learning for: • assessment checklists

3. Today’s Agenda • Introduction to ECE 100 5 min • Quality In Student Work 20 min • characteristics • customer needs • assessment process • Checklists 20 min • creation • use

4. Learning Objectives Tree

5. ECE 100 Concepts Session Laboratory Session Modeling Session ASU Course Structure 1. 120 students 2. introduction to problem solving 3. quizzes & an exam 4. lecture / discussion / work 5. work in ad hoc groups 1. 80 students 2. practice problem solving 3. work on a) artifact dissection b) mousetrap powered device 4. do assessment 5. create design notebook 6. work in semester team 7. faculty guidance 8. hands on work 1. 40 students 2. work on chapters from How to Model It 3. create models 4. start work in class & finish outside of class 5. give oral presentations of in class work 6. work at computer - two students per computer 7. faculty guidance

6. Quality - What Is It? • Pick a workproduct that a student submits in one of your classes • Is the quality of the workproduct something explicitly addressed in the assignment? • Write down your definition of what would constitute a quality submission (4 minutes) • Share your definition with a neighbor(5 minutes)

7. Quality - Some Definitions • Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy implied or stated needs (ANSI/ASQC Standard A3 - 1987) • The least expensive method of achieving quality is to deliver a design that cannot fail to satisfy the customer (Genichi Taguchi) • Quality is a characteristic of thought and statement that is recognized by a nonthinkng process. Because definitions are a product of rigid, formal thinking, quality cannot be defined. But even though Quality cannot be defined, you know, what Quality is! (Robert Pirsig)

8. Kano’s Customer Needs • Expected Needs (Requirements)Expected requirements are those basic attributes that the customer assumes are present in generically similar products or services • Revealed Needs (Requirements)Revealed requirements are those attributes that customers talk about when describing what would make a product or service better • Exciting Needs (Requirements)Exciting requirements are those attributes whose presence makes the customer say WoW!

9. Kano’s Needs vs Satisfaction

10. Grader/Faculty Faculty Student Expectations Do Assignment (assess work) Assess Work Meet Expecta-tions Exceed Expecta-tions Needs Improvement NI Exceeds Expectations E YES YES NO NO Time Bring Work Up to Expectations One Week MeetsExpectations M Assess Improvements Meet Expecta-tions Needs Improvement NI MeetsExpectations M NO YES

11. What Are Checklists • A method to reduce the possibility of multiple interpretations possible with written and/or oral instructions • An assessment checklist is a collection of Yes/No questions that enumerates the Expected Requirements

12. Why Are Checklists Hard To Make? • Since the checklists are given out before the work is started, they must be complete, nothing can be added later • A complete set of Expected requirements means nothing important is left out • A complete set cannot contain items that you might sometimes consider as not expected

13. How To Make Checklists • List the learning objectives for the assignment • List all the things (attributes) that have to be present in the work product if Terry Sleuth is to demonstrate that the learning objective has been accomplished • Refine the second list until it contains only definitive Yes/No questions • Select the important items from this second list

14. Creating A Plotting Checklist(pairs/triads - 5 minutes) • Assume the Learning Objective in assigning a plot are to master presentation of data in a chart • Create a list of attributes that would show that Terry Sleuth has mastered the learning objective

15. Part of Checklist II Chart Requirements (Exceeds or Meets expectations, Needs Improvement) (meeting expectations requires a YES for all of the following items) A For all of the charts in the work product, the following items must all be checked: 1. If you found the chart in the middle of the street, would you understand it? (YES, NO) 2. If the chart were reproduced, would it still be readable? (YES, NO) 3. Do both axes have descriptive titles (N.B. not a single letter) which include units? (YES, NO) 4. Are there labeled divisions (text or numbers) on the axes? (YES, NO) 5. Are the “values” for the axes at the origin of the chart clear? (YES, NO) 6. Is the dependent variable (item measured or predicted) on the vertical axis? (YES, NO) 7. Is there an appropriate, descriptive title? (YES, NO) 8. If the variables are presented in the chart’s title, is the dependent variable mentioned first? (YES, NO) 9. If there is more than one chart line, is there a descriptive legend? (YES, NO) 10. Are the data points shown? (YES, NO) B Comments if Charts Exceed Expectations or Need Improvement

16. Use Checklist

17. ECE 100 Home Page • http://www.public.asu.edu/~ece100 • Assignments, Assessments, and First Day Materials • Download Engineering Core Workbook for Active Learning, Assessment & Team Training