1 / 35

Test it in Service

The 17th Annual International Deming Research Seminar. Test it in Service. a.k.a. Voice of the Customer. What We Plan to Cover (AIM). Why Test in Service? Why Good Sample Design? Accuracy versus Precision Some Statistical issues Some Methodological Issues Some Data Issues

ross
Download Presentation

Test it in Service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The 17th Annual International Deming Research Seminar Test it in Service a.k.a. Voice of the Customer

  2. What We Plan to Cover (AIM) Why Test in Service? Why Good Sample Design? Accuracy versus Precision Some Statistical issues Some Methodological Issues Some Data Issues Some Questionnaire Issues The 17th Annual International Deming Research Seminar

  3. Deming Lecture Japan 1950 The 17th Annual International Deming Research Seminar

  4. Evolution of Shewhart Cycle Production Make Specification Design Inspection Sell (a. Old Way) 4. Test in Service 5. Redesign 1. Design Specification Production 2. Make 3. Sell Inspection (b. Original Shewhart 1939) (c. Deming 1950 Version)

  5. Production Viewed as a System Design and Redesign Consumer Research Suppliers of Materials and Equipment Consumer A Receipt and test of materials Consumer B Production, assembly, inspection Consumer Consumer Consumer C Consumer Tests of Processes, machines, methods, costs D Consumer The 17th Annual International Deming Research Seminar

  6. Process Voice of the Customer Voice of the Process GAP Input Conversion Output The 17th Annual International Deming Research Seminar

  7. How do we get the Voice of the Customer? Voice of the Customer Test it in Service Consumer Research Input Conversion Output The 17th Annual International Deming Research Seminar

  8. Test it in Service Thought Data Knowledge The 17th Annual International Deming Research Seminar

  9. Survey Impact on Management Wrong Design Wrong Decision Wrong Information The 17th Annual International Deming Research Seminar

  10. What is the aim of the survey? Who determines the aim? The Subject Matter Expert Why is the aim important? Without an aim there is no system The 17th Annual International Deming Research Seminar

  11. Who is involved in Surveys Subject Matter Expert Statistician Respondents The 17th Annual International Deming Research Seminar

  12. The Subject Matter Expert • Commissions the survey • Determines the aim • Can act on the results • Has budget authority The 17th Annual International Deming Research Seminar

  13. Aspects of Sample Design • Statistical • Sample size • Variance • Accuracy • Methodological • Sample selection • Computation The 17th Annual International Deming Research Seminar

  14. What is a sample • Complete coverage gives answer • A subset of the interest group Draw Sample The 17th Annual International Deming Research Seminar

  15. The Frame • The Frame is a list of sample units • Frames are not always complete The Gap The 17th Annual International Deming Research Seminar

  16. Data vs. Information The sample yields data Data Results Analysis The Subject Matter Expert needs Information Information The 17th Annual International Deming Research Seminar

  17. Information contains • Result • Measure of uncertainty The 17th Annual International Deming Research Seminar

  18. Which is more important Sample size? or Sample method? Sample size? or Sample method? Bias Total Error Sampling Error The 17th Annual International Deming Research Seminar

  19. Accuracy and Precision Sample Method Not Accurate Accurate Not Precise Sample Size Precise The 17th Annual International Deming Research Seminar

  20. Sample Size (n) Expected result percentage Desired precision Where is the actual result t the number of standard deviations under the Normal Curve Then The 17th Annual International Deming Research Seminar

  21. Sample Size Example Q. Was the Conference useful? Please check one: Yes □ or No □ If one wants 95% reliability that the actual responses fall within 10% of the response of 50% then . The 17th Annual International Deming Research Seminar

  22. The 17th Annual International Deming Research Seminar

  23. Because it is the maximum sample size The 17th Annual International Deming Research Seminar

  24. Sample Method • Clear Aim • Random Sampling • Unbiased Questions The 17th Annual International Deming Research Seminar

  25. Clear Aim What does the owner wish to accomplish with the data? How will the data be used in practice? Is the data measureable Cause Effect Is the data useable? e.g. measure engagement with an Association e.g. measure how often contacted by Association The 17th Annual International Deming Research Seminar

  26. Random sample • Equal probability of selection • Random numbers • Random Start nth Sample • Replicated design with random start per zone • Deming's Replicated Design:Total sample cut into zoneslike sliced bread One zone Total sample size The 17th Annual International Deming Research Seminar

  27. Reduce Bias in Questionnaire If questionnaire… …is for interviewing then train interviewer to Avoid loaded questions Avoid verbal bias Avoid body language bias …is for respondent's completion Avoid loaded questions Avoid question overload The 17th Annual International Deming Research Seminar

  28. An open-ended question is designed to encourage a full, meaningful answer using the subject's own knowledge and/or feelings. Opposite of closed-ended question, which encourages a short or single-word answer. Open-ended questions also tend to be more objective and less leading than closed-ended questions. Open Ended Questions Courtesy MyriamOchart, ASQ CSSBB, CMQ/OE

  29. Open-Ended Question Examples Courtesy MyriamOchart, ASQ CSSBB, CMQ/OE

  30. Prevent Response Bias Avoid Ambiguity: Questions not focused enough to obtain the needed information. Poor question: How often do you buy fast food? Better question: The last 10 times you’ve eaten lunch out, how often have you purchased each of the following types of food? Courtesy MyriamOchart, ASQ CSSBB, CMQ/OE

  31. Prevent Response Bias Avoid questions that are too general. Poor question: Do you like orange juice? (Do you mean taste? texture? price?) Better question - more specificity: Do you like the taste of fresh-squeezed orange juice? Courtesy MyriamOchart, ASQ CSSBB, CMQ/OE

  32. Prevent Response Bias Avoid loaded questions: use of language likely to bias response: Do you actually support Proposition X? Courtesy MyriamOchart, ASQ CSSBB, CMQ/OE

  33. Prevent Response Bias Avoid leading questions: Questions that lead respondents toward a particular answer. Isn’t it true that women are more likely to talk on the phone while driving? Courtesy MyriamOchart, ASQ CSSBB, CMQ/OE

  34. What We Covered Why Test in Service? Why Good Sample Design? Accuracy versus Precision Some Statistical issues Some Methodological Issues Some Data Issues Some Questionnaire Issues The 17th Annual International Deming Research Seminar

  35. Questions? The 17th Annual International Deming Research Seminar

More Related