Adapting Pretesting Strategies To Self-Administered Questionnaire

1. Adapting Pretesting Strategies To Self-Administered Questionnaire

2. Adapting Pretesting Strategies To Self-Administered Questionnaire • Interviewer- administrated survey: Strategies such as behavior coding or systematic interviewer rating can be used because question-and-answer process is carried out orally • Self- administrated questionnaire: All the key of the question-and-answer behavior take place inside a respondent head

3. Adapting Pretesting Strategies To Self-Administered Questionnaire • In Self- administrated questionnaire we want to be sure that the question is understood consistently, and the answer meet the question objective

4. Adapting Pretesting Strategies To Self-Administered Questionnaire • Focus group • Cognitive interview (more important for Self-Administered Questionnaire) • A distinctive feature of Self-Administered Questionnaire compared to interviewer, is that respondents are looking at all response alternative (Give the respondent a version of question they are asked)

5. Filed Pretesting Without Observation Procedure • Replicate mail survey • Answer a few debriefing question about Confused question Difficulty answering Other problem Advantage Give information about willingness to write return the questionnaire and How long it takes to fill out

6. Filed Pretesting Without Observation Problem This way does not provide very much information about question-and-answer process

7. Using Observation to Evaluate Self-Administrated Questions Procedure Watching people fill out self- administrated questionnaire Advantage Give information about: How long the questionnaire will take Identify the questions that have a problem (if the respondent take a long time to answer these question)

8. Using Observation to Evaluate Self-Administrated Questions Problem Need an observer to look over the shoulder of each respondent to identify the question that may take longer than average Note This method consider an area in which further research might be valuable

9. Debriefing Respondents Procedure 1. Allow the respondents to fill out the questionnaire 2. Carry out a brief with each or group respondents about the survey 3. Asked about any problem they had with reading or answering.

10. Tabulating Answers

11. Tabulating Answers • Tabulate the distribution of answers can help to evaluate questions • < 20 the value of tabulating is minimal (# sample consider a limitation) • >30 the tabulation is informative • Non representative sample is consider a limitation

12. Tabulating Answers Information can be gleaned from tabulating : • Response distribution across subjective scale( right response alternative have been offered) • In some case, survey constructed so that follow-up question are asked for people who give a particular response Example: In general, in describing the way you feel about the medical care you receive, would you say you are generally satisfied or generally not satisfied

13. Tabulating Answers Information can be gleaned from tabulating : Example: Q1: In general, in describing the way you feel about the medical care you receive, would you say you are generally satisfied or generally not satisfied? Q2: (IF not satisfied) what is the main way in which you are not satisfied? To have more information we can make some changes

14. Tabulating Answers To have more information we can make some changes • Change the initial question, in an attempt to elicit more dissatisfied response. • Ask a follow question for a border sample or for all respondent Q3: In what ways, if any, would you say you are dissatisfied with the medical care you receive ? c. Drop the follow-up question altogether.

15. Tabulating Answers Information can be gleaned from tabulating : 3. The rate of item non-response, the percentage of respondents who do not give an answer at all to the question. 4. Analysis of the relationship among questions can be done from data collected in pretest

16. Tabulating Answers Information can be gleaned from tabulating : • If several questions are aimed to measure the same thing , it is useful to cross-tabulate and look for correlation • Highly related (drop one of the question without losing information) • Not Highly related (the questions are measuring two different things , need to rewriter one or both questions)

17. Tabulating Answers A special use of pretest is to provide information about the answers given to questions that call for narrative (Open-ended) answers Problem with Open-Ended Question Difficult to code Answers are often diverse Complicating analysis

18. Tabulating Answers Advantage of Open-Ended Question Used to evaluate questions by: • Detect certain inadequacies of the question as posed • Provide information about the range of answers that people are likely to give especially if the researchers think there is a long list of plausible answers. • Based on the pretest, the researchers may include a short list of answers will cover most people have to say or transformed the question to closed-ended question

19. End of the Chapter 5 Questions ?