Presurvey Evaluation of Questions

1. Presurvey Evaluation of Questions

2. FIELD PRETESTING

3. FIELD PRETESTING • Experienced interview conduct 15-35 interviews with people similar to those who will be respondents in a planned survey • People interviewed are likely to be chosen on the basis of convenience and availability.

4. FIELD PRETESTING • Interviewers report on particular problems, such as typographical errors, faulty instructions about skipping questions and inadequate arrangements for recording answer. • Find out how long interviews take.

5. FIELD PRETESTING • The criteria for question evaluation often are not well articulated (interviewers are likely to differ in their perceptions of what constitutes a question problem. • The ability of interviewers to diagnosis questions is confounded by 1. putting them in a dual role (interviewers + observers) 2. the experience of interviewers (senior interviewers are likely not to be sensitive to the question problems that they are successful in solving

6. FIELD PRETESTING • Interviewers in pretests have small samples of respondents. When a respondent has problem with question, there is a judgment to be made about whether the fault lies with respondent or with the question. • Debriefing sessions may not be a good way to get information about interviewer evaluations

7. FIELD PRETESTING • Some question problem may not be apparent in the course of a pretest interview without a special steps such as focus group and intensive interviewers.

8. FIELD PRETESTING Strategies for strengthening the traditional pretest • Systematic coding of interviewer and respondent behavior during the pretest interview • The use of systematic rating forms by interviewers to evaluate questions. • The use of special probes for debriefing respondents to pretest interview.

9. FIELD PRETESTING Coding Interviewer and Respondent Behavior • Pretest interviews are tape recorded. This can be done by person or telephone (it is important to inform respondents explicitly that the interview is being tape recorded and get their permission.

10. FIELD PRETESTING Coding Interviewer and Respondent Behavior • The way that interviewers read the question is evaluated. A) question read as worded (no error) B) Question read with minor error C) Question read with major error • Minor error : reading the question slightly different from how it is written without changing the meaning • Major error: reading the question differently from how it is written and changing the meaning

11. FIELD PRETESTING Coding Interviewer and Respondent Behavior • If the interviewers correctly skipped the question. • The respondent interrupted the reading of the question, either by asking or attempting to answer • The initial response of the respondent was inadequate and met the objective .To improve the answer the interviewer • Repeated the question • Used other kind of probe to get an answer that met the objective.

12. FIELD PRETESTING Coding Interviewer and Respondent Behavior • The respondent asks for clarification of the question. This code is used any time a respondent asks the interviewer anything about the question

13. FIELD PRETESTING Coding Interviewer and Respondent Behavior

14. FIELD PRETESTING Coding Interviewer and Respondent Behavior Interrupted , clarification, and inadequate answers are significant • Indication of un-clarification and not consistent understood • Do not clearly specify the kind of answer that will meet the objective • The more often interviewers are required to solve the problem before an answer is obtained the more likely they are influence answers and the less standardized the data collection

15. FIELD PRETESTING Coding Interviewer and Respondent Behavior • Coding pretest interviewers is quite easy and efficient • Coding does not require any particular background or experience. • Coding process can be done almost at the rate of the interview.

16. FIELD PRETESTING Coding Interviewer and Respondent Behavior • The strength s of behavior coding results are they objective , systematic, replicable, and quantitative • using the result of behavior coding by conducting debriefing session

17. Questions ?