Question Rating Forms for Interviewers

1. Question Rating Forms for Interviewers • The traditional way to obtain information from interviewers about survey question is to have a meeting after the pretest interview. • Problem of this way • The criteria interviewers use to evaluate questions are likely to vary from interviewer to interviewer. • A group discussion is not a very systematic way to elicit people’s view.

2. Question Rating Forms for Interviewers • Solution • To improve the value of what pretest interviewers contribute to the question evaluation, researchers recently have been experimenting with having interviewers fill out systematic rating of each question

3. Question Rating Forms for Interviewers • Rating forms obtain more quantitative information by asking interviewers to rate each question in the pretest questionnaire on selected characteristics of interest to the researchers (whether the interviewer had trouble reading the question as written and whether the respondent understood the words or ideas in the question, among others).

4. Question Rating Forms for Interviewers • Interviewers are asked to rate each question 1. Difficulty in reading question exactly as worded. 2. Contain words or concepts that are not easily or consistently understood by respondent 3. Do respondents have difficulty in knowing or providing the answer to the question?

5. Question Rating Forms for Interviewers • Interviewers are asked to rate each question • A= No evidence of problem • B= Possible problem • C=Definite problem

6. Question Rating Forms for Interviewers Interviewer Rating Form We can developed better than this form Do not make so complicated

7. Question Rating Forms for Interviewers • Rating forms actually serve different functions • Provide a handy way for researchers to tabulate the initial perceptions of the interviewers about questions problem • Access to opinions of all interviewers in evaluating questions. • Focus interviewer attention on the critical aspects of survey question that sometimes are not the main focus of interviewers attention.

8. Question Rating Forms for Interviewers • Rating forms actually serve different functions 4. Focus the conversation in the debriefing on those questions that interviewers consider most problematic. Standardized interviewer debriefing questionnaires collect information about the interviewers' perceptions of the problem, prevalence of a problem, reasons for the problem, and proposed solutions to a problem. They can also be used to ask about the magnitude of specific types of problems and to test an interviewer's knowledge of subject-matter concepts.

9. Question Rating Forms for Interviewers • Standardized interviewer debriefing questionnairescollect information about the interviewers' perceptions of the problem, prevalence of a problem, reasons for the problem, and proposed solutions to a problem. They can also be used to ask about the magnitude of specific types of problems and to test an interviewer's knowledge of subject-matter concepts .

10. Special Questions for Respondents

11. Special Questions for Respondents

12. Special Questions for Respondents

13. Questions ?