Presurvey Evaluation of Questions

1. Presurvey Evaluation of Questions

2. INTENSIVE INDIVIDUAL INTERVIEW

3. INTENSIVE INDIVIDUAL INTERVIEW • The priority of the process is to find out how respondents understand question and perform the response task • Respondents often are brought into a special setting in which interviewers can be recorded and observed (laboratory interview) • People who conduct the interview usually are not regular survey interviewers. Sometime they are research staff ,psychologists

4. INTENSIVE INDIVIDUAL INTERVIEW • Most critically ,the basic protocol involves reading question to respondents, having them answer the question, and then same strategy for finding out what was going on in the respondents minds during question-and-answer process

5. INTENSIVE INDIVIDUAL INTERVIEW • Selection of respondents, people to be interviewed in this process should be representative of the range of the individuals who will be interviewed in the actual survey

6. INTENSIVE INDIVIDUAL INTERVIEW • Procedures for monitoring the interviewing • Think aloud interviews • Asking probe or follow-up questions after each question or short series of questions • Going through the question twice, first having respondents answer them in the usual way , then returning to the question and having a discussion with respondents about the response task.

7. INTENSIVE INDIVIDUAL INTERVIEW • Think aloud interviews • Respondents are trained to think out loud, to try to articulate their thoughts and their cognitive processes for understanding the question , search their mummeries for information required, and turn the information they have.

8. INTENSIVE INDIVIDUAL INTERVIEW • Think aloud interviews • Plus • give a good window into how questions are being understood and the answers are being generated.

9. INTENSIVE INDIVIDUAL INTERVIEW • Think aloud interviews • Negative • Respondent vary how they will perform the task, some more verbalizing than other • Concerned that think-aloud itself affects the way respondents address answering question.

10. INTENSIVE INDIVIDUAL INTERVIEW Asking question about the question-and-answer process 1. Paraphrase their understanding of the question 2. Define terms 3. Uncertainties or confusions they had about what the appropriate answer was

11. INTENSIVE INDIVIDUAL INTERVIEW Asking question about the question-and-answer process 4. Confidence about their accurate answer 5. If the question called for a numerical figure, asking respondents how they arrived to the number, asking respondents to talk about the process they went through to decide on the answer.

12. INTENSIVE INDIVIDUAL INTERVIEW This method “Asking question about the question-and-answer process” can be done in two forms 1. Asking these question after each question 2.Going through the question twice, first having respondents answer them in the usual way , then returning to the question and having a discussion with respondents about the response task.

13. INTENSIVE INDIVIDUAL INTERVIEW The first form has advantage that when questions fellow immediately after the respondent went through the task, It is easier for respondents to talk about their thought processes. But it breaks up the interview and any relationship that may exist among questions

14. INTENSIVE INDIVIDUAL INTERVIEW • INTENSIVE INDIVIDUAL INTERVIEW are best and most productive if a specific objectives and concerns have been laid out. • Researchers should flag particular issues with respect to vocabulary, comprehensive , and forming a response that they consider potentially problematic. • INTENSIVE INDIVIDUAL INTERVIEW are complementary to focus group.

15. INTENSIVE INDIVIDUAL INTERVIEW

16. INTENSIVE INDIVIDUAL INTERVIEW • Aspects of good question design that neither focus group nor intensive individuals interviews address • Test whether a question is easy and comfortable for interviewers to read as written. • Task that paid volunteers are able and willing to do under laboratory conditions may not be the same as when respondents are interrupted in the middle of their day

17. INTENSIVE INDIVIDUAL INTERVIEW • Questions that are clearly problematic under laboratory conditions will almost certainly be problematic under normal survey interviews conditions as well, the opposite may not be the case

18. Questions ?