The research process, surveys, questionnaires & interviews

1. The research process, surveys, questionnaires & interviews Communication Research Week 3

2. Steps in the research process • 1) Establish the need for research • 2) Define the problem • 3) Establish research objectives • 4) Determine research design • 5) Identify information types and sources • 6) Determine methods of accessing data Communication Research Spring 2005

3. Steps in the research process • 7) Design data collection forms • 8) Determine sample plan and size • 9) Collect data • 10) Analyse data • 11) Prepare final research report Communication Research Spring 2005

4. The research process DETAILS OF STUDY MEASUREMENT Extent of researcher interference Minimal: Studying events as they normally occur Manipulation and/or control and/or simulation Measurement & measures Operational defn Items (measure) Scaling Categorising Coding Types of investigation Establishing: Causal relationships Correlations Group differences, ranks etc Purpose of the study Exploration Description Hypothesis Study setting Contrived Non contrived PROBLEM STATEMENT Data-analysis 1.Feel for data 2.Goodness of data 3. Hypothesis testing Data-collection method Observation Interview Questionnaire Physical measurement Unobtrusive Unit of analysis (pop to be studied) Individuals Dyads Groups Organisations Machines Sampling design Probability/non probability Sample size (n) Time Horizon One-shot (cross-sectional) Longitudinal Communication Research Spring 2005

5. The Classical Ideal of Science Induction Theories Deduction Generalization Hypotheses Method & measurement Operationalization Observation Communication Research Spring 2005

6. DEDUCTIVE Argues from the general to the particular Eg you observe that all deciduous trees lose their leaves … You therefore reason that your bare tree is deciduous INDUCTIVE Argues from the particular to the general Eg if you burn one finger on a hot stove … You therefore reason that you could burn all of them Deductive vs Inductive Reasoning Communication Research Spring 2005

7. Preparing an hypothesis or research question • An hypothesis is a ‘reasonable scientific proposal’ or a ‘statement of expected results’ • It is not a statement of fact but a declarative statement which tells the reader what you are going to do NOT how you plan to do it • An hypothesis is common to scientific research methodologies while qualitative research more commonly uses research questions to focus Communication Research Spring 2005

8. Writing a hypothesis • A well written hypothesis • Is stated in declarative form • Posits a relationship between variables • Reflects a theory or body of literature upon which it is based • Is brief and to the point • Is testable Communication Research Spring 2005

9. Developing an hypothesis • An hypothesis should be developed from a well-researched body of knowledge which is both logical and feasible eg • Inland waterways are becoming polluted through fuel discharge from petrol-driven powerboats • Powerboats are noisy thus creating auditory pollution • Alternatives to petrol engines exist • Electric (battery-powered) boats will decrease pollution on inland waterways Communication Research Spring 2005

10. Ideas, questions & hypotheses Communication Research Spring 2005

11. Rats & scientific research Communication Research Spring 2005

12. Problems with asking questions as evidence (from Vance Packard (1956) The Hidden Persuaders) • 1. You can’t assume that people know what they want • In a survey of male drinkers the men expressed a strong preference for a ‘nice dry beer’. When they were then asked how a beer could be dry, they were stumped.Those able to offer any answers at all revealed widely different notions. Communication Research Spring 2005

13. Problems with asking questions as evidence (from Vance Packard (1956) The Hidden Persuaders) • 2. You can’t assume people will tell you the truth about their wants and dislikes even they know them. • Psychologists at the McCann-Erikson ad agency asked a sampling of people why they didn’t buy one client’s product – kippered herring.The main reason the people gave under direct questioning was that they just didn’t like the taste of kippers. More persistent probing however uncovered the fact that 40% of the people who said they didn’t like kippers had never in their entire lives tasted kippers. Communication Research Spring 2005

14. Problems with asking questions as evidence (from Vance Packard (1956) The Hidden Persuaders) • 3. It is dangerous to assume that people can be trusted to behave in a rational way. • A test was designed to establish the influence of the package on the product. It gave housewives three different boxes filled with detergent and requested that they try them all out for a few weeks and then report which was the best for delicate clothing (Note: actually only the boxes were different, the detergents were identical – one box was predominantly yellow; the second was blue and the third was blue with splashes of yellow). • Result: the detergent in the yellow box was too strong, in the blue box it left the clothes dirty and in the blue and yellow it was ‘fine’ and ‘wonderful’. Communication Research Spring 2005

15. Survey research • Purpose is to determine the current status of a population with respect to one or more variables • Can be qualitative or quantitative, depending on data sought Communication Research Spring 2005

16. Advantages of surveys • Inexpensive • Can obtain current information • Enable the researcher to obtain a great deal of information at one time • Provide quantitative or numerical data • Very common and so some of the info you seek may have already been gathered eg ABS Communication Research Spring 2005

17. Problems with surveys • People often don’t tell the truth, especially about personal matters • People make mistakes about what they’ve done • Obtaining representative samples is frequently difficult • People often refuse to participate • Relatively small percentages of people answer and return questionnaires • Writing good survey questions is difficult Communication Research Spring 2005

18. ADVANTAGES Inexpensive No interviewer bias to worry about You can ask about very personal matters You can ask complex, detailed questions DISADVANTAGES People may misinterpret questions Low response rates the norm You don’t know who actually filled out the questionnaire Sampling errors frequent Self-administered questionnaires Communication Research Spring 2005

19. Stages of a survey • 1. Define your information needs • what information? • purpose of information? • who knows the answers? • practical issues eg time • 2. Carry out background research – what work has been done before in this area? • 3. Choose a survey technique • questionnaires • diaries • individual interviews • group discussions Communication Research Spring 2005

20. Stages of a survey • 4. Define and test your methodology • Open or closed questions? • Pre-test questions • Ordering of questions • Trial questioning techniques • 5. Administer your survey – consider any problems • By mail/email • Is target group representative? • Were all questions answered satisfactorily? • 6. Analyse your results • Did enough people reply? • Is target group representative? • Were all questions answered satisfactorily? Communication Research Spring 2005

21. Stages of a survey • 7. Present your findings • Usually in report format • Outline parameters of survey critically • Do your results prove or imply results? • Acknowledge limitations eg time, sample size, demographics of sample etc Communication Research Spring 2005

22. Problems with Survey Research • Inadequate response • Including unrelated items on questionnaire • Poorly worded items • Complex items • Leading questions • Assuming facts not necessarily in evidence • Analysing open-ended questions Communication Research Spring 2005

23. Problems with Survey Research • People often don’t tell the truth about themselves • People make mistakes even if they are trying to tell the truth eg ratings books • Obtaining representative samples is difficult • Relatively small percentage of people answer and return questionnaires Communication Research Spring 2005

24. Survey & Questionnaire design • When designing a survey or questionnaire you need to consider the type of question which will give you the most accurate data • There are five main types of questions • 1. Close-ended questions which use yes/no responses Q. I have good communications with my supervisor A.  Yes  No Communication Research Spring 2005

25. Survey & Questionnaire design • 2. Open-ended questions which allow the respondents to give an unlimited answer Q. What problems are you having with your supervisor? Communication Research Spring 2005

26. Survey & Questionnaire design • 3. Checklist which present a list of items where participants are asked to check those items that apply to their particular situation. Q. Please check the following types of communications that you have with your supervisor.  Informal meetings  After hours discussions  Formal meetings  Telephone  Written reports  Social gatherings  Emails  Committee meetings Communication Research Spring 2005

27. Survey & Questionnaire design • 4. Multiple-choice questions which offer several choices and the respondent is asked to select the most correct one. Ensure the choices presented cover all the possible options. Q. How often do you purchase items from the company vending machines? a) Once a day b) 2-3 times a day c) 3-5 times a day d) 6 or more times a day Communication Research Spring 2005

28. Survey & Questionnaire design • 5. Ranking scales which require the participants to rank order a list of items. Q. Of the following list of five types of communication that you might have with your supervisor, rank from 1 (most important) to 5 (least important). — Formal meetings — Informal conversations — Written reports — Letters or emails — Telephone discussions Communication Research Spring 2005

29. Survey & Questionnaire design • 6. Likert scales which usually measure attitude toward a concept or idea. It allows the respondent to indicate the degree of agreement usually on a 5 or 7 point scale. Q. Please indicate the degree to which you agree or disagree with this statement. “The 1995 Ford Falcon is a substantial improvement on the 1994 model.” Strongly agree Agree Neither agree Disagree Strongly nor disagree Disagree 5 4 3 2 1 Communication Research Spring 2005

30. Survey & Questionnaire design • 7. Semantic differential scales measures attitudes by displaying pairs of opposite terms and asking respondents to check which term best describes their feeling toward the concept or topic. Q. Place an X in the space between the two terms that best describes how you see the XYZ Corporation Office. XYZ CORPORATION DISTRICT OFFICE [the topic or entity being evaluated] Pleasant - - - - - - - - - - - - -- - - - - Unpleasant Efficient - - - - - - - - - - - - -- - - - - Inefficient Not helpful - - - - - - - - - - - - -- - - Helpful Professional - - - - - - - - - - - - -- - Unprofessional Insensitive - - - - - - - - - - - - -- - - Sensitive Friendly - - - - - - - - - - - - -- - - - - Unfriendly Slow - - - - - - - - - - - - -- - - - - Fast Rigid - - - - - - - - - - - - -- - - - - Flexible Communication Research Spring 2005

31. Other types of qualitative research – depth interviews • one-on-one interview which is relatively unstructured with a subject by a trained interviewer. • The direction of the interview is guided by the responses to the questions Communication Research Spring 2005

32. ADVANTAGES Interviewer can explain Qs in detail Interviewer can use a variety of data collection methods Interviewer can spend a lot of time with respondents You know who is answering the questions A higher likelihood of achieving the desired response rate Not intimidating DISADVANTAGES Can be intrusive (too personal) Time-consuming and expensive Hard to find people in sample at times People are reluctant to answer some questions Needs well-trained interviewers Possible language barriers Other types of qualitative research – personal interviews Communication Research Spring 2005

33. Other types of qualitative research – focus groups • Often used in market research • A group is asked a series of structured questions and guided through structured sessions • Designed to probe their attitudes and feelings about a range of issues Communication Research Spring 2005

34. Other types of qualitative research – projection techniques • Such as word association tests, sentence and story completion, cartoon tests, consumer drawings, photo sorts • These techniques belong to the field of clinical psychology • They are designed to probe and penetrate a person’s defense mechanisms and allow true feelings to emerge Communication Research Spring 2005

35. Communication Research Spring 2005