1 / 62

SURVEY RESEARCH

SURVEY RESEARCH. TTM531 Fall 2001 Class Presentation Nancy Whelchel Coordinator for Survey Research University Planning and Analysis NCSU nancy_whelchel@ncsu.edu 515-4184 http://www2.acs.ncsu.edu/UPA/survey/survey_research/. Major Tasks. ID Purpose of Research

orsen
Download Presentation

SURVEY RESEARCH

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SURVEY RESEARCH TTM531 Fall 2001 Class Presentation Nancy Whelchel Coordinator for Survey Research University Planning and Analysis NCSU nancy_whelchel@ncsu.edu 515-4184 http://www2.acs.ncsu.edu/UPA/survey/survey_research/ Survey Research: TTM531 Class Presentation

  2. Major Tasks • ID Purpose of Research • Decide on Survey Design and Format • Select Sample • Design Survey Instrument • Pre-test Instrument • Collect Data • Analyze Data • Report Findings Survey Research: TTM531 Class Presentation

  3. Purpose of Your Survey Ask Yourself: • WHY do I want to/need to do a survey? • WHAT do I want to learn? • WHO is my audience? • HOW will I use the information I gather? Survey Research: TTM531 Class Presentation

  4. Purpose of Your Survey Our Example: • Why = President of NCTextiles.com, where you work, told you to. • What want to learn = How does employee satisfaction and productivity compare between NCTextiles.com telecommuters and non-telecommuters. • Who is audience = Company president and executive board. • How use = Set new policies for telecommuting. Survey Research: TTM531 Class Presentation

  5. Purpose of Your Survey • Description Discover the distribution of certain traits or characteristics (e.g., #telecommuters, $revenue generated) • Explanation Understand why something happens (e.g., is there a difference between telecommuters and non-telecommuters in job satisfaction and in productivity) • Exploration Help inform a research design Survey Research: TTM531 Class Presentation

  6. Think About Available Resources • What information is already available from non-survey sources? • Have any surveys already been done on this question? • Who is going to: • Design the survey • Administer the survey • Enter the data • Analyze the data • Write up the results • Present the findings Survey Research: TTM531 Class Presentation

  7. Think About Available Resources Should I hire a professional??? • What resources/skills are available in-house? • What types of decisions are going to be made based on your findings? • How much money do you have for the survey? Survey Research: TTM531 Class Presentation

  8. Basic Survey Designs • Cross-sectional • Data collected at one point in time from a sample selected to describe a larger population • Longitudinal • Trend • Different samples from a general population studied at different points in time. • Cohort • Same specific population studied over time (may be different samples). • Panel • Same sample studied over time. Survey Research: TTM531 Class Presentation

  9. Basic Survey Formats Format determines much of research design (e.g., sampling, questionnaire design, administration, data analysis, etc.) Basic formats include: • Self-administered mail survey • Telephone interview • Face-to-face interview • Focus group • Self-administered Web survey • Self-administered Email survey Survey Research: TTM531 Class Presentation

  10. The Survey Sample WHO or WHAT do I want to survey? Sampling allows you to study just a portion of the population/group and make estimated assertions about the population from the sample. Warning: Bad Sample = Bad Data Survey Research: TTM531 Class Presentation

  11. Why Sample? • Time Shorter time in the field decreases the chance that external factors will influence results. • Quality Doing fewer surveys increases control over the entire process (interviewing, data entry, etc.) and therefore decreases error • Cost Survey Research: TTM531 Class Presentation

  12. What is a Sample? Some definitions: • Element or Sampling Unit Unit about which information is collected (e.g., employee) • Survey Population Aggregate of all elements interested in from which sample is selected (must be clearly defined) (e.g., NCTextiles.com non-executive employees with job tenure 3+ months ) Survey Research: TTM531 Class Presentation

  13. What is a Sample? Some more definitions: • Sampling Frame Actual physical source listing elements in the survey population e.g., Edited list from company employee directory . • Sample Population Collection of elements selected from the survey population sampling frame Survey Research: TTM531 Class Presentation

  14. What is a Sample? A Warning: Findings from a survey are only representative of the aggregate of elements in the sampling frame. In other words… You can’t use your results to say anything about the folks who didn’t have a chance to be included. Survey Research: TTM531 Class Presentation

  15. Sample Size Depends on: • Cost • Level of precision want (“margin of error”) • Level of confidence want • Anticipated response rate • Number of responses needed per subgroup interested in Survey Research: TTM531 Class Presentation

  16. Sample Size Example: Margin of Error +/- points confidence interval 80% 90% 95% 5 150 275 400 10 40 70 100 If 45% of 400 respondents agree that NCTextiles.com provides “excellent” health benefits, this means that 95 out of 100 times between 40% (45%-5) and 50% (45%+5) believe the company provides “excellent” health benefits. Source: McClelland, Lou (1993), “Survey Research Mechanics in Three Checklists,” (working paper). Office of Research and Information: University of Colorado at Boulder. Survey Research: TTM531 Class Presentation

  17. Selecting a Sample • Probability Sample • A representative sample • Everyone in the sample population has an equal chance of getting in the sample. • Results can be generalized to the sample population. • Non-Probability Sample • Non-representative sample • Purposive, quota, who’s available • Results can not be generalized beyond the sample. Survey Research: TTM531 Class Presentation

  18. Probability Sampling • Simple random sample • List the elements in the sampling frame and use a random number chart to select elements. • Systematic sampling • List the elements in the sampling frame and take every Nth element, based on how many needed in sample. • Warning: Be attentive to any inherent patterns in how sampling frame is ordered. Survey Research: TTM531 Class Presentation

  19. Probability Sampling • Stratified sampling • Draw appropriate proportion of elements from homogeneous groups interested in (based on theory). • Note: Think about using disproportionate sampling then “weighting” the data. E.G., If there are relatively few numbers in one sub-group you are interested in, include proportionally more of them in your sample, then “weight” the data to reflect their true proportion in your population. Survey Research: TTM531 Class Presentation

  20. Probability Sampling • Stratified sampling continued An Example: 250 out of your 2000 employees in the sample population telecommute (12.5% of sample population). With stratified sampling with sample size of 800, would have 100 telecommuters (800 * .125), and 700 non-telecommuters (800 * .875). Survey Research: TTM531 Class Presentation

  21. Probability Sampling • Stratified sampling continued An Example continued: BUT, you might want to “oversample” telecommuters (even include all 250) to be able have less error in their results, and “undersample” non-telecommuters (e.g., 550). When analyzing all results, “weight down” telecommuters to reflect their true proportion in the sample population, and “weight up” non-telecommuters. Survey Research: TTM531 Class Presentation

  22. Instrument Design Start with: • Conceptualization • Define the concepts you want to study • Operationalization • Specify indicators of your concepts. • Make exhaustive list of everything the concept could include Survey Research: TTM531 Class Presentation

  23. Instrument Design Conceptualization Job Satisfaction = “A pleasurable or positive emotional state resulting from the appraisal of one’s job.” Includes both extrinsic and intrinsic aspects. Has several dimensions: • importance to employee • motivation of employee • current level of attainment • satisfaction with attainment Survey Research: TTM531 Class Presentation

  24. Extrinsic: pay promotion supervision co-workers company physical conditions security social status work itself Intrinsic: variety of work autonomy and freedom responsibility feedback creativity recognition meaningfulness of work Instrument Design Operationalization: Job Statisfaction Survey Research: TTM531 Class Presentation

  25. Questionnaire Construction The way a question or statement is worded and the response options offered determine the nature of the data received. Bad question = Bad data Survey Research: TTM531 Class Presentation

  26. Questionnaire Construction Some jargon… • Questions need to be VALID “…an empirical measure adequately reflects the real meaning of the concept under consideration.” (Babbie, 1990) • face validity = the measure jibes with common agreement of what a concept entails. • predictive validity = the measure accurately predicts behavior • content validity = the measure covers the range of meanings included within the concept. • construct validity = the measure relates to other measures theoretically related to concept. Survey Research: TTM531 Class Presentation

  27. Questionnaire Construction Some more jargon… • Questions need to be RELIABLE “…a particular technique, applied repeatedly to the same object, would yield the same result each time.” (Babbie, 1990) Survey Research: TTM531 Class Presentation

  28. Questionnaire Construction Open-end Questions • Respondents write question response in own words • probably need to enter data by hand • develop coding scheme for responses • content analysis ? Suggestion: It’s a good idea to always include an open-end question giving the respondent the opportunity to add any additional comments they might have. Survey Research: TTM531 Class Presentation

  29. Questionnaire Construction Closed-end or Forced-choice Questions • Respondent selects question answer from among a list of possible responses Suggestion: It’s a good idea to ask multiple indicators of a single concept, and then create a scale by combining those indicators. Survey Research: TTM531 Class Presentation

  30. Good Questions... Good Questions... • Are clear and use simple language • Are concise • Are specific • Are possible to answer • Are relevant to the respondent • Do not use negatives • Avoid biased terms • Have only 1 part (not “double-barrel”) Survey Research: TTM531 Class Presentation

  31. Good Questions... Use simple language... My job is usually interesting enough to keep me from getting bored. Is better than : My job is usually engaging enough to permit me from reaching a banausic state of mind. Survey Research: TTM531 Class Presentation

  32. Good Questions... Are concise... How interested would you be in a flexible work schedule in which you would have to work an 8 hour day in the office, but would be able to arrive at work anytime between 6:00 and 10:00 AM and leave between 2:00 and 7:00?. Is better than: How interested would you be in being able to leave your house either before the morning rush hour or after the morning rush hour, and then arrive at the office not at the standard 8:00 AM time that is required according to the employee handbook, but at whatever time you got there, and then work a 8 hour day at the office (excluding and hour for lunch), and then leave the office so you are not commuting during rush hour but either before or after rush hour? Survey Research: TTM531 Class Presentation

  33. Good Questions... Are specific... In what year did you first start working at NCTextiles.com? Is better than: When did you start working here? Survey Research: TTM531 Class Presentation

  34. Good Questions... Are possible for the respondent to answer... NO: My contribution to the company has had a direct impact on its ability to control a broader share in the global market. Survey Research: TTM531 Class Presentation

  35. Good Questions... Are relevant... NO: Polyester doesn’t breathe as well as natural fibers. Survey Research: TTM531 Class Presentation

  36. Good Questions... Do not use negatives... The lack of flexibility in my work hours makes it difficult for me to spend the amount of time I would like to with my family. Is better than: The lack of flexibility in my work hours does not make it possible for me to spend the amount of time I would like to with my family. Survey Research: TTM531 Class Presentation

  37. Good Questions... Do not use biased terms or suggestions... Do you agree or disagree with the following statement: All employees need to be in the office during regular business hours. Is better than: Don’t you agree with the president of the company when she says that all employees need to be in the office during regular business hours? Survey Research: TTM531 Class Presentation

  38. Good Questions.. Do not ask two questions in one (“double- barrel”)... I find real enjoyment in the work I do. I like the hours I keep at work. Is better than: I find real enjoyment in the work I do and the hours I keep. Survey Research: TTM531 Class Presentation

  39. Response Options Response options should reflect the concepts you are trying to measure, and fit with the question wording. Suggestion: Avoid simple “yes” or “no” type responses and try to measure intensity of feeling when possible. Survey Research: TTM531 Class Presentation

  40. Response Options Must be: • Mutually exclusive can select only one appropriate answer Example: NOTmutually exclusive... Q. On most days, what time do you leave your house for work? 6:00 AM or earlier 7:30-8:00 AM 6:00-6:30 AM 8:00-8:30 AM 6:30-7:00 AM 8:30-9:00 AM 7:00-7:30 AM 9:00 AM or later Survey Research: TTM531 Class Presentation

  41. Response Options • Mutually exclusive continued Example: IS mutually exclusive... Q. On most days, what time do you leave your house for work? Before 6:00 AM 7:30-7:59 AM 6:00-6:29 AM 8:00-8:29 AM 6:30-6:59 AM 8:30-8:59 AM 7:00-7:29 AM 9:00 AM or later Survey Research: TTM531 Class Presentation

  42. Response Options Must be: • Exhaustive all possible answers are listed (including e.g., “other, “ “don’t know,” etc.) Example:NOT exhaustive Q. On most days, what time do you leave your house for work? 6:00-6:29 AM 7:30-7:59 AM 6:30-6:59 AM 8:00-8:29 AM 7:00-7:29 AM 8:30-8:59 AM Survey Research: TTM531 Class Presentation

  43. Response Options • Exhaustive continued Example:IS exhaustive Q. On most days, what time do you leave your house for work? Before 6:00 AM 7:30-7:59 AM 6:00-6:29 AM 8:00-8:29 AM 6:30-6:59 AM 8:30-8:59 AM 7:00-7:29 AM 9:00 AM or later Survey Research: TTM531 Class Presentation

  44. Question Order • Be attentive: Initial questions effect answers to subsequent ones. • Ask less interesting questions (e.g., demographics) at end of self-administered survey and beginning of interview survey. Survey Research: TTM531 Class Presentation

  45. Instructions for Respondent • Provide general instructions in beginning of self-administered survey • Brief explanation could include: • why doing survey • why answers are important • how to answer questions • stress confidentiality (if appropriate) • Do NOT ask permission - - assume respondent will want to participate. • Provide specific instructions to questions as necessary Survey Research: TTM531 Class Presentation

  46. Instructions for Respondent Example Instructions: The following statements are about some features of your job. Please tell us how satisfied or dissatisfied you feel with the features of your present job by circling one of the response options listed below each statement. There are no right or wrong answers. All replies are confidential so give your honest opinion on each of the statements. Please do not discuss your answers with other people. It is your opinion which matters. Even though it may be hard to decide, please be sure to answer all questions. Thank you for your help. Survey Research: TTM531 Class Presentation

  47. Questionnaire Format • Should be visually attractive and nicely reproduced • Should be spread out and uncluttered • better to have more than 1 page than cramped page • Should be broken into logical sections when possible • Should have clear skip patterns for contingency questions Survey Research: TTM531 Class Presentation

  48. Questionnaire Format • Should have clear spaces for respondents to mark answers • use boxes • parentheses ( X ) • or numbers to circle • avoid lines to put check on X_ X 1 Survey Research: TTM531 Class Presentation

  49. Questionnaire Format Helpful hint…. Be sure to seek advice on question wording, response options, and questionnaire format from those who will be doing the data entry and data analyses. Survey Research: TTM531 Class Presentation

  50. Pre-Test Test the survey instrument before doing the actual study. Suggestion:It’s also a good idea to test out your sample design, data collection methods, data processing, and data analysis if possible. Survey Research: TTM531 Class Presentation

More Related