1 / 23

Survey Dilemmas: Determining Sample Size Improving Response Rate Measuring Satisfaction

Survey Dilemmas: Determining Sample Size Improving Response Rate Measuring Satisfaction. Daniel R. Judd, Ph.D., M.P.A. Judd Research. 2009 NACEP Member Survey. Why do Surveys?. Collect data from CEP stakeholders to Measure for regulatory requirements, e.g., NACEP accreditation standards

jess
Download Presentation

Survey Dilemmas: Determining Sample Size Improving Response Rate Measuring Satisfaction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Survey Dilemmas:DeterminingSample Size Improving Response Rate Measuring Satisfaction Daniel R. Judd, Ph.D., M.P.A. Judd Research

  2. 2009 NACEP Member Survey

  3. Why do Surveys? Collect data from CEP stakeholders to • Measure for regulatory requirements, e.g., NACEP accreditation standards • Monitor quality outcomes for improvement • Example: Retention of CEP students

  4. CEPs are being asked to Quantify their Contributionto the Sponsoring Institution

  5. Does greater involvement in CE increase the likelihood that students will attend the Sponsor Institution?

  6. Comparison Current Status

  7. How do I Conduct a Survey? • Plan in established ethical context – OIR/IRB • Formulate research questions - latent variables example: Does CEP affect student development? • Simultaneously, develop instrument & get list • Compose tracking strategy – submit to IRB • Pilot test instrument and tracking strategy • Send out a request for responses – follow-up... • Collect and process data • Perform analysis – develop a report

  8. AAPOR PROFESSIONAL ETHICS • We shall not knowingly make interpretations of research results that are inconsistent with the data available, nor shall we tacitly permit such interpretations. • We shall not knowingly imply that interpretations should be accorded greater confidence than the data actually warrant. American Association for Public Opinion Research, Code of Professional Ethics and Practices.

  9. First Dilemma: How Many Respondents Do You Really Need?

  10. Determining sample size is a very important issue: Samples that are too large may waste time, resources, and money, while samples that are too small may lead to inaccurate results.

  11. Sample Size Formula Sample size = z2 * (p) * (1-p) c 2Where: z = z value (e.g. 1.96 for 95% confidence level) p = percentage picking a choice, expressed as decimal (.5 used for max. )c = confidence interval, expressed as decimal  (e.g., .04 = ±4)

  12. Reduced n because of homogeneity: p = 80% n=1.96² x .8(1-.8) .05² n =3.8416 x .16 .0025 n =.61465 .0025 n=245.86 ~ 246 p = 30% n=1.96² x .3(1-.3) .05² n=3.8416 x .21 .0025 n =.8068 .0025 n=322.72 ~ 323

  13. I was better prepared

  14. Calculating a Minimum Sample • Determine confidence level and interval • 95% confidence level • confidence interval is the plus-or-minus range, if question asked of entire population—usually ±5% • Use Internet: sample size calculator • surveysystem.com • researchinfo.com

  15. Second Dilemma How Important is the Response Rate?

  16. Dilemma • Response rate: the number of eligible sample units that cooperate in a survey -- has historically been central to survey research in the United States. . . . but the relationship between response rates and survey quality has become much less clear. • Results that show the least bias have turned out, in some cases, to come from surveys with less than optimal response rates. http://www.aapor.org/Response_Rates_An_Overview.htm

  17. NACEP Member Survey - March 2009 • Note about Methodology: A notice sent to all 255 members of the Partnership Listserv at the end of February 2009, directed people to a survey in SurveyMonkey. Reminders were sent 1 week, 2 weeks, and 3 weeks . . . • Response rate: 183 of the 255 members responded, for a 72% response rate.

  18. What works? • Grades!! UU grants early access to grades for completion of online survey = 90% rr • Follow-up. . . by telephone and/or postcard • Parents? (next) • What have you found that works?

  19. Can Parents Answer for Students? Ranking of Decision Factors - Identical Results Concurrent Enrollment as a Vehicle for Recruitment & Retention, 2008

  20. Third Dilemma • What is satisfaction? • The gap between expectation and experience • Why measure satisfaction? • Word of mouth recommendation is powerful. • Who is it best to ask? • Students, parents, faculty, HS counselors

  21. Q8. How do you rate the value of NACEP?

  22. Comparison Credit and Satisfaction2006-07 1-Yr. Follow-up Study

  23. Thanks Complete the Evaluation!

More Related