1 / 13

Survey Development

Survey Development. Rita O'Sullivan Evaluation, Assessment, & Policy Connections (EvAP) School of Education, University of North Carolina-Chapel Hill ritao@unc.edu or 919/843-7878. Overview. Evaluation Theory & Practice Elements of an Evaluation Plan: Logic Models Evaluation Instruments

lloyd
Download Presentation

Survey Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Survey Development Rita O'Sullivan Evaluation, Assessment, & Policy Connections (EvAP) School of Education, University of North Carolina-Chapel Hill ritao@unc.edu or 919/843-7878

  2. Overview • Evaluation Theory & Practice • Elements of an Evaluation Plan: Logic Models • Evaluation Instruments • Q & A

  3. Surveys for a Purpose • What information is needed? • Who needs the information? • How will it be used? • Who has the information you need? • What’s the best way to get it? • What resources are available?

  4. Evaluation Planning - Logic Models

  5. Guide to Instrument Selection

  6. Logic Model –All Stars

  7. Validity and Reliability • VALIDITY: Is the degree to which the assessment is appropriate to its purpose. • RELIABILITY: Will this assessment get the same response every time. • TRIANGULATION: Verifying findings through multiple data sources.

  8. Sampling QUALITATIVE DATA COLLECTION HIGH Lack of Information about the Topic Need for Explanation/Rich Description Issue Complexity Difficulty around Obtaining Information LOW QUANTITATIVE DATA COLLECTION

  9. Types of Sampling • Simple Random • Stratified • Cluster • Systematic • Purposeful • Convenience • Snowball

  10. Response Rates • Factors that influence response rates • Saliency • Reminders • Ease of response • Increasing sample size will not increase response rates

  11. Ensuring Data Quality • Sample representativeness? • Consistent across individuals? • Appropriate to purpose? • Collected in a culturally appropriate manner?

  12. Draft Program Student Survey

  13. Sample Focus Group Protocol • SCALE wants to know how the Student to Student program is working and what impact it is having on your campuses and with your community partners. They have asked us to conduct this focus group to begin exploring these areas. After we talk, we will summarize what you’ve said to capture the essential points of our discussion; we will make sure that individuals cannot be identified from the summary. Before we share the summary with SCALE, we will share it with you to make sure that what we’ve summarized fits with what you intended to say. If no one objects we’d like to record this focus group so that we won’t lose any detail of the discussion. • Start-Up/Campus Recruitment • What kinds of expectations are realistic and reasonable in terms of initial campus start up? • What barriers to recruitment of community partner(s) did campuses face? • What barriers to recruitment of tutors did campuses face? • Tutor Training: • What worked in terms of the initial tutor training SCALE did on your campuses? • How can this initial tutor training be improved? • What kinds of expectations are realistic and reasonable in terms of initial tutor training? • What ongoing training do tutors need? • What ongoing tutor training opportunities are being offered at campus sites? • How, if at all, is the tutoring manual being used during ongoing tutor training?

More Related