1 / 21

Assessing K-12 Service-Learning Impacts and Quality

Assessing K-12 Service-Learning Impacts and Quality. Shelley H. Billig, Ph.D. RMC Research Corporation. Overview. Evaluation Toolkit Logic Models and Theory of Change Measuring Impact Academic impact Civic impact Personal/social impacts Others Measuring Quality. Evaluation Toolkit.

cecily
Download Presentation

Assessing K-12 Service-Learning Impacts and Quality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing K-12 Service-Learning Impacts and Quality Shelley H. Billig, Ph.D. RMC Research Corporation

  2. Overview • Evaluation Toolkit • Logic Models and Theory of Change • Measuring Impact • Academic impact • Civic impact • Personal/social impacts • Others • Measuring Quality

  3. Evaluation Toolkit • How-To Guide For Traditional Evaluation • Overview • Steps for Planning and Implementation • Worksheets • Detailed Guidance

  4. Session Experiences • “Dip in” for various aspects of the evaluation and assessment process.

  5. Logic Models • Shows how and why the program will work; • Connects program activities to program outcomes through outputs: • e.g., activity (input)  Increased student engagement in schools (output)  Increased student academic achievement (outcome).

  6. TX K-12 Learn and Serve Logic Model

  7. Try It! • Start with your activities – what will you do? • Specify your goals – what will you accomplish? • Identify those factors that will influence results, both positively and negatively. Include demographics, context, implementation issues

  8. Logic Model Use • Once the logic model has been created with stakeholders, review it. • Use the logic model to design or revamp the evaluation. • Use the constructs identified in the logic model to guide the development of measures (instrumentation).

  9. Logic Model Resources • University of Wisconsin-Extension Program Development and Evaluation, www.uwex.edu/ces/pdande • W.K. Kellogg Foundation, www.wkkf.org • Evaluation Handbook, 1998 • Logic Model Development Guide, 2001

  10. Assessing Academic Impact • Where is service-learning likely to have the greatest academic impact? • Knowledge areas (general and specific) • Types of skills (general and specific) • Moderators • What measures are available?

  11. Academic Skills • Areas of likely impact • Higher order thinking skills such as problem solving, analysis, transfer of learning • Specific knowledge related to service activities, e.g., measurement, writing, synthesizing materials, research • What are the available measures? • Rubrics • Test scores • Writing scores • Problem solving essays • Measures of academic engagement • Absenteeism • Surveys • More!

  12. Assessing Civic Impacts • Where is service-learning likely to have the greatest civic impact? • Knowledge? • Skills? • Dispositions? • Attachment to community? • Moderators • What measures are available?

  13. Civic Impacts • Areas of likely impact • Knowledge of community needs • Skills for planning and implementing projects • Taking responsibility • Ethic of service • Available measures • Surveys • Focus groups/interviews • Hours of service • Actual changes

  14. Assessing Social/Personal Impacts • Where is service-learning likely to have the greatest impact on social/personal areas? • Knowledge areas (general and specific) • Types of skills (general and specific) • Moderators • What measures are available?

  15. Social/Personal Outcomes • Areas of Likely Impact • Development of character traits such as responsibility, trust, and caring • Respect for diversity • More bonds with adults • More protective factors such as resilience • Greater self-efficacy • Measures • Surveys • Focus groups/interviews • Rubrics • Self-assessments

  16. Sample Measures of Outcome Areas • Examine the samples • What is the best use of each? By whom and for what purpose? • What do you like and dislike about each?

  17. Measuring Moderators Moderators are characteristics that influence outcomes. Examples: Instructor experience; program design; program quality. Need to identify and measure those factors like to serve as moderators.

  18. What moderators are likely to be in play in your settings? • Consider demographics. • Consider program design. • Consider quality. • Consider instructor characteristics like experience, levels of training, and expertise. • Consider partner characteristics.

  19. Characteristics of Good Assessments • Valid • Reliable • Objective • Useful • Items measure only one construct at a time (not double barrelled…) • Related to the logic model or assessment questions

  20. Open Discussion • Questions and Answers

  21. Resources • National Service-Learning Clearinghouse www.servicelearning.org • Billig, S. H., & Waterman, A. S., eds. (2003). Studying service-learning: Innovations in education research methodology. Mahwah, NJ:Lawrence Erlbaum Associates. Available at www.erlbaum.com • Bringle, R. G., Phillips, M. A., & Hudson, M. (2004). The measure of service learning: Research scales to assess student experiences. Washington, DC: American Psychological Association. Available at www.apa.org/books

More Related