Service Learning 2007 - 2009

1. Service Learning2007 - 2009 Dr. Albrecht Research Team EXAMPLE of EVALUATION RESEARCH

2. About The Project • Dr. Carol Albrecht was hired by the State of Texas Service Learning Center • 18 public school districts in the state of Texas were involved in Service Learning Projects • We met with them to identify how the program impacted public school children, teachers, community partners and parents of students

3. the process

4. Step #1

5. Identify the Objectives

6. Identify the outcomes

7. Develop Indicators

8. Step#2

9. Select a sample

10. Identify Diverse schools

11. Service Learning Projects • Middle/High School • Raising money for various causes • Tutoring Elementary children • Landscaping Parks • Building playground equipment • Painting murals on boy/girl club walls

12. Step #3

13. Use a Survey to develop indicators

14. Relationship between success and levels of service learning participation

15. How the students felt about participating in the program • Development of Leadership Skills • Development of Problem Solving Skills • Increased Affect Toward School • Increased Academic Aspirations • Positive Affect Toward Program

16. Step #4

17. Select Research Design • We decided to use multi-methods to increase the validity and reliability of our findings. • Used Experimental Design to evaluate the program and test our hypotheses • Pre-test survey • Intervention • Post-test survey • Used Focus Groups • In addition we used focus groups to gain additional information/insights and further test the validity/reliability of our survey.

18. Step #5

19. Administering the Test • Texas A&M students went all over Texas and administered the pre-test • Five months later they returned to administer the post-test • 12 schools were selected for the focus groups • They were recorded by the students • They were later transcribed • A coding schema was developed to compare and contrast answers across schools

20. Step #6

21. Writing the report • We wrote a final report and prepared a power point presentation for the teachers and administrators. • It was important to interpret the information accurately even though it was not what the Texas Center for Service Learning was hoping to hear. • In summary, we found very small and sometimes even differences comparing the pre- and post-tests. Most of the difference were NOT statistically significant. • These findings were consistent across two years • However, we had collected data that allowed us to do a more in-depth analysis, and so we proceeded.

22. Evaluation • We now looked at the quantity and quality of the experience more specifically – and how that influenced outcomes. • To measure quantity of Service Learning, we measured how many total hours students had spent in service learning that year. • To measure quality, we measured sense of ownership • Did students help plan the Service Learning Project? • Did students make decisions about the Service Learning Project?

25. CHART 1. High School Students’ Perception of How Good They are at Speaking in Front of Groups by Whether or Not They Made Decisions about Service Learning Projects (p <0.0001)

26. CHART 2. High School Students’ Perception of How Good They are at Finding Resources by Whether or Not They Made Decisions about Service Learning Projects (p <0.0001)

27. Even though I had done research for many years, I learned and “re” learned some important things about evaluation research from this project. • Click here to link to power point outlining the things we learned about: • The Impact that Timing can have on Results. • Asking the “Right” Questions and Selecting the “Right” Indicators. • The Impact of Social Desirability on Findings and how to Deal with it. • The Importance of using Multi-Methods • when doing Evaluation Research.