REC 375—Leadership and Management of Parks and Recreation Services

1. REC 375—Leadership and Management of Parks and Recreation Services Jim Herstine, Ph.D., CPRP Assistant Professor Parks and Recreation Management UNC Wilmington

2. Evaluation REC 375—Leadership and Management of Parks and Recreation Services

3. Evaluation Defined • Evaluation is the systematic process of determining the effectiveness of current practices, procedures, and plans • Evaluation is a process that judges worth

4. Why Evaluate? • Determine that funds are wisely spent • Conduct services efficiently • Reinforce staff efforts or recommend new directions • Assist policy makers in directing the organization toward more productive channels • Evidence the strengths of services

5. Types of and Approaches to Evaluation • Outcome Evaluation • Process Evaluation

6. Outcome Evaluation • Also called product evaluation, program evaluation, impact evaluation or summative evaluation • Documents the effect of a service on a client

7. Process Evaluation • Also called formative evaluation • Determines whether the service has been conducted in an efficient, legal, and ethical manner • Produces information during and after the delivery of the program or service that judges the worth of its development and operation

8. Scope of Evaluation • Annual evaluation of goals and objectives • Annual review of policies • Needs assessments • Management information systems • Performance appraisal of personnel • Program effectiveness • Equipment cost-effectiveness • Risk management • Fiscal accountability

9. Steps in the Evaluation Process • Determine the evaluation issues • Select an evaluation design • Collect the information • Analyze the information • Prepare a report • Implement the findings

10. Maintaining Research Integrity • Be objective • Minimize bias • Approach the project fairly and honestly • Understand the principles and procedures of evaluation • Have exceptional mastery of the tools of research • Be concerned with honest and careful evaluation

11. Sampling Techniques • Probability Sampling • Non-probability Sampling

12. Probability Sampling • Probability sampling methods are based on randomness • Desire is to be able to “generalize” to the entire population from which the sample is drawn • Most common method is simple random sampling—all members of the study population have an equal opportunity to be chosen • Fishbowl Technique • Random sampling with replacement • Random sampling without replacement • Also can use a table of random numbers or computer based selection

13. Probability Sampling, continued • Not always possible to select respondents with complete randomness • Stratified sampling • Systematic sampling • Deliberate sampling

14. Stratified Sampling • The overall population of respondents is divided into several different subgroups according to common characteristics—age, gender, ethnic or racial identity, socio-economic status, etc. • 20 basketball teams, 30 soccer teams, 25 volleyball teams, 35 baseball teams • Want a 25% survey sample • Take 25% of each category as the stratified sample

15. Systematic Sampling • This selection method occurs when some “system” is applied to the subjects • Interview every 5th person that comes by • While systematic selection approximates a random sample, it does not strictly satisfy the definition of random sampling and, hence, is not entirely bias free • It is quick, efficient and saves time and energy

16. Deliberate/Purposive Sampling • With this method, the researcher knows that specific characteristics exist in a certain segment of a population • Since these traits are extremely critical to the results of the investigation, the researcher deliberately selects those subjects who contain the characteristics • This selection process is always biased

17. Non-Probability Sampling • Samples not selected at random • Intact classes, volunteers, a typical group or individual • Intact or available groups impose a serious restriction on the researcher’s ability to generalize the data obtained to the larger population from which the sample was drawn

18. Information Gathering Methods • Questionnaires and structured interviews • Unstructured interviews and focus groups • Observations • Document study/secondary data review • Non-reactive or unobtrusive measures • Physical traces

19. Analysis of Information • Information gathered or measured is essentially nothing more than numbers or words—”raw” data • Need to analyze the “raw” data to determine exactly what it means

20. Quantitative Data Analysis • Describing Data • Frequency distributions • Measures of central tendency • Mean, mode and median • Measures of variability • Standard deviation, spread of scores • Measures of relationship • Chi-square test (named variables) and Pearson correlation (numerical variables)

21. Quantitative Data Analysis • Inferring or inferential statistics • Measure differences between variables • T-test—is there a significant difference between the scores of 2 groups (compares means) • Analysis of variance—is there a significant difference between the scores of 3 or more groups

22. Quantitative Data Analysis • Always have issues regarding “Reliability” and “Validity”

23. Reliability • Refers to whether the repetition of a study or measurement will result in the same or a very similar answer • Do I get the same answers again and again? • If subjects stand on a scale and the subjects are the same weight, does the scale record their weight as equal

24. Validity • Refers to whether the evaluation actually measured what was intended to be measured • Is the weight of the subjects who stood on the scale accurate? Do they really weigh what the scale says?

25. Qualitative Data Analysis • Not all data that is collected is quantitative • Some of the data is qualitative • Text (narrative), photographic or video form • Look for themes and commonalities

26. Evaluation Jim Herstine, Ph.D., CPRP UNCW herstinej@uncw.edu 910.962.3283