PRR 389

1. PRR 389 • Module 1: Concepts, Philosophy, Process • Module 2: Goals, Objectives, Perf. Measures • Module 3: Data Gathering & Analysis • Module 4: Forecasts & Impacts • Module 5: Communication/Reporting

2. Planning/Research/Evaluation • Concepts: Definitions & Philosophy • Purposes: Why plan, evaluate? • Processes : Systematic Steps • Types of Plans, Evaluations, Research

3. Definitions • Planning = preparation for decision making • Evaluation = process of judging the merit or worth of something • Research = application of scientific methods to answer questions • Pairs: Evaluation research, research planning, planning evaluation

4. Related terms • Design • Policy • Management • Data gathering … • Decision making • Analysis

5. Comprehensive planning • Systems of facilities and programs • Multiple providers and stakeholders

6. Why Plan? • Prepare for/anticipate future • Establish goals, directions • Coordinate various activities & programs • Balance needs/wants with available resources • Establish systematic steps to achieve goals • Requirements for funding

7. Messes • 1. Not well defined • 2. Multiple ill-defined objectives • 3. Multiple decision makers • 4. Incomplete information. • 5. Dynamic. • 6. Multiple stakeholders • 7. Uncertainty

8. Characteristics of Recreation & Tourism • Nature of R/T • Diversity of activities • Hard to define • Freedom of choice • A “frill”, worthy of serious study? • Supply Characteristics • Services • Fragmented providers/industries • Public-private– non-profit mix • Role of consumer in providing experience • Externalities, public goods, common property resources

9. Characteristics (cont) • Demand Characteristics • Demand for what? • Dispersed use • Substitution not understood • Multi-purpose trips • Peaking of demand • Prices/markets • Methodological • Fuzzy concepts • Weak Measures/measurements • Planning- management-research gaps • Interdisciplinary, young as sciences go

10. Trends • Government-private roles, privatization • Federal, state, local roles • Partnerships, cooperation vs competition • Increasing accountability • Convergence of public & private/business approaches • Scale, pricing • Shifting markets, activities • Linkages with health, crime, aging, economic dev., diversity & other societal issues

11. Incremental vs Comprehensive Planning • Lindblom, “science of muddling through” planning is .. • Incremental • Remedial • Serial • Exploratory • Fragmented • Satisficing • Iterative

12. Patton & Sawicki Tips for planning/policy analysis • Focus quickly on central decision criteria • Avoid tool box approach • Learn to deal with uncertainty • Say it with numbers • Make analysis simple and transparent • Check the facts • Learn to advocate positions of others • Give client analysis, not decisions • Push boundaries of policy envelope • No such thing as correct, rational, complete, comprehensive analysis

13. Steps in Planning Process • Define the system • Establish goals and objectives • Gather information • Generate Alternatives • Evaluate Alternatives • Select & Implement • Monitor • Iterate

14. Types of plans • By geographic scope/extent • Building, Facility, Site Master Plan (Design) • Community, Regional, State, National, System of Parks/Programs • By Activity/Function • Housing, transportation, recreation tourism, land use, open space, physical plan, comprehensive plan … • Program, maintenance, safety, security, risk management, budget personnel, marketing… • By Time • Feasibility, Short – Long Range, Strategic Plan, annual performance plan, EIS.

15. Specific Types • Community recreation plan • SCORP • Environmental impact statement • Business Plan • Feasibility study • GMP, LMP, … • Community or Regional tourism plan

16. Why Evaluate - the Academic list • To assess merits of alternative programs • To discover whether & how well objectives are being fulfilled • To determine the reasons for successes & failures. • To uncover the principles underlying a successful program. • To refine, revise, update or track a program

17. Why Evaluate- "The Real List" 1. Because we are required to 2. To make better decisions. 3. To learn from experience 4. To justify programs 5. To kill programs

18. Research Evaluation Evaluation Research

19. Purposes of Research • Answer management questions - applied research • For sake of knowing - pure or basic research • Answer research questions - methodological • Develop/test management alternatives - developmental research • Assess worth or merit of programs - evaluationresearch

20. Research Purposes - II • Exploratory • Descriptive • Explanatory • Predictive

21. Ways of Knowing or Establishing belief or truth • Tradition • Authority • Repetition/Tenacity • Science

22. Body of Knowledge systematic abstract general parsimonious Method of Inquiry logical induction deduction self-corrective empirical Science

23. Scientific Management • Application of scientific principles to management and decision making • systematic information gathering • empirical, objective, self-corrective

24. Research define problem objectives/hypotheses literature review research methods gather data/analysis conclusions further research Evaluation describe program evaluation criteria program scoping evaluation methods gather data/analysis conclusions modify program? Process -- Steps • Planning • define system • specify goals/objectives • gather info • generate alternatives • evaluate alternatives • choose & implement • monitor, evaluate • Iterate

25. Types of Evaluationby Program Stage • formative (conceptualization/design)] • process (implementation) • summative (outcomes, impacts, efficiency)

26. Types - By Approach • Standards • norm-based • criterion-referenced • Goals and objectives • Impacts or effects

27. Types- by method • Qualitative - quantitative • Secondary data - Primary • Survey - Experiment • Internal vs external • Who - Peers, participants, superiors, subordinates, scientists

28. Evaluation Criteria • Effort - qnty and qlty of inputs • Performance - qnty and qlty of outputs • Adequacy - meet needs? • Efficiency - benefits/costs • Equity - distributional issues, fairness

29. Process Evaluation • Identifies how and why program works • attributes • recipients • conditions • effects • single or multiple • intended or side effects • timing & duration, long/short term • cognitive, affective or behavioral

30. Types by program area/subarea • Emphasis: Parks, Tourism, Interp, Program Mgmt, Commercial Rec, ... • Program : lands, facilities, personnel, budget, marketing, PR, maintenance, design, policies, plans

31. Types of Research • Basic - Applied • Exploratory, Descriptive, Explanatory, Predictive • In-house / out- house • by discipline - sociological, psych, economics • by topic - boating, wilderness, legal,tourism, ... • by method - survey, expmt, ...

32. Examples of Evaluation Studies • Community needs assessment • Feasibility study, SWOT analysis • Program Audits, Market audit • Cost/benefit, Cost effectiveness analysis • Impact assessment • PPBS, MBO, GPRA, CAPRA, LAC, ... • Peer review • Importance-performance analysis

33. AEA Principles for Evaluators • Systematic Inquiry • Competence • Integrity/Honesty • Respect for people • Responsibility for public welfare

34. Scriven- main points • Merit vs Worth • Grading vs Ranking • Facts and values • Big six: program, personnel,performance, policy, proposal, product • Need vs market assessment • Holistic vs analytic • Indicators vs true measures of merit • Goal Achievement Eval vs Goal Free Evaluation

35. Goal free evaluation (GFE) • Don’t worry about goals – look at all effects • Problems with goal achievement evaluation • Whose goals, how measured? • Unintended/side-effects; interaction effects • Values left behind • Costs, alternatives/competing programs • Generalizability • Synthesis