Process Use: Intentional Practice or Just Good Practice?

1. Process Use: Intentional Practice or Just Good Practice? anzea 2013 Conference 22–24 July 2013 Alexandra Park, Epsom, Auckland Michael Blewden Massey University

2. Overview • Background • Research question/approach • Case study of findings • Implications for practice

3. Process Use • Learning and development from stakeholder participation in evaluation • Influence/consequence of evaluation processes • Distinct from/independent of findings use

4. e.g. Stakeholder participation… …enhances willingness to use findings …develops evaluative thinking or action …develops shared understanding …influences the evaluand

5. Patton (1997) says: “Evidence of process use is represented by the following kind of statement after an evaluation: “The impact on our program came not just from the findings but also from going through the thinking process that the evaluation required”

6. Shaping this study • Process use: - enhances value and utility - often an unintentional side-product - more likely if we purposefully seek it

7. Patton (1997) again “...the possibility and desirability of learning from evaluation processes as well as findings can be made intentional and purposeful…” “…instead of treating process use as an informal offshoot, explicit and up-front attention to the potential impacts of evaluation logic and processes can increase those impacts and make them a planned purpose for undertaking the evaluation…”

8. The question of ‘intent’ Patton infers evaluators may • choose to deliberately seek process use • adopt specific practices to achieve it • increase the value and utility of evaluation

9. The question of ‘intent’ • …historically, process use more typically regarded as “...an informal offshoot” (Patton, 2007) • …few methodologies intentionally seek process use…rarely an integrated goal of practice (Morabito, 2002) • Observations of NZ practice and context

10. The (research) question • Why and for what purpose do evaluators seek process use? • Why do they choose the practices they do to achieve it?

11. Research Approach • Process use as ‘sensitising concept’ • Process use as a ‘construction’ • Interpretivist explanation • Importance of context

12. Interpretivist explanation Meaning + beliefs + desires = behaviour e.g. action X was done because person held belief Y according to which doing X would fulfil desire Z

13. Journey to practice Practice setting Traditions Beliefs about evaluation Evaluation practice Values Process use intent and practice Beliefs about outcomes Beliefs about practice Project setting Beliefs about role Evaluation theory Cultural context Beliefs explaining beliefs

14. Meaning Why important Embedded ‘rules’ Intent and practice is ‘understandable’ Justifications, reasons Point and purpose Expectancies

15. Process use examples considered important and intentional General awareness and experience of process use

16. Participants • 24 practicing evaluators • Eligibility criteria • In-depth face to face interviews • Auckland and Wellington location

17. Assumptions • Desirable for evaluators to seek process use but not necessarily always • Pursuing process use may have risks • Understanding, use, relevance or appropriateness of the term not assumed

18. Evaluation as development Evaluation as process Evaluation as capacity building Evaluation as findings use

19. Beliefs about evaluation Social betterment Evaluation as intervention Knowledge is experiential and constructed Intent and practice Enabling Equality

20. Beliefs about practice Collaborative, transparent, understandable, trustful Tools and procedures as learning Accountable to relational ethics and morals Intent and practice Should address issues of power and inclusion

21. Beliefs about role Should facilitate mutual learning, development, improvement Intent regarding process impacts Intent and practice Should act in the interests of those with less power Responsibility to give back/return value

22. Beliefs about outcomes Accept evaluative conclusions/findings Data quality Critical engagement Intent and practice Equality Attitudinal/affective change Capacity development and learning outcomes

23. Explanations • Findings are ‘ideographic’ - however…. • PU integral and inevitable • Intent/practice understandable when evaluators are understood as thinking, ‘meaning makers’ • Enhancing process use about debating the evaluator’s mandate, role, responsibility

24. Reflections • Do these evaluator beliefs have implications – positive or negative? • Are there risks to evaluation? • Could there be process use misuse? • How should we respond?