1 / 16

Responding to the Client: Choices from Across the Spectrum of Evaluation Practice

Responding to the Client: Choices from Across the Spectrum of Evaluation Practice. John Owen (Dr) Centre for Program Evaluation The University of Melbourne IPEN Conference, Batumi, Georgia, September 2011. Key Ideas . Responding to client needs Developing appropriate evaluation protocols

butch
Download Presentation

Responding to the Client: Choices from Across the Spectrum of Evaluation Practice

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Responding to the Client: Choices from Across the Spectrum of Evaluation Practice John Owen (Dr) Centre for Program Evaluation The University of Melbourne IPEN Conference, Batumi, Georgia, September 2011

  2. Key Ideas • Responding to client needs • Developing appropriate evaluation protocols • Applying relevant theoretical concepts

  3. Example 1 Organisation: AusAID Program: Introductory Academic Program [IAP] • Induction program for overseas graduates • Implementation in 38 universities Nature of the evaluation: One ‘point in time’ review of IAP.

  4. Example 1 Evaluation Protocol: IAP • Process-outcomes focus • Mixed-methods approach • Report uncertainty in findings Manager control turned to positive involvement

  5. Example 1 Key Theory - Pragmatic Principles • Are key questions answerable? • Can a design be developed to answer questions? • Can we implement this design? • Will the results be usable?

  6. Example 2 Organisation: Asia Education Foundation Program: Studies of Asia in Australian Schools • Influencing school curricula and teaching about Asian culture Nature of the evaluation: Advice required on impact of range of existing and future initiatives

  7. Example 2 Evaluation Protocol - Studies of Asia Evaluation Decisions • Several small studies at intervals over three years • Mixed-methods • Strategic reporting Evaluator involved in influential political settings

  8. Example 2 Key theory Strategic Interactivity Professional linkage between knowledge producer (evaluator) and knowledge user (policy maker) increases the quality of program related decision-making.

  9. Example 3 Organisation: University of Third Age Program: U3A Adult Education Program for Seniors Nature of the evaluation: Provision of information about future program directions.

  10. Example 3 Evaluation Protocol: U3A Seniors Evaluation decisions • Trained evaluator manages evaluation • Needs assessment to establish program direction • Involvement of U3A members in data management • Conference to celebrate findings High work load for trained evaluator Facilitation and management role of evaluator essential

  11. Example 3 Key theory Transformative Participative Evaluation Program deliverers and participants control all aspects of organisational decision making, including evaluative action.

  12. Roles for Evaluators: Type 1

  13. Roles for Evaluators: Type 2

  14. Roles for Evaluators: Type 3

  15. Lessons Learned: Evaluator Roles • Expansion of what counts as evaluation • Clients may prefer one approach • Evaluators indifferent operational roles (independent/interested/involved) • Types have theoretical bases • Types have theoretical champions • Pragmatic: eg Datta • Interactive: eg Huberman • Participative: eg Cousins

  16. Issues for Discussion • What approaches to evaluation are currently favored in your workplaces? Why? • Are there implications for your work - based on this presentation?

More Related