1. The Problem of Policy Implementation: the case of performance appraisal

1. The Problem of Policy Implementation: the case of performance appraisal 2. Teacher Learning Through Portfolios

Group 8 Members Lam Ka Po, Angela Lam Wing Chung, Terence Lok Wai Yi, Begonia Ng Siu Kai

The Problem of Policy Implementation The case of performance appraisal

Aim of the research • To identify the obstacles which lead to slow and sporadic policy implementation • To investigate new perspectives of implementing policy as solving a problem • To identify how policy implementation could be improved

Obstacles of current Appraisal system • Interest group approach • Initiatives come from governments and their associated ministries and departments • Affected by the political origins of these initiatives • Conflict between the government and the teachers associations

Policy as Solving a Problem • Offer a conceptualization of performance appraisal as a complex problem • Specify the requirements for a quality solution at both national and local levels • Prevent satisfying the requirements of particular interest groups rather than teachers themselves

Definition of Problem All the conditions or constraints on the solution (variously weighted) plus the demand that the solution (an object satisfying the constraints) be found - by Thomas Nickles

Constrains on the solution • Values • the policy should recognize the professionalism of teachers • Practical requirements • it must not cost more than schools can afford • Goals • it must improve the quality of teaching and learning • or any combinations

Integration of constraints • Particular constraints is likely to be in conflict in complex problems • Need to integrates the constraints taken as a set by reaching a compromise between different interest groups • Require shifts in the definition of particular constraints to obtain an adequate solution

Phases in Policy Making • Intended Policy • What various groups wants • Actual Policy • The document, legislation and/or report • Policy-in-use • Regional and school-level reaction - by Bowe et al.

ScenarioHighlights Principal of Riverview High School John Burton’s dilemma

Two Years Ago NOW • Appraisal system • Aim :to improve teaching and learning • Safeguard :negative evaluations of individual staff performance would not be reported to the senior management team. • Education Review Office • reported: • appraisal system failed to give the management information on the performance of staff.

Staff were comfortable Staff were honest with their chosen appraiser Allow staff to identify professional development goals that would impact positively on children’s learning Staff has the authority to report and to report what Good points of the old system

John Burton’s Dilemma How to make the appraisal system more structured and accountable while simultaneously supporting my staff to challenge themselves professionally and to be open about identifying their professional development needs.

A Revised Appraisal System • Guidelines: • Staff could choose their own appraiser • limited to themes determined by the SMT • observe the work of another staff member in an area in which they wanted to improve • Outcomes: • 40% of staff returned the questionnaire on this revised system • Some staff were critical and some made very positive comments. • 60% of staff ignored the appraisal guidelines

Do I have the authority to check up on those who have not responded to the questionnaire? • What I can do about those who had not participated?

Staff’s desire • non-threatening scheme that emphasized professional development • Management’s • concern • enhance teaching and learning • introduce a more accountable system that mettheir reportingrequirements

Theory-in-use Constraints Practices Consequences SMT suggests what should be appraised & requires classroom visits Entire process remains confidential Self-appraisal emphasized Peer support encouraged & individuals select own appraiser Enhances teaching & learning Equality of authority Non-judgmental & non-threatening Some staff positive, others critical Participation rates & impact of appraisal unknown Principal unable to fulfil reporting requirements

The Intervention Process By the School of Education, University of Auckland Invited by the School Management Team of Riverview High School

Aims of the Intervention • to assist Riverview High School to develop an effective performance appraisalsystem

Methodology : Problem Based • Examined the way Riverview solved the problem by discovering • the constraints they set on the solution • the strategies they used to integrate those constraints.

Data Collection Methods • 3 interviews with the senior management staff • 6 interviews with other staff • 2 feedback meetings with the senior management team • Observations and audio transcripts of both staff and departmental meetings. • Review relevant documents, e.g. appraisal policy

How to implement • The assistance comprised of 8 workshops • focusing on the adequacy of current practice • formulation of a new set of constraints • developing appraisal practices consistent with those constraints • interpersonal skills training • enabled the SMT to debate the constraints that led to an integration, rather than opposition

The First Phase • To identify the values, beliefs and practical issues that underlay current practice in order to determine what needed to be revised and what could be further developed.

The second Phase • Formulate a new set of constraints which enabled the SMT to make explicit their requirements. • Five nominated constraints were • made a difference to teaching and learning • linked to professional development • respected and supported staff • held all levels of staff accountable for performance • practical and efficient

The Third Phase • Meetings of staff • answer questions • staff learnt that the problems are difficult because they had multiple requirements with considerable tensions between them. • Continued to debate and resolve questions

The Last Phase • To en-skill the senior management team to articulate the tensions between their proposed constraints and debate their resolution with their staff.

Two full staff meetings 11 weeks later • Both meetings devoted to appraisal • Showed a shift in the staff’s thinking and the quality of the debate

First meeting • The Principal made no mention of the list of constraints as some separate points but an interacting whole • Staff discussed selected constraints that were of concern to them • The trained SMT explained that making a difference in the classroom was the key factor but the allocation of staff development was a link only

Second Meeting (11 weeks later) • Staff began to acknowledge • the tensions between the constraints and to search for ways to integrate them • they were concerned with the relationship between appraisal and other school practices • how aspects of the proposal met two or more of the proposed constraints • Both staff and SMT deepened their understanding of the integrated nature of the appraised problem

Evaluation of Riverview practice • Enabled the senior management team to : • Determine what was desirable • Determine what was problematical • Identify the values and beliefs that sustained it • Enable new policy to be formulated in addressing current constraints • Determine what should be retained and revised

Implications for National Policy Formulation of a new constraint set at national level would encourage policy makers to identify: • The purpose of appraisal • The values guiding its introduction • The practical problems • What practices are ruled in or ruled out

Skills required for improvement • Demands new interpersonal skills • Satisfy a complex constraint set under considerable tensions • Requires skills to confront and debate the merit of competing perspectives • Requires co-operations • Requires the commitment and skills to develop a shared understanding

How policy implementation could be improved ? • Integrating the constraint set as a whole • Testing proposed constraints by developing exemplars of practice • Revising either the constraints, the practice or both • Consulting with practitioners and interest groups • Formulating new approach – Portfolio ?

Teacher Learning Through Portfolios

Background behind Using Portfolio for Professional Development • Charles Handy (1996) • Incremental change becomes discontinuous change in the Information Age • Under Incremental change: learning by following preceder’s footsteps • Under discontinuous change: learning becomes the voyage of exploration, of questing and experimenting

Teaching Portfolio • Can be seen as a voyage of exploration, of questing and experimenting • Definition by Wolf & Dietz (1998): “A teaching portfolio is a structured collection of teacher and student work created across diverse contexts of time, framed by reflection and enriched through collaboration, that has its ultimate aim the advancement of teacher and student learning.”

Why portfolios are a good idea for PD (I) • The learning portfolio documents • not only the teaching experience • but also, the reflection and enquiry which produces professional learning

Why portfolios are a good idea for PD (II) • Groundwater-Smith (1999): “… is underpinned by the idea of the scholarship of practice” • Gibbons et al (1994): practitioners are “active agents in the definition and solution of problems as well as the evaluation of performance.” • Grant & Huebner (1998): “… to have significant impact on the development of teacher knowledge and professional practice.” • Frid & Reid (1999): “…(being) the formation of professional subject – a ‘teaching self’”

Purposes of Portfolios • Learning portfolio: promote teacher reflection and ownership over the learning process. • Assessment portfolio: presents administrators with information about a teacher’s effectiveness. • Employment portfolio: provides prospective employers with information about a teacher’s suitability for a position.

Process of Portfolios • It is an extended period of time rather than a specific event. • There are different processes • To be successful, we need: a good deal of structured support for the individuals concerned, such as: • Teacher release time, • Resources for participating in professional development activities, • Timetable flexibility for participating meetings, • Teachers’ willingness to collaborate on the purpose and the process of the portfolio

Content of Portfolios • Different purposes and processes will require different kind of content • More than a collection of artifacts for teaching or professional activities: • Retallick & Groundwater-Smith (1996): “…(to be) a compilation of evidence which demonstrates the acquisition, development and exercise of knowledge and skills in relation to your work practice. • Wolf (1996): “… (to be) a set of accomplishments attained over an extended period • Dietz (1993): “… to be dynamic and changing as the learner experiences discoveries that lead to new directions and activities”

Assessment of Portfolios • Depending on the purposes, • For self-reflective professional learning • Informal assessment (such as self assessment, Peer evaluation process) • For other purposes (such as for promotion, employment): • Formal assessment (also known as high-stakes assessment): a higher level of monitoring

Two problem areas for assessment • Comparability • Subjectivity

Two problem areas for assessment (I) • Comparability: • Difficulty arises from the wide range of approaches adopted by different persons as each portfolio is on individual basis. • Suggestion: stipulating certain core items to be included • Problem again: what are these items? Who makes such decision?

Two problem areas for assessment (II) • Subjectivity: • As the evaluation is qualitative rather quantitative, the question becomes how to make the evaluation reliable and valid. • Suggestions: • A panel of assessors to integrate different views and to avoid bias, • A common understanding of the set of assessment criteria used.

University accreditation: Formal assessment • More universities are changing their policies on credit to allow students to submit a portfolio of their on-job learning for credit purposes • This indicates an acceptance of the portfolio approach

Summary of advantages by portfolios • (by Edgerton, Hutchings & Quinlan, 1992) • capture the intellectual substance and ‘situatedness’ of teaching • encourage teachers to take important new roles in the documentation, observation and review of teaching • …. to use these new roles as powerful tools for improvement; and, • … are forging a new culture of professionalism about teaching

Areas of tensions • Multi-purpose nature: for self-reflection, for assessment (comparability and subjectivity) etc. • Resources allocated: time allocated to the process, teacher’s timetable flexibility etc • Teacher’s willingness: may largely hinges on the above areas.

Local context • The Hong Kong Institute of Education conducted the study “Levels of Information Technology (IT) competency, core course elements and assessment tools for teacher training in IT in Education” in 1999

1. The Problem of Policy Implementation: the case of performance appraisal