Improving Assessment Practices through In-service Teacher Education

1. Improving Assessment Practices through In-service Teacher Education Case Study of Master of Education Students

2. LITERATURE Sinha (p.247) comments: ... the shortcomings of the traditional examination system: memorization was over-emphasized, coverage of the pupil growth process was limited, mercurial subjectivity was omni-present; inadequate use was made of test results; and receptivity to reform and change was discouraged. The traditional situation not only nullified all the criteria of a sound evaluation system, but also encouraged the adoption of slipshod methods of teaching and learning. Besides, it provided wide openings for malpractices to enter.

3. In Pakistan, Farooq (1994, p.17) comments that, “Too much importance is given to scores in academic examinations. As a result, there is enormous corruption in the examination system. The students, teachers, parents and heads of institutions are equally responsible for this situation.”

4. Bethel et al (1995, p.1) state: Many question papers contain errors in subject content, language, and technical construction. In addition, they focus on a narrow range of low-level skills and are dominated by the content of the approved textbooks. In consequence, the examinations have a negative effect on the educational process in Pakistan.

5. Alternative approaches to traditional assessment The transition from traditional instructional and grading practices to an authentic system of assessment represents a significant educational transformation. During this transition, classrooms are changing from a teacher-centered testing culture, where students work individually and learning is done for the test results, to a collaborative assessment culture, where assessment takes many forms, reaches multiple audiences, and distinctions between learning and assessment are blurred (Barron & Boschee, 1995, p.77).

6. Sinha (1993) states that, “Reform is invariably a difficult task to accomplish, because the traditional procedures and practices have become deeply rooted in the system.”

7. In the process of examination reform, the following hurdles have been encountered: • 1. Inherent resistance to change: • a) by the State Departments of Education and Boards of Secondary Education. This is because examination reform involves a re-organization of the administrative machinery and procedures. • b) by the teachers, because of their unpreparedness for taking up the challenges of the new system and because of additional work involved. • c) by the students, as post-reform examinations are likely to become valid and reliable, and will require more precise and regular study.

8. C0NTINUED • d) by unscrupulous, elements, since malpractices are likely to lose ground as examinations are improved. • e) by vested interests which are desirous of maintaining a status quo which protests their powerful positions and (sometimes) financial gain. • Lack of suitable sample evaluation material of a sort that provides illustrations of the nature of reforms envisaged. • 3. Lack of financial support, as the reform measures, in terms of over-all cost, tend to make the examinations more expensive than the traditional ones. • 4. Inadequate training of the teachers for keeping up with the changed situation.

9. CONTEXT • Aga Khan University, Institute for Educational Development in Karachi. AKU-IED was established in 1993 in the context of “a continued deepening decline in the quality, effectiveness, relevance and outreach of education systems in Pakistan and elsewhere in the developing world in the face of growing numbers of children and shrinking resources” ( A Proposal to the Board of Trustees, AKU-IED, 1991, p.6).

10. Grand Question • In what ways do students reconceptualize assessment practices and what are the implications for subsequent professional practice?

11. Subsidiary questions • What have been the course participants’ (CPs) previous experiences with assessment practices? • To what extent and in what ways have CPs reconceptualized their views on assessment while at IED? • What are/were the factors that help(ed) to reconceptualize (if any)? • What assessment practices do they use / plan to use in their work contexts? For what reasons? • What barriers/ constraints have they encountered or do they expect to encounter? What strategies do they identify for addressing such barriers and constraints?

12. SAMPLE • Total: 28 people • 1) 21 current and former course participants • 2) seven faculty members

13. FINDINGS: ASSESSMENT PRIOR TO IED • The purposes of assessment were primarily to promote the students to the next class or to select them for further education, thus it adds to the importance of the final examination, the result of which would decide the future of a person. • The main requirement for assessment was memorization (rote learning) or covering material limited to the textbook or teachers’ knowledge. • Examinations and tests were often the only forms of conducting assessment. • These forms of assessment generally promoted and required low-order thinking among students, with little if any emphasis on understanding.

14. - CONTINUED • Assessment usually came at the end of a term, unit, half year or year, thus it was normally of a summative nature with little or no meaningful, constructive and formative feedback for the improvement of learning. • Formal assessment was the basic feature of assessments; informal assessment was done very little and mostly not systematically. • The type of assessment experienced promoted different kinds of malpractice such as cheating, leakage of question papers, taking bribes or other unfair means. • Students are assessed individually but on a normative basis. They viewed their peers as opponents with whom they had to compete.

15. - CONTINUED • There has been ‘ranking’ or ‘labeling’ of the students as a result of assessments. • The teaching and learning process was driven by the exams, thereby creating what the literature terms the ‘washback effect’(Harris & McCann, 1994). • Competitive marking negatively affected the classroom environment; that is, students did not help each other. • The examination system was taken for granted rather than being questioned. • Tension and anxiety appeared to accompany examinations, partly because examinations were not adequate to assess within three hours what has been learned in a whole year, thereby increasing stress.

16. ASSESSMENT AT AKU-IED • Reconceptualization of Teaching and Learning • Identifying existing beliefs and practices. • Challenging beliefs and practices. • Exploring alternatives. • Mediating with context. • Making choices. • Planning for improvement. • Executing plans. • Reflection on improvement.

17. Reconceptualization 1) Awareness Being aware of the alternatives to teaching and learning. 2) Dissatisfaction In order to undergo reconceptualization, the CPs become dissatisfied with the position that they are coming from. 3) Buying into a new paradigm The CPs need to ... go back to equilibrium. They have to replace their old conceptions with a more powerful conception which they really believe in. They have to explore the options thoroughly, they have to discuss, rigorously with others. 4) Application or testing Once they accept a new paradigm, they have to go out and test it. They have to be really convinced that this new paradigm is more powerful, and it works in the new setting. Only then they will be ‘truly reconceptualized’, and the new paradigm cemented.

18. Assessment Practice at IED • No summative forms of assessment • Written assignments • group work • practical mini-activities • class participation • reflective journals • self assessment • peer assessment • presentations. • Teaching practica

19. A typical breakdown of assessment criteria: • class participation 10 to 20 % • major assignment(s) 40 to 60% • presentation 20 to 40% • practical activities • (e.g. micro-teaching, workshops) 20 to 40 % • reflective journals 0 to 20% • peer assessment no weighting

20. The purpose is to diagnose the students’ strengths and weaknesses, provide information on the their progress, give feedback to them, parents and teachers on teaching and learning process. • Teachers also get feedback on their teaching and curriculum implementation, then make modifications if necessary. • Assessment is seen as integral part of teaching and learning, not an isolated component. • Assessment may be measuring not only memorization, although that is necessary too, but also understanding, synthesizing, application and evaluation levels of taxonomy.

21. All three domains (affective, psychomotor, and cognitive) should be assessed. • There are multiple tools and ways of gathering information for assessment, not only restricting oneself to examination and tests. It can also improve the quality of tests and examinations. • Assessment gives meaningful formative feedback for the improvement of students’ learning. • Informal assessment is used together with formal assessment. • To eliminate or reduce malpractices while assessment is taking place. • Assessment is seen as a cooperative venture; students do not compete.

22. They may be assessed against the criteria developed by teachers and students together. • The students are seen as people of different multiple intelligence, and not given an oversimplified label such as ‘intelligent’ or ‘dumb’. • Assessment is done to question, reflect and develop assessment practice. • Assessment is done to reduce negative psychological effects of assessment that might cause alienated behavior while assessment. • Assessment is to make the students aware of the fact that grades are not the end of everything.

23. Espoused and actual assessment practices • Director of IED: • Many students had endorsed other than traditional ways of assessment because of IED’s impact. However, in some respects, IED’s actual practice in assessing the M.Ed. students seemed to reflect more traditional assessment assumptions than its own teaching recommended or advocated. Consequently, in a number of cases the students reverted to behaviors and attitudes associated with traditional assessment, including those students who accepted the notions of alternative assessment in principle or theory. As a result, the IED has been experiencing a paradoxin which espoused beliefs contrast with practical realities.

24. The impact of grading • Half way through a course they stopped giving us grades, they said that grades are causing the tension. Maybe some people must have gone up to them (administrators), with some complaints, some dissatisfaction. • We were given grades for two or three modules and then they stopped giving grades to us after module X. There were a lot of furors. People were very upset over certain things. Some people said that tutors were biased against them and there was a lot of commotion. So it was decided among the faculty members that no grades will be given.

25. The impact of grading • When we got grades, I think the people become frustrated. Division has started. Division means categories. Some people who got A they are in A category, who got B are in B category, some were in C. So it was observed and felt that this grading is affecting collegial attitude. And it is not good for cooperation, for cooperative learning. Then they did not share the grading with us. But assessment was there, we would get only the feedback. • There was no objection to grading, the objection was the criteria. If someone does not follow the criteria, then there is conflict. I heard remarks as “he was not so good, but got B, and person X was so good, worked really hard, but got C grade”.

26. At the time of assignment I think the colleagues would usually hide, cover page of the books, not help each other. • ... nobody wants to see C (grade), it was personal attack. It was like harsh blow to them [the students who got poorer grades]. “Oh I got C when I am teaching so many years.” It was more of a shock to them rather than a learning experience. • I don’t know if IED can do away with grades. I feel that what they have said if you get four As and in dissertation A, then you will get a distinction. They are going against policy. They say that (grade) does not matter. How can you say if you get an A you will get a distinction, and don’t think about grades? It is a mismatch.

27. In one of our modules a lot of us got A grades. Then director came and said it was not the right thing. “The tutor who marked was not really paying attention.” They said that it is wrong, it should be bell curve... they were not happy why everybody was getting an A.

28. Faculty members’ reactions • What happened in our first M.Ed. programme, we shared assessment policy with the students and we had been doing things, but unfortunately in the middle of the programme there was a problem. People got some frustration with the assessment marks. We thought that since IED's purpose is to flourish, nurture the environment of the collaboration and collegiality. We thought perhaps collaboration and collegiality are going to be defeated. So we made a decision and stopped distributing grades. • We had been debating in faculty whether we should disclose or not, because our first experience was not very good. There is still a problem, some people are not happy. But it is a natural process. • I favour telling people their grades, and building enough support to help them to cope with, know that they can do better about it. • I don’t think that people are graded on what they learn from the books only. We are not assessing on their English, we are assessing on their thinking, what they do.

29. If somebody talks a lot in the class, does not give other people a chance, does not really listen to what other say, how do we assess? • If I had my choice, I think I would do away with the grades. I think they are in some sense a deterrent to learning at IED, because they represent in an area IED is trying to get away from; increased weighting, emphasis on the actual grades. Because we know in education that we cannot measure quantitatively many of the things that we are grappling with at IED to the way that people think we can. • ... the CPs when they were students performed extremely well. ... here people were getting B and Cs, they just could not stomach that, they felt that it was below their dignity... We had to council them that it is not the letter grade that really matters, it is their ability to function as teacher-educators in the end, which will really tell us exactly how successful we have been in helping them to become good teacher-educators.

30. What does that achieve by hiding books. I don’t see how hiding your books can get you more excellent marks... It is just a very spiteful, it is a very negative way to behave. I don’t know what word to use here... It is displaying a lack of good feeling, lack of trust. All sorts of qualities that we want to build in professional relationships is denying all of those. ... am I so arrogant that I think that only my ideas are important? I can learn so much from hearing from people’s ideas whoever they are. I think I need to talk to some of the people who are saying these. I should get very angry. • I don’t believe that grades are not important. I think they are important. I understand how people feel what are grades. My view is that we should not hide the grades... But some of my colleagues disagree with me. Because they know how hurtful it is to some people not to get A grade.

31. Assessment after IED • Barriers to assessment practices • parents • head-teachers or management • school facilities (classrooms, furniture, resources) • time factor and large classes • Board / NationalExaminations • colleagues • malpractice (Cheating, leakage of papers, setting papers, etc.)

32. Faculty member’s concern • There is a danger that IED graduates might revert to their old ways of assessment. Those people who end up teaching higher classes, they might go back whether they like it or not. If they are teaching class which has got Board Examination, surely they will have to revert. In Pakistani schools, tests mean a lot. They can’t change the school overnight. When you are trying to change the school, the school is also trying to change them.

33. Implications • To bring assessment to centrality of education reforms • Faculty development in the area of assessment • Conduct courses on assessment at IED • Education issues on assessment with various stakeholders (heads of schools, District Education Officers, University teachers, school teachers, parents, etc.)