Quality assurance – how to monitor our (school)exams? 28th March 2011 Anne Beeker

1. Quality assurance – how to monitor our (school)exams? 28th March 2011Anne Beeker

2. Reasons for choosing this particular workshop? Please raise hands if it is due to.... • new pass / fail system > consequencesforposition of English in the curriculum? • management putting pressure on youtominimize gap between SE and CE results? • wishtoimproveyourownexam ‘habits’? • wishto meet CEFR requirements? • otherreasons?

3. Today: central issue - quality assurance - how is that achieved? Receptive skills: basically no problem Cito – listening & reading Productive skills: writing and speaking more problematical... How many use Cito tests for productive skills? How many usetheirown school tests?

4. Survey among teachers of English 2009 • Survey amongteachers 2009/2010 (English) • A few results: • -Do youcreate a ”key” (antwoordmodel / beoordelingsmodel) prior to setting a test? • often: 11 sometimes: 157 never: 41 (209 respondents) • -do youcompare (your) school examstothose of colleaguesfromother schools? • often: 6 sometimes: 74 never: 129 (209 respondents) • Conclusions......???

5. Quality: when is a test good? The big five... • Valid: • Official attainment targets (eindtermen) are tested • Reliableandobjective: • Officiallyagreed model of assessment • Criteria for assessment lead toobjectiveinterpretationby assessors • Scoring system determined in advance • Sufficientnumber of test-items • Minimally 2 assessors • Test results not open to coincidence/ should be consistent

6. When is a test good? (2) • Transparant: • It tests what students have practised • Students know in advance howtheywillbetestedandhowtheywillbeassessed • Workable: • The degree of difficulty of the test is relatedto the amount of time that pupils should have spent on the subject. • Feasible: • Test questions are clearlyformulated – no room formisinterpretation; the test has a good lay-out. • Sufficient time is allocatedto make/do the test in.

7. Where to start? What does the exam syllabus prescribe? (today only time for one skill...) Please write down the official ‘eindterm’ (attainment target) for ‘schrijfvaardigheid’ (writing) English...

8. Examination syllabus • Domain D: Writing • Sub-domain D1: Language skills • 5. The candidate can: •  react adequately in written contacts with users of the target language; •  ask for and provide information; •  adequately present information gained with an eye to the goal and the audience, and in doing so describe things or persons, and express his/her feelings and viewpoints. • Sub-domain D2: Strategic skills • 6. The candidate can: •  write a text with the help of a word processor; •  draw up texts with the help of (electronic) reference materials.

9. Or in Dutch..... • Domein D: Schrijfvaardigheid • Subdomein D1: Taalvaardigheden • 5. De kandidaat kan: • - adequaat reageren in schriftelijke contacten met doeltaalgebruikers; • - informatie vragen en verstrekken; • - verworven informatie adequaat presenteren met het oog op doel en publiek, en daarbij zaken of personen beschrijven en uitdrukking geven aan gevoelens en standpunten verwoorden; • - een verslag schrijven in de betreffende taal. • Subdomein D2: Strategische vaardigheden • 6. De kandidaat kan met behulp van: • - een tekstverwerkingsprogramma een tekst schrijven; • - (elektronisch) naslagmateriaal teksten opstellen.

10. But how do we define ‘schrijfvaardigheid’, i.e. writing as a skill? Schrijfvaardigheid is de vaardigheid tot het schriftelijk produceren van een samenhangende en afgeronde tekst in vrij te kiezen formuleringen. (definitie gehanteerd door L. Melse in zijn proefschrift Schrijftoetsen voor de MVT, van Walraven, 1989) i.e. writing is the skill to produce written work in a consistent and well-rounded text formulated in words of one’s own choosing

11. Bloom’s Taxonomy

12. Lower and higher thinking skills Bloom’s taxonomy CREATING: generating new ideas, products, or ways of viewing thingsDesigning, constructing, planning, producing, inventing. EVALUATING: Justifying a decision or course of actionChecking, hypothesising, critiquing, experimenting, judging ANALYSING: Breaking information into parts to explore understandings and relationships.Comparing, organising, deconstructing, interrogating, finding. APPLYING: Using information in another familiar situationImplementing, carrying out, using, executing. UNDERSTANDING: Explaining ideas or conceptsInterpreting, summarising, paraphrasing, classifying, explaining REMEMBERING: Recalling informationRecognising, listing, describing, retrieving, naming, finding Higher-order thinking

13. Workshop: In groups (of four/five) • Cards withpossibleassignments • Which appeal tolower thinking skills, whichtohigher thinking skills? • Whichsuitablefor havo? Why? • Similarly, whichfor vwo? Andwhy? • CEFR related? Ifso, how?

14. Outcome of groups discussions

15. SLO-project / Kennisnet • Project SLO: started in 2009 • Aim: to offer support forqualityproof‘’PTA’’ – suggestionshowtoimprove test cycle • Tocreate awareness wide range of possibilities • Checklists forspecific subjects nowavailable • Using principle – “tick off list’’ • www.schoolexamensvo.nl