ALIGNING TEACHING AND TESTING IN BULGARIA IN SUPPORT OF INTEROPERABILITY

1. ALIGNING TEACHING AND TESTING IN BULGARIA IN SUPPORT OF INTEROPERABILITY

2. ENGLISH LANGUAGE TRAINING FOR THE BULGARIAN ARMED FORCES Mission: To provide quality English language training to military and civilian personnel for the needs of the Bulgarian Armed Forces.

3. ENGLISH LANGUAGE TRAINING SYSTEM2006-2010 • Strategy for Developing the English Language Training and Testing System for the Period 2006-2010

4. 2006-2010 • ELT curricula for cadets in National Military University (NMU) and Naval Academy (NA) standardized • ELT curricula for officers in Rakovski National Defence Academy (NDA) • content of the General English Intensive Courses (Levels I-IV) standardized • course entry and exit requirements and student assessment standardized

5. 2006-2010 • Foreign Languages Department (FLD) in the town of Shumen established • 5 Self-Access Centers established (NMU, NA and NDA) • Center for Language Training and 4 language labs established in the units with the help of the Kingdom of the Netherlands • new courses (Specialized English, refresher, distance learning) and materials for SACs developed

6. 2006-2010 • through FMF and IMET programs: • 421,000 USD worth of ALC materials • language labs (incl. upgrade and maintenance) – 14 labs seating a total of 245 students in the military academies, 3 labs in the units • Mobile Training Teams - ELT advisor from DLIELC in the FLD in Shumen • teacher training in the US and Canada

7. 2006-2010 • ELT program coordinator in MoD • coordinators in the staffs of the three services • commission on ALCPT use (+ test control officers locally) • STANAG 6001 testing team under the MOD • annual seminar attended by all parties involved in the ELT process

8. English Language Training for the BAF

9. ENGLISH LANGUAGE TRAINING in NMU, NA and NDA NMU and NA • cadets • 700 hours of English language training • homogeneous groups based on ALCPT scores • exit requirements - minimum SLP 1111 upon graduation. Rakovski NDA • officers doing an MA degree in Military Science • 760 hours of English language training • homogeneous groups based on ALCPT scores • exit requirements - minimum SLP 2222 upon graduation.

10. English Language Courses

11. English Language Courses – ctnd. • Student screening and selection • Student assessment - BQs - ALCPTs (screening, mid-course (5/8 week), final) • Certificates • End-of-course evaluation report • Disenrollment

12. Course graduates (Jan 2009-July 2010)

13. SLPs received by students within 6 months of graduation (% students)

14. STRATEGY 2011-2015:Main Points To Be Considered • cadets – min SLP 2222 upon graduation as of 2012/2013 • NDA officers – min SLP 3332 upon graduation as of 2013/2014; admission requirements for language skills - min SLP 2222 • mandatory STANAG 6001 exam after completing a course • DL courses and module-based courses designed for the needs of the deployable forces • Level IV course content to be revised • revising course length for all levels

15. Qualification Courses Taught in English • specialized qualification courses for commanding officers with modules taught in English • academic staff teaching Military Science in military academies and Rakovksi NDA – min SLP 2222

16. PHASE TWO - TESTING

17. They are native speakers but are they interoperable? A COLONEL ISSUED THE FOLLOWING DIRECTIVE TO HIS EXECUTIVE OFFICERS: “Tomorrow evening at approximately 2000 hours Halley’s Comet will be visible in this area; an event which occurs only every 75 years. Have the men fall out in the battalion area in fatigues, and I will explain this rare phenomenon to them. In case of rain, we will not be able to see anything, so, assemble the men in the theater and I will show them films of it.” EXECUTIVE OFFICER TO COMPANY COMMANDER: “By order of the Colonel, tomorrow at 2000 hours, Halley’s Comet will appear above the battalion area. If it rains, fall the men out in fatigues, then march to the theater where this rare phenomenon will take place, something which occurs only once every 75 years.” COMPANY COMMANDER TO LIEUTENANT: “By order of the Colonel, be in fatigues at 2000 hours tomorrow evening. The phenomenal Halley’s Comet will appear in the theater. In case of rain in the battalion area, the Colonel will give another order, something which occurs once every 75 years.” LIEUTENANT TO SERGEANT: “Tomorrow at 2000 hours, the Colonel will appear in the theater with Halley’s Comet, something which happens every 75 years. If it rains, the Colonel will order the comet into the battalion area.” SERGEANT TO SQUAD: “When it rains tomorrow at 2000 hours, the phenomenal 75-year-old General Halley, accompanied by the Colonel, will drive his comet through the battalion area in fatigues.”

18. SUCCESSIVE AND SUCCESSFUL TESTING • Successive – comes after language training: • the cadet graduates of “V. Levski” NMU and “N. Vaptsarov” NA, as well as the officers who graduate “G. S. Rakovski” NDA; • the military and civilian personnel who complete English language courses in military educational institutions.

19. SUCCESSIVE AND SUCCESSFUL TESTING (cont.) • Successful • tests should be reliable and valid.

20. Reality 1. Finances are limited. 2. There are 1200-1300 test candidates per year. 3. There are 5 testing sites. 4. We develop 324 reading items, 324 listening items, and 60 writing prompts each year. 5. The testing team consists of 5 experts.

21. Bulgarian English Language STANAG 6001 Testing Sector • One test team is part of the Ministry of Defense, Human Resources Management Directorate. • The same testing team, develops and administers the STANAG 6001 tests, issues the certificates and is responsible for the data update in the Personnel Automated System.

22. Bulgarian STANAG 6001 Testing Team

23. The Importance of Being Earnest How do we strive to ensure objective testing ?

24. All STANAG 6001 documents published on the Internet sites of MoD, NMU, NA and NDA. STANAG 6001 Specifications widely spread. Example tasks for reading, listening and writing on the Internet sites. Three guidebooks for self preparation. Clear rating procedures.

25. Objectivity in Reading • The papers are anonymous. • Multiple-choice items only. • 18 items per each level. • 70% pass rate. • The three versions of the test are piloted on one and the same group of students and the results should be closely compatible.

26. Objectivity in Listening • The papers are anonymous. • Multiple-choice items only. • 70% pass rate. • 18 items per each level. • The three versions of the test are piloted on one and the same group of students and the results should be closely compatible.

27. Objectivity in Writing • The papers are anonymous. • Always 3 tasks in each test version. • Rating Scale for Writing. • Before each STANAG 6001 exam session – norming rater-refresher sessions. • Two raters read the papers separately and, if they give different levels, the paper is read by a third rater. • After each session – raters reliability analysis.

28. CAT

29. Objectivity in Speaking • Before each STANAG 6001 exam session – norming rater-refresher sessions. • Rating Scale for Speaking. • The oral interview is taped. • There are two testers in one testing board. • There is a strict scenario for the different stages of the interview. • After each session – statistical analysis of raters reliability.

30. What does “our English face” look like?

31. Diagram of Levels

32. Levels 2/2-3/3/3-4/4 Certificates Distribution

33. STT efforts = success Students + Teachers + Testers =