B ureau for I nternational L anguage C oordination

1. Bureau for International Language Coordination Julie J. Dubeau BILC Secretary Istanbul, Turkey May 24, 2010

2. Outline of Presentation • General information • LNA report • BAT Report

3. What does BILC do? Annual Conference in Spring June 2009 hosted by Italy - “Bridging the Gap: Language Requirements vs. Language Reality” May 2010 hosted by Turkey - “Mapping The Road: Success in Language Training” Professional Seminar in Fall October 2009 hosted by Denmark – "The 21st Century Classroom: Keeping up with the Times!“ October 2010 hosted by Bulgaria – “Aligning Training and Testing in Support of Interoperability”

4. Ratified by the nations and promulgated by NSA in February 2009 English and French versions can be downloaded from the BILC website www.bilc.forces.gc.ca Steering Committee will be discussing a proposal for admin change during conference Responsible for STANAG 6001

5. What else does BILC do? • Language Training Assessments • Assistance to National Testing Programmes • Language Testing Seminars: • LTS & ALTS • Special Projects: • Benchmark Advisory Test (BAT) • Language Needs Analysis (LNA)

6. General Information ACT is tasked by IMS to investigate the feasibility of introducing a computer based learning tool for use by nations to familiarize with NATO terminology… BILC is investigating requirement through JSSG

7. Complementary ADL SolutionsIMS Report Dated November 26 2009, Bratislava Follow-up Tasking on Counter Insurgency (COIN) • “ACT should develop ADL solutions that will complement nations’ efforts in language training, understanding of different local government structures and understanding familial, clan and tribal cultures”. • ACT has requested BILC’s assistance • Please contact the BILC Secretariat if you are interested in providing an ADL product

8. LANGUAGE NEEDS ANALYSIS OFNATO CRISIS ESTABLISHMENT (CE) POSTS • November 2008 Chairmen’s Meeting of the NTG in Brussels • The aim of this study was to show whether language requirements appeared to be set at appropriate levels to enable military personnel to perform their duties adequately in the NATO OPS context, in this case ISAF.

9. LANGUAGE NEEDS ANALYSIS OFNATO CRISIS ESTABLISHMENT (CE) POSTS • To broadly identify whether the mandatory SLP requested for the post was: • At Level ~ A: meaning that from the tasks described in the JD, the profile appeared to be adequate; • High ~ H: meaning that the SLP requested for the post appeared to be higher than the functions defined in the job description; or • Low ~ L: meaning that the SLP requested for the post appeared to be lower than the functions defined in the job description.

11. Summary of Results

12. Results: Levels 1 & 2 • Job Descriptions requiring SLPs of Level 1: • 30 JDs required SLPs of 1111 & all were High. • Job Descriptions requiring mixed SLPs of level 1 & 2: • 4 required SLPs of 2211 & all were at level. • 2 required SLPs of 2221 & both were at level. • Job Descriptions requiring SLPs of level 2: • 61 JDs required SLPs of 2222. • 15/61 (26.2%) were at level. • 43/61 (70.4%) were High.

13. Results: Levels 2 & 3 Job Descriptions requiring mixed SLPs of Levels 2 & 3: • 156 (25.6%) JDs required mixed SLPs of Levels 2 and 3. • Of these 156, 66 (42.3%) were considered as having SLPs that were at level Further Breakdown: • 1 required SLP 3221 (High/Low depending on skill) • 9 required SLP 3222 (7/9 High) • 1 required SLP 3223 (High) • 4 required SLP 3232 (3High/1Low) • 118 required SLP 3322 (47 at level (39.8%), 56 High (47.4%) & 3 Low (2.5%)) • 23 required SLP 3332 (17 at level, 6 Lownote that these 6 also req Dari)

14. Results: Level 3 Job Descriptions requiring SLPs of Level 3: • 301 (49.4%) required SLP 3333 • 240/301 (79.9%) were considered at level • 44/301 (14.6%) were considered High • 13 were considered as Low SLPs for functions, (all req expert language knowledge with 3s in Dari and Pashto)

15. Results: Levels 3 & 4 Job Descriptions requiring mixed SLPs of level 3 & 4: • 43 profiles of 4343, from which 22/43 were at level(51.1%), and 21(48.8%) were not scored at level (some H some L) • 1 profile of 4344 (Low), 1 profile of 4443 (High)

16. Results: Level 4 • Job Descriptions requiring SLPs of level 4: • 8 profiles of 4444 (6/8 were considered at level, with comments referring to the functions as requiring expert language ability, diplomatic or sophisticated language use, 2 were H) • Out of the 53 JDs requiring some skills at level 4, 10 (18.8%) also req languages other than Eng such as French, Dari, Pashto, etc. • Out of these 10, 6 (60%) were at level.

18. Recommendations • NATO CE Post JD SLPs should be reviewed • SLPs should be based on an analysis of the tasks performed by incumbents in relation to STANAG 6001 Ed 3. • It is strongly recommended that the next analysis of requirements be done with the assistance of BILC language experts. • CE posts master sheet should include the SLPs requested for the posts.

19. What’s next for interoperability? • IMS-Tasking on Language Proficiency and Education on NATO Standards & Terminology. 12 February 2010 • Analysis of CE/PE posts • Setting realistic requirements • Review of linguistic requirement for Officers for operationally deployed duties‘ has been initiated • Benchmarking

20. Benchmark Advisory Test (BAT)- PURPOSE To provide an external measure against which nations can compare their national STANAG test results To promote relative parity of scale interpretation and application across national testing programs To standardize what is tested and how it is tested

21. Benchmark Advisory Test (BAT) • Allocation to 11 Nations (200 ‘free’ tests) • Tests administered by LTI, the ACTFL Testing Office via the Internet and Telephone • January 2010 – End of Benchmarking Process • Positive Feedback from Nations

22. Benchmark Advisory Test (BAT) Results

23. FACTORS CONTRIBUTING TO NON-ALIGNMENT • Test purpose • Testing method • Use of plus ratings • Alignment of author purpose, text type, and reader/listener task • Inadequate tester/rater norming (productive skills) • Inconsistencies in the interpretation of STANAG 6001 • Cut-off score setting, etc.

24. BAT – Way Ahead • Project successful • Demand for administrations will dictate future development needs and modes • BILC SC to formulate recommendations • Nations who require BILC support (post-BAT) can request it

25. QUESTIONS? Enjoy the Conference!