LEVEL 4 IAW STANAG 6001 - A CONCEPTUAL MODEL -

1. LEVEL 4 IAW STANAG 6001- A CONCEPTUAL MODEL - Jana Vasilj-Begovic BILC Secretary/Canada Gerard Seinhorst Head, Language Office, Defence Institute Security & Intelligence, Netherlands

3. SCOPE OF PRESENTATION • Background • Methodology • Conceptual Framework • Implications for Testing • Way ahead

4. Background • BILC conference 2010 – Istanbul • Formation of level 4 WG • Members: Bulgaria, Canada, Denmark, Germany, Italy, Netherlands, Shape, Sweden, USA

6. Methodology • Survey and compile literature • Develop a conceptual framework • Find illustrative samples • Discuss testing implications • Provide guidelines for test development

7. Conceptual Framework • Aims to answer the questions: • how to interpret the STANAG 6001 descriptors for Level 4? • what are the typical and shared (linguistic) characteristics of Level 4 language users? • what are the implications for assessing Level 4 language ability? • Purpose: to enhance the common understanding of Level 4 proficiency.

8. Level 4 Characteristics • Linguistically complex • Low redundancy • Multiple meanings in socio-cultural context • Rarely encountered in daily life • Unfamiliar cultural concepts • Highly individualized / sophisticated • interpretation of the layers of explicit and implicit meaning

9. Types of Comprehension • Literal • Inferential • Evaluative • Understanding meaning “beyond the lines” • Evaluating the significance of the message

10. The Level 4 Language User • STANAG 6001: “Expert” • This level of proficiency is typically, although not necessarily and not exclusively, achieved by individuals who use the foreign language extensively to handle intellectually highly demanding issues as part of their profession or specialisation

11. The Level 4 Language User • Very taxing academic and professional settings • Deal with demanding topics at the highest levels • Examples of military-related tasks: • Spokesperson responsible for press releases and press conferences requiring nuanced, culturally appropriate communications • Arbiter between warring factions • Providing official interpretation services • Analyzing the hidden communicative intent of diplomatic pronouncements

12. The Level 4 Language User • Adapts and tailors language to suit the purpose and situation • Understands/expresses subtleties and shades of meaning • Consistently maintain high degree of grammatical and lexical accuracy • Uses metaphors, clichés and proverbs spontaneously • May have fossilized lapses in pronunciation and intonation

13. Significant Differences (1) LEVEL 3LEVEL 4 • Understands/Can produce formal and informal language, for most everyday social and work-related situations. • Understands/Can produce highly sophisticated language appropriate for almost all topics, and social as well as professional situations. • Understands/Can produce language relating to abstract concepts and hypotheses. • Readily adapts language to any situation. Shows a firm grasp of various levels of style and can understand/express subtle nuances and shades of meaning. • Understands/Can produce extended discourse and conveys meaning correctly and effectively. Understands arguments/Can argue for and against different options. • Uses the language with great precision for all social and professional purposes, including persuasion, negotiation, or the representation of an official point of view.

14. Significant Differences (2) LEVEL 3LEVEL 4 • Can understand/express meaning “between the lines”. • Can understand/express meaning “beyond the lines”. • Understands/Demonstrates humour and irony, but may not fully understand some allusions, as well as implications of nuance and idioms. • Good command of idiomatic expressions, figures of speech and colloquialisms. • Rarely requests repetition. Has a natural flow, without searching for words. • Consistently maintains a high degree of grammatical and lexical accuracy. Uses the language effortlessly. • Occasional errors rarely disturb the native listener/reader. • Errors are rare and without pattern.

15. Implications for Testing Testing receptive skills (listening and reading) Tasks • Text type • Sample • Testing techniques

16. Text Type (listening)Adaptation of Clifford’s Text Characteristics

17. Text Type (listening)Adaptation of Clifford’s Text Characteristics

18. Text Type (listening)Non-participatory Situations

19. Text Type (listening)Participatory Situations

20. Text Type (reading)Clifford’s Overview of Text Characteristics

21. SAMPLE LISTENING TEXT • Extracted from the Charlie Rose Show • Topic: Science and Health http://www.charlierose.com/view/interview/12276

22. SAMPLE READING TEXTAN AMORAL MANIFESTO, Joel Marks (Philosophy Now) Hold onto your hats, folks. Although it is perhaps fitting that the actual day on which I sit here at my computer writing this column is April 1st, let me assure you that I do not intend this as a joke. For the last couple of years I have been reflecting on and experimenting with a new ethics, and as a result I have thrown over my previous commitment to Kantianism. In fact, I have given up morality altogether! This has certainly come as a shock to me (and also a disappointment, to put it mildly). I think the time has come, therefore, to reveal it to the world, and in particular to you, Dear Reader, who have patiently considered my defences of a particular sort of moral theory for the last ten years. In a word, this philosopher has long been labouring under an unexamined assumption, namely, that there is such a thing as right and wrong. I now believe there isn’t.

23. Testing Techniques • Alignment of text and task level • Constructed response vs. multiple choice • What line of argumentation does the author present for his relinquishment of morality? • One skill vs. combined skill testing • Rating decisions

24. Implications for Testing Testing productive skills (speaking and writing) • Tasks • Sample • Testing techniques

25. What L4 Speakers/Writers Can Do • Tailor speech to a specific audience • Counsel • Persuade • Advocate a position at length • Elaborate on abstract concepts • Express nuances and shades of meaning

26. Sample of Speaking Proficiency • Task: discuss abstract topic • Testing/elicitation technique (OPI): descriptive prelude • Level of task aligned with level tested

27. Sample of Writing Proficiency • Task: analyse abstract issue (NATO related topic) • Testing technique/writing prompt: descriptive prelude • Level of task aligned with level tested

28. SAMPLE WRITING PROMPT One of the most significant trends in recent decades in the arena of international military and political affairs has been the perceived change in the scope and purpose of NATO commitments and engagements. In the post-Soviet Era, the mission of NATO has apparently shifted from “defending the free world” to “saving the innocent from ethnic cleansing or genocide”—whether at the hands of their own rulers or of neighboring nations—through intervention in an armed struggle on the side of a party chosen on the grounds of moral righteousness and justice. The policy of intervening to prevent genocide or ethnic cleansing is seen as the moral equivalent of humanitarian aid to save innocent human lives.

29. Sample of Writing Proficiency • In the year 2000, the UN published the Brahimi report on peacekeeping and one of its probative findings was that peacekeeping was significantly more effective, when there was regional ownership of the mission. There are several reasons for this, but one of the major contributing factors is that few conflicts can be adequately solved, unless there is a solid regional backing for the peace process. Additionally, peacekeeping soldiers from the same region and culture are generally much more attuned to the cultural particularities of any given conflict zone and as such they may be better able to understand and confront the underlying cleavages in a society torn by war and atrocities.

30. Implications for Testing • Finding appropriate texts • Selecting appropriate testing techniques • Determining proficiency/performance ratio • Sampling widely to elicit a representative/ratable sample of proficiency • Reconciling requirements for wide sampling with those of practicality • Validating (target population) • Training testers/raters

31. WG Status • Paper finalized and disseminated (available on BILC Website www.natobilc.org) • Format and protocol for level 4 testing under consideration • Pilot project of level 4 testing protocol planned

32. QUESTIONS THANK YOU!