Best Practices in STANAG 6001 Testing

1. Study Group #4 Best Practices in STANAG 6001 Testing Peggy Garza Birgitte Grande

2. Best Practices in STANAG 6001 Testing Nations Represented • Lithuania • Netherlands • Norway • Slovenia • Spain • Sweden • Ukraine • United Kingdom • United States of America • Bosnia and Herzegovina • Canada • Czech Republic • Denmark • France • Germany • Georgia • Hungary • Italy

3. Best Practices in Testing • Considered the European Association of Language Testing and Assessment (EALTA) • Guidelines for Good Practice in Language Testing and Assessment • Discussed the uniqueness of STANAG 6001 testing best practices • BILC documents • Self Assessment for STANAG 6001 Testing Programs • LTS materials: Test Development Process • Roadmap to a Validity Argument

4. Four Categories • Test purpose • Test design • Test development procedures • Test administration procedures

5. STANAG 6001 Test Purpose The purpose of STANAG 6001 proficiency tests can be viewed as three inseparable tasks • Assess an individual’s general language proficiencyusing the criteria captured in STANAG 6001’s 0 through 5 proficiency scales, i.e. measure the individual’s ability to consistently complete the real-world communication tasks in the specified situations with the level of accuracy expected in those situations. • Determine which STANAG 6001 level describes the level of sustained ability. • Report that proficiency level to the appropriate stakeholders. • Because the purpose of STANAG 6001 tests is to assess an individual’s spontaneous abilities in frequently-occurring real-world communicative settings, STANAG 6001 tests are different from curriculum-based tests

6. STANAG 6001 Test Design Test specifications should include (not necessarily in the following order): • The purpose of the test, • The target testing population, • What is going to be tested (skills, subskills), • Levels according to STANAG 6001 (single, bi-, multi-level test) • Format of the test (how long, how many texts, tasks, items…), • Criterion referenced testing, • Types of tasks (MCQ, open –ended, matching, writing and speaking tasks…), • The language of tasks (mother tongue, target language), • List of topics, • Authenticity of the materials

7. STANAG 6001 Test Development Test Construction Process General recommendations • Item writers should not work on their own, but in and as a team • Before the team starts developing items, there should be consensus about what and how to test (as laid out in the test specs) • Make sure that each item and prompt measures unrehearsed general language proficiency, and not discrete-point grammar and vocabulary, or curriculum-based performance. Quality control measures • Keep the testing section separated from the teaching section in order to avoid conflicts of interest • Item writers should be properly trained and (re)normed against the STANAG scale • Keep all testing materials secure at all times and provide access only on a need to know basis • Update items regularly to avoid them becoming outdated or compromised

8. STANAG 6001 TestAdministration Procedures Prior to test administration • Familiarization – on MOD websites, clear instructions prior to the exam • To achieve proper planning, incl. accommodation etc. – Is a formal schedule published with the same time, date, and location of the testing sessions? • Are the proctors given an official list of approved names prior to the testing session? • Is there a clear policy to prevent and penalize cheating of any kind? Test administration • Ensure all candidates have the same conditions • Teachers should not administer tests to their own students • Regulations concerning the number of speaking tests conducted each day • All speaking tests should be recorded • Develop security procedures – preventing test compromise, storing of test materials, Item database, archive, prevent the use of electronic devices (smartwatchetc.)

9. Best Practices inSTANAG 6001 Testing Thank you for your input. This is a work in progress. The final product will be posted on the BILC website.