1 / 22

Republic of Bulgaria Ministry of Defense

Republic of Bulgaria Ministry of Defense. Piloting STANAG 6001 CAT Reading and Listening. Plan for STANAG 6001 CAT implementation; STANAG 6001 CAT software; Evaluation algorithms – computer-assisted and computer-adaptive; Piloting STANAG 6001 CAT for reading;

ethel
Download Presentation

Republic of Bulgaria Ministry of Defense

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Republic of BulgariaMinistry of Defense Piloting STANAG 6001 CAT Reading and Listening Emilija Nesheva

  2. Plan for STANAG 6001 CAT implementation; STANAG 6001 CAT software; Evaluation algorithms – computer-assisted and computer-adaptive; Piloting STANAG 6001 CAT for reading; Piloting STANAG 6001 CAT for listening; CAT – pros and cons. Content Emilija Nesheva

  3. Plan for STANAG 6001 CAT Implementation • since exam session Autumn 2013 – STANAG 6001 CAT and pen-and-paper tests; • since exam session Spring 2015 only STANAG 6001 CAT. Emilija Nesheva

  4. STANAG 6001 CAT software • Administrative Module main menus : • Lists • Protocols • Tasks • Tests • Evaluation Emilija Nesheva

  5. 6-9 May 2013 Tbilisi, Georgia Emilija Nesheva Emilija Nesheva 5

  6. 6-9 May 2013 Tbilisi, Georgia Emilija Nesheva Emilija Nesheva 6

  7. 6-9 May 2013 Tbilisi, Georgia Emilija Nesheva Emilija Nesheva 7

  8. STANAG 6001 CAT software (cont.) • Candidates’ module: • Reading Test • Listening Test • Writing Test Emilija Nesheva

  9. Piloting STANAG 6001 CAT reading Emilija Nesheva

  10. Piloting STANAG 6001 CAT listening Emilija Nesheva

  11. Evaluation algorithm computer-assisted test • follows the current Specifications for STANAG 6001 exam for the Bulgarian Armed Forces; • starts with items for L1, then L2, then L3 ; • level is given for min. 70% correct answers; • 50%-69% correct answers for the next level → (+)level. Emilija Nesheva

  12. Evaluation algorithmcomputer-adaptive test • Each candidate is given 7 items for L2 and depending on the correctness of the answers, the test continues moving further to L3 or L1 and again to L2 if appropriate. The pass rate is 70% per level (50-69% for + levels). Emilija Nesheva

  13. STANAG 6001 CAT Piloting READING and LISTENING • Methods of validation: • - statistical analyses of the performance of the participants comparing their performance at a recent pen-and-paper test and at the CAT; • - retrospective written feedback by the participants in the piloting. Emilija Nesheva

  14. Piloting STANAG 6001 CAT Reading Results Emilija Nesheva

  15. STANAG 6001 CAT Piloting Reading Results Emilija Nesheva

  16. Piloting STANAG 6001 CAT Listening Results Emilija Nesheva

  17. STANAG 6001 CAT PILOTING LISTENING RESULTS Emilija Nesheva

  18. Participants’ Feedback • Nearly 90% of the participants in the piloting approve the implementation of STANAG 6001 CAT for Reading and Listening. Emilija Nesheva

  19. Pros: candidates can manage their time and this makes them calm and confident; decreased candidates’ frustration -immediate feedback; -highly objective feedback; -less items are compromised; -testing-time is saved; -new items can be piloted and the statistical calculations can be made automatically. Cons: -due to test method and conditions (e.g.computer screen clarity, motor skills needed to respond to test items, and ability of test administrators to troubleshoot); -due to test-taker attributes (e.g. computer familiarity, ability to concentrate in front of the computer, danger of bad time-management). CAT – pros and cons Emilija Nesheva

  20. NB! Lessons Learnt! Emilija Nesheva

  21. Questions? Emilija Nesheva

  22. May 6th – St. George’s Day Valor Day – Day of Bulgarian Armed Forces Emilija Nesheva

More Related