1 / 23

Challenges in Developing and Delivering a Valid Test

Challenges in Developing and Delivering a Valid Test. Michael King and Mabel Li NAFLE, July 2013. Contents. 1. Integrated Language Education System 2. Basic Concepts in Language Assessment 3. Considerations in Language Assessment – IELTS Approach 4. Q&A.

radha
Download Presentation

Challenges in Developing and Delivering a Valid Test

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Challenges in Developing and Delivering a Valid Test Michael King and Mabel Li NAFLE, July 2013 www.britishcouncil.org

  2. Contents 1.Integrated Language Education System 2. Basic Concepts in Language Assessment 3. Considerations in Language Assessment – IELTS Approach 4. Q&A www.britishcouncil.org

  3. 1. Integrated Language Education System Curriculum Learners Curriculum Delivery Language Assessment www.britishcouncil.org

  4. 2. Language Assessment: Basic Concepts www.britishcouncil.org

  5. What is assessment? • Noun • The evaluation or estimation of the nature, quality, or • ability of someone or something: "the assessment of • educational needs". • Synonyms • appraisal - rating - valuation - estimation - evaluation www.britishcouncil.org

  6. Why do we need to do assessment? Motivate students Evaluate teaching Evaluate learning Rank students Benchmarking Inform future teaching Find gaps in learning Selection Measure achievement www.britishcouncil.org

  7. What kinds of assessment are there? Test Quiz Essay Group task Assignment Observation Questionnaire Self assessment Presentation Peer assessment Portfolio Conversation Checklist www.britishcouncil.org

  8. 3. Considerations in Language Assessment www.britishcouncil.org

  9. What are the qualities of a good test? www.britishcouncil.org

  10. R V P I IELTS Approach to Test Qualities • Validity • Reliability • Impact • Practicality

  11. Validity: test content • Test content matched to needs of candidates • Task-based approach to testing • Testing of integrated skills • Variety of Listening and Reading item types used • Face to face Speaking Test and direct marking • Accepting all standard varieties of native-speaker English

  12. Validity: continual research based development • 1980 ELTS introduced (English Language Testing Service) • Original 4 skills test • 1989 tri-partite international partnership created, 1st delivery of IELTS • 1995 major systems and content revision • 2001 revised Speaking Test • 2005 revised Writing assessment criteria • 2007 introduction of half band for Speaking and Writing • Constants up to date:9-band scale, skill profile & overall score, f2f interview

  13. Reliability of test materials

  14. Reliability of results • Scoring system • Marker/Examiner selection and training • Double marking and EOR • Research and validation analysis • Monitoring: sample and targeted • Results benchmarked with international standards

  15. How do you create effective test tasks and rating scales for Writing and Speaking? • Theoretical relevance • The test’s criteria and tasks should be defensible in terms of valid theories of oral performance • Discriminating power The test criteria and tasks should enable relevant distinctions to be made between candidates Assessability The features selected for assessor attention should be salient enough for a single assessor to notice them

  16. IELTS W/S Band Descriptors

  17. Results benchmarked with int’l standards http://www.cambridgeenglish.org/about-us/what-we-do/international-language-standards/ www.britishcouncil.org

  18. Impact • Long-standing concern for the relationship between IELTS and its contexts of use • Test development partly shaped by this concern • Early 1990s growing interest in test impact/washback • Systematic investigation of IELTS impact began in 1995

  19. Consultation processes with stakeholders to ensure - IELTS is fit for purpose - cut scores are set appropriately • Use of GT module for immigration and employment • Standards setting events • General Medical Council, UK (June 04) • National Council for State Boards of Nursing, USA (Oct 04) • Canadian Immigration and Citizenship (May 05)

  20. Main outcomes of IELTS Impact Studies • IELTS is a reasonable direct communicative performance test of the four macro skills • IELTS is challenging but fair • IELTS uses authentic and appropriate content • IELTS is motivating for learners and teachers • IELTS engenders a positive effect on classroom activities and learning processes

  21. Practicality 可行性 • 48 test administrations per year • Global network of 900 test venues in over 130 countries/regions • Emphasis on standards, consistency and security • global standards • test centre selection, training, auditing • secure stock management • on-line results verification • Provision of information through a variety of channels • On-line registration • Results available 13 days after the test • Post test services

  22. 4. Q&A

  23. Acknowledgement • References: • Presentation for teachers event, Nick Charge • www.ielts.org www.britishcouncil.org

More Related