Understanding and Setting New Scores

1. Understanding and Setting New Scores

2. iBT TOEFL Scores The score report universities receive will contain: • Four skill scores • Reading 0 - 30 • Listening 0 - 30 • Speaking 0 - 30 • Writing 0 - 30 • A total score: 0 – 120

3. Setting New TOEFL Scores An opportunity to think more about English language needs & requirements • Are all skills equally important for success in your program? If not, do your score requirements reflect this? • Students with the same total score can have different abilities • Consider using: • skill scores instead of total scores • skill scores and total scores

4. Understanding & Setting New Scores • Rubrics for Writing & Speaking • Speaking and Writing samples • Score comparisons • Percentile data • English language competency descriptors • Results from standard setting studies • Standard setting on your own campus

5. Score Comparisons: iBT, CBT, PBT • Based on a field test of 3,000 test takers in 30 countries • Approx 80% of TOEFL volume comes from these 30 countries • Test takers took TOEFL iBT and TOEFL CBT (computer-based) • Score comparisons for CBT and PBT (paper-based test) established in 1998

6. Using Score Comparisons: iBT, CBT, PBT Score-to-score and range-to-range comparisons for: • Reading • Listening • Writing • Reading, Listening & Writing only total • Total score (all four skills)

7. Setting Speaking Scores No Speaking section on TOEFL CBT Standard setting panel of ITA experts Professional judgment, not statistically determined • Minimally acceptable level of speaking for lowest level of contact with undergraduates: 23 • TSE 50 score equivalent: 26

8. Using Score Comparisons: iBT, CBT, PBT Score comparisons can be useful for understanding relationships, but differences in the tests can make it difficult to establish exact comparisons.

9. Using Score Comparisons: iBT, CBT, PBT Remember: • If you’re comparing total scores, CBT and PBT TOEFL do not measure Speaking, but iBT does. • Using skill scores or skill scores and a total score will give you more information about your applicant. Use the whole profile. • ETS encourages you not to set a rigid cut score

10. Percentile Data • Based on field test • Should be used with caution • Sample size • Motivation & performance • Unfamiliarity with the test, communicative competence & integrated tasks • Addition of Speaking • Participants had lower ability level than general TOEFL population New data from operational test to be published after first year

11. English Language Competency Descriptors • Based on 2,300 students who took the new iBT test and responded to questions about their English language ability in each skill • Competency descriptors tell us what a student can do, and has difficulty with, in English

12. Results from ETS Standard Setting Studies • Panels at 5 universities recommended scores based on seeing the new test and reviewing examinee performance. • Results to be published after decision makers discuss recommendations and set final scores. Final scores to be posted on our Web site. • Schools similar to those in the study may wish to adopt similar scores.

13. Panel Recommendations –NOT FINAL SCORES

14. Standard Setting at Your Institution • The best way to ensure your language requirements match your institution • Over 2 days, a panel of college or university faculty and staff work view the test and student responses to it. • The panel recommends the level of English just high enough to cope with studies at your school.