2013-2014 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division

1. 2013-2014 Holistic Rating Training RequirementsTexas Education AgencyStudent Assessment Division

2. Rater Credentials Each teacher selected to rate an ELL must have the student in class be knowledgeable about the student’s ability to use English in instructional and informal settings hold valid education credentials such as a teacher certificate or permit be appropriately trained, as required by TEA TEA

3. TELPAS Rater Responsibilities A student’s TELPAS rater is the teacher designated by the district as the official rater of the student’s English language proficiency. The student’s rater must rate the student in all domains for which the student is eligible. A student is not permitted to have one rater for some domains and another rater for other domains. TEA

4. New and Returning Rater Definitions for K–1 and 2–12 The training webpage informs raters that districts may sometimes require a returning rater to complete new-rater training and to consult with their testing coordinator if they need clarification. TEA

5. TELPAS Administration Procedures Training • As part of annual spring TELPAS administration procedures training, holistic rating training requirements are reviewed with raters, as well as information about how to access the online training and calibration components. • In addition, raters receive training on assessment procedures such as how to assemble writing collections, how to record students’ proficiency ratings, etc. TEA

6. Two Types of Training Online basic training courseThis course isfor new raters. It provides instruction on using the PLD rubrics and gives raters practice rating students in each language domain. There are separate courses for K–1 and 2–12. Online calibration This is for all raters. Raters use the PLDs to rate students in each language domain. Raters have three opportunities to calibrate on assigned grade cluster. TEA

7. Training and Calibration Grade Clusters • Raters must know their assigned grade cluster to select the appropriate online training. • Raters should consult with their campus testing coordinator if they are unsure of their assigned cluster. Grade Clusters Grades K–1Grade 2 Grades 3–5 Grades 6–8 Grades 9–12 TEA

8. Preparing for Calibration Sets • New raters must complete the online basic training course before beginning calibration. • All raters have the option to review the online basic training course (which includes practice rating activities) before beginning calibration. TEA

9. Other Things to Know About Calibration • Raters affirm online that they will keep the contents of the calibration sets secure and confidential. • Calibration activities are taken from a bank and randomized. Trainees will rate different sets of students. • Raters can work at their own pace, go back and review students, and change ratings as they work. TEA

10. Other Things to Know About Calibration • Raters can exit and return later to finish. They click a “submit” button when they are finished with a set. • After completing a calibration set, raters immediately see results. Results show both the rating assigned by the rater as well as the correct rating. • Raters see annotations explaining the correct ratings. Raters should use the annotations to go back and review any incorrectly rated students. TEA

11. Is calibration a test? No, it is a training method that ensures that raters have enough guidance, practice, and support to assess students consistently and accurately. TEA

12. Will raters be able to refer to any resources during calibration activities? Yes, raters should use their rating rubrics (PLDs) and refer to, as needed, information from the: • online basic training course • holistic rating PowerPoints produced by TEA • TELPAS Rater Manual • Educator Guide to TELPAS TEA

13. How many students must be rated successfully? To be successful, raters need to rate students in their assigned grade cluster with at least 70% accuracy. TEA

14. Supplemental Holistic Rating Training • Raters not successful after sets 1 and 2 must receive supplemental training. • The rater will meet with a district-appointed supplemental support provider. • After the rater has received supplemental training, he or she will be able to access the third and final calibration set. TEA

15. Recap of Calibration Process • There are 3 sets of 10 students. • Raters who calibrate on set 1 are done. • Raters who don’t calibrate on set 1 go on to set 2. Raters who calibrate on set 2 are done. • Raters who don’t calibrate on set 2 receive supplemental training. • Raters attempt third and final calibration set. • Raters who calibrate on set 3 are done. TEA

16. Course and Calibration Certificates • Raters who take the basic training course get a certificate from the online TrainingCenter after completing the course components. • Raters will receive a certificate of successful calibration when they calibrate. TEA

17. Writing Collection Overview 2013–2014 Texas Education Agency Student Assessment Division

18. Building Collections • Strive to gather more than 5 writing samples for each student. • Choose at least 5 samples that meet the criteria and do the best job of portraying the student’s current proficiency level. • If a student is near the border between two proficiency levels, consider including samples written in the latter part of the TELPAS assessment window. The more recent the writing samples, the more accurately they will reflect the proficiency level of these students.

19. Building Collections • Collections should contain some papers in which students showcase English they know and feel comfortable using. “Comfort zone” writing is especially important for students at lower proficiency levels. • Collections should also include papers in which students are stretched and pushed beyond their comfort zone so the collection shows that a student has not yet reached the next level (the student is beginning but not yet intermediate, intermediate but not yet advanced, advanced but not yet advanced high).

20. Building Collections • The papers you assemble need to help you determine and justify your ratings. They must give you evidence to say, “I know the student is at least at X proficiency level because of these characteristics in his or her writing. I know the student is not yet at the next proficiency level because of these other characteristics in his or her writing.” Reminder: The characteristics you consider must come from the PLDs.

21. Beginning Level The ELPS require ELLs of all proficiency levels to learn to write in English. It is not acceptable for writing collections of students at the beginning level to include samples written primarily in the native language. Such samples do not provide evidence of English writing proficiency. Writing tasks of these students should be adapted to their needs. Their writing in English will likely be formulaic or memorized, include recently practiced vocabulary, lack detail, etc.

22. Summarizing Tips • Students who are capable of expressing themselves in English in a detailed, extended way should do so. Do not include brief responses from students who know enough English to respond to writing tasks in extended ways. • In other words, consider students’ English-language proficiency levels in determining whether papers are too brief to be included.

23. Summarizing Tips • Build writing collections that have a balance of writing from language arts and other core content areas. • Collections should show what the student knows and can do as well as what the student struggles with in second language acquisition. • Remember, build the collections to portray the student’s overall ability to communicate in writing in English.

24. Verification of Collection Contents Campuses follow procedures outlined in the TELPAS Rater Manual to ensure that the writing collections are assembled correctly and include the necessary number and types of writing.

25. Resources with More Details • District and Campus Coordinator Manual • Updated annually and available in late fall from the Assessment A–Z Directory at http://www.tea.state.tx.us/student.assessment • Shipped to districts in January • TELPAS Rater Manual • Updated annually and available in December at http://www.tea.state.tx.us/student.assessment/ell/telpas/ • Shipped to districts in January • Online course titled Assembling and Verifying Grades 2–12 Writing Collections • Available online January 13 at http://www.texasassessment.com/TexasTrainingCenter