Holistic Rater Training February 14, 2013

1. Holistic Rater TrainingFebruary 14, 2013

2. Overview of the Rating Process • Each campus will identify ELLs on their campus and the principal will designate raters to assess students • All raters must complete the appropriate rater training • Raters gather writing samples for ELLs in grades 2–12 to include in each student’s writing collection • The rater is responsible for assessing the English language proficiency of students and assign them a rating • The rater will record the students’ ratings and related info on the TELPAS Rating Roster form • Student’s ratings will be reviewed by the campus principal

3. Overview of the Rating Process • This year SMSD will complete a “Drop Everything and Write Day” • The CTC will ensure the writing collections are kept in the students’ permanent record files for two years • Campuses must maintain a file of the rating rosters

4. Holistically Rated Components of TELPAS Online multiple-choice reading tests are used to assess the reading domain for grades 2–12. TEA Division

5. Holistic Rating Training Requirements

6. Rater Credentials Each teacher selected to rate an ELL must have the student in class be knowledgeable about the student’s ability to use English in instructional and informal settings hold valid education credentials such as a teacher certificate or permit be appropriately trained, as required by TEA

7. New and Returning Rater Definitions for K–1 and 2–12 The training webpage informs raters that districts may sometimes require a returning rater to complete new-rater training and to consult with their testing coordinator if they need clarification. TEA

8. Training Requirements for K–1 and 2–12 • In the Fall ELPS-TELPAS professional development for new teachers and new raters – foundational (awareness) training • In the Spring (required and in addition to administration procedures training) • New raters – complete online basic training course and online calibration activities • Returning raters – complete online calibration activities

9. Training Flowchart Fall ELPS-TELPAS Foundational (Awareness) Training For teachers who will be trained as new TELPAS raters in the spring if they lack this foundation Spring TELPAS Administration Procedures Training As a key part of this training, information from the TELPAS Manual for Raters and Test Administrators is reviewed to prepare raters to proceed with online holistic rating training Returning Raters New Raters Online Basic Training Course Online Calibration (Sets 1 and 2*) Online Calibration (Sets 1 and 2*) If not calibrated: Supplemental Holistic Rating Training *Set 2 required only if not successful on Set 1 Final Online Calibration (Set 3)

10. T Two Types of Training Online basic training courseThis course isfor new raters. It provides instruction on using the PLD rubrics and gives raters practice rating students in each language domain. There are separate courses for K–1 and 2–12. Online calibration This is for all raters. Raters use the PLDs to rate students in each language domain. Raters have three opportunities to calibrate on assigned grade cluster.

11. Grade Clusters of Training and Calibration • Raters must know their assigned grade cluster to select the appropriate online training. • Raters should consult with their campus testing coordinator if they are unsure of their assigned cluster. Grade Clusters Grades K–1Grade 2 Grades 3–5 Grades 6–8 Grades 9–12

12. Raters with Students in Multiple Grade Clusters within Grades 2–12 • Raters are required to train and calibrate in the cluster in which they have the most ELLs. • As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s). • Example: A grade 2–5 ESL teacher has most of her ELLs in grade 3. She must complete training and calibration in grades 3–5. As a best practice, she should review the online practice activities for grade 2 to check her readiness to apply the rubrics appropriately.

13. Raters with Students in Multiple Grade Clusters that Include Grades K–1 • Because of differences in the instructional content and rating rubrics, these raters must complete training and calibration for K–1 and at least one other cluster in 2–12. • Raters with more than one additional cluster should train in the cluster in which they have the most ELLs. • As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s). • Example: A grade 1–3 ESL teacher has most of his ELLs in grade 2. He must complete training and calibration in grades K–1 and 2. As a best practice, he should review the online practice activities for grades 3–5 to check his readiness to apply the rubrics appropriately for his 3rd grade students.

14. When and where do raters take the online training? • Raters may either complete training and calibration during school hours, after school, or on weekends. • The online training system allows raters to access the training from any computer that meets the minimum system requirements.

15. Why is calibration necessary? • Calibration helps ensure that raters have adequate training, including ample practice and feedback, before they assess their students in the spring. • Calibration ensures that raters clear their heads and consider only the elements of student performance included in the PLDs. • Calibration supports assessment validity and reliability and is an important part of holistically scored assessment processes.

16. Preparing for Calibration Sets • New raters must complete the online basic training course before beginning calibration. • Returning raters have the option to review the online basic training course (which includes rating activities) before beginning calibration.

17. Other Things to Know About Calibration • Raters affirm online that they will keep the contents of the calibration sets secure and confidential. • Calibration activities are taken from a bank and randomized. Trainees will rate different sets of students. • Raters can work at their own pace, go back and review students, and change ratings as they work. • Raters can exit and return later to finish. They click a “submit” button when they are finished with a set. • After completing a calibration set, raters immediately see results. Results show the rating assigned by the rater and the correct rating. • Raters see annotations explaining the ratings. Raters should use the annotations to go back and review any incorrectly rated students.

18. Is Calibration a Test? No, it is a training method that ensures that raters have enough guidance, practice, and support to assess students consistently and accurately.

19. Will raters be able to refer to any resources during calibration activities? Yes, raters should use their rating rubrics (PLDs) and refer to, as needed, information from the: • online basic training course • TELPAS Manual for Raters and Test Administrators • Educator Guide to TELPAS

20. Supplemental Holistic Training • Raters not successful after sets 1 and 2 must receive supplemental training. • The rater will meet with a district/campus-appointed supplemental support provider. • After the rater has received supplemental training, he or she will be able to access the third and final calibration set.

21. Recap of Calibration Process • There are 3 sets of 10 students. • Raters who calibrate on set 1 are done. • Raters who don’t calibrate on set 1 go on to set 2. Raters who calibrate on set 2 are done. • Raters who don’t calibrate on set 2 receive supplemental training. • Raters attempt third and final calibration set. • Raters who calibrate on set 3 are done. TEA

22. What happens if a rater is unsuccessful in calibrating? • If the individual is needed to serve as a rater, the district must implement rater support procedures to ensure that the rater’s students are evaluated consistent with the rating rubrics.

23. Course and Calibration Certificates • Raters who take the basic training course get a certificate from the online TrainingCenter after completing the course components. • Raters will receive a certificate of successful calibration when they calibrate. • Raters must submit their calibration certificates to their Campus Test Coordinator

24. Assembling and Verifying

25. Assemble and Verify Writing Collections • Raters of grades 2-12 students, must assemble student writing collections and base their ratings on the writing in the collections. • Each writing collection must contain at least 5 writing samples and must include at least one narrative writing sample about a past event and at least 2 academic writing samples from math, science, or social studies. • Writing assigned on or after February 18, 2013, may be considered for the writing collections. Writing prior to February 18, 2013 is not eligible. • Writing samples may continue to be gathered until the time raters are required to turn in students’ ratings to the CTC. • All writing should reflect a student’s current proficiency level.

26. Assemble and Verify Writing Collections cont. • All writing assignments must include the student’s name and date. • The cover sheet and verification checklist located in Appendix must be completed and stapled to each student’s writing collection. • Photocopies of classroom writing assignments may be included in the writing collections as long as all copied pages are clear and legible. • Writing samples may be typed provided that spell check and grammar check are disabled.

27. Types of Good Writing Collections • In all writing assignments, encourage students to take their time and write as much detail as they can. • In each collection, include some writing tasks that will allow students to show what they can do. • Also include some writing tasks that stretch and push students’ limits so you can see the full extent of their language abilities and any second language acquisition they still need to develop.

28. What Not to Include in a Collection • Papers containing language copied directly from a textbook, lesson or written source • Papers that require the student to rely on the dictionary • Papers that have teacher’s corrections on them • Papers that have been edited by peers, parents or teachers • Papers where the student writes primarily in their native language

29. Four Language Domains • Listening – The ability to understand spoken language, comprehend and extract information, and follow social and instructional discourse through which info is provided • Speaking – The ability to use spoken language appropriately and effectively in learning activities and social interactions • Reading – The ability to comprehend and interpret written text at the grade-appropriate level • Writing – The ability to produce written text with content and format to fulfill grade-appropriate classroom assignments

30. Procedures for Rating Students • Review the holistic rating process before beginning • Always refer to the PLDs when rating students; you may photocopy the PLDs if necessary • Begin with the students whose English Language proficiency levels are clearest to you • Your ratings should be based on observations of the student in a variety of social and academic settings over time • Evaluate the current level of English language proficiency • The proficiency rating you designate should represent the level at which a student performs most consistently • Collaborate with other teachers and school personnel regarding student’s ratings • Complete all ratings for one student prior to proceeding to the next student

31. Procedures for Rating Students • Compare each student to the PLDs and do not let the proficiency levels of other ELLs you teach influence your ratings • With the exception of grades 2-12 writing, you should be able to determine a student’s rating by simply reflecting on the PLDs and the student’s current ability to understand and use English in social and academic settings. • Remember – academic language proficiency is not the same as academic achievement.

32. Review ELPS-TELPAS Proficiency Level Descriptors

33. Creating and Managing TELPAS Rating Entry Groups

34. Drop Everything and Write (DEW Day)

35. Questions!!!