TELPAS Holistic Rating & TELPAS Online Testing

1. TELPAS Holistic Rating&TELPAS Online Testing Courtney Hill Keta Solis Lisa Hernandez Mary Lou Garcia

2. Topics • Assembling writing collections • Holistic rating training requirements, including supplemental training • Tips on using reports to monitor course completions and qualification activities • Updates • Future plans

3. Topics • Use of TELPAS electronic submission process • to enter/verify identification, program, and demographic information • to submit holistic ratings and rater information • to submit grades 2-12 reading tests • to verify data • Final three-day verification window • Interface assignments • Student tutorials

4. Holistically Rated Components of TELPAS There are multiple-choice reading tests to assess the reading domain for grades 2–12.

5. Overview of the Rating Process • Identify ELLs in grades K-12 and designate raters* • Holistic rating training conducted* • Gather writing samples for ELLs in grades 2-12 to include in writing collection (Feb. 1) • Raters use PLDs to assign students a rating in each language domain assessed *completed in fall semester

6. Assembling Writing Collections

7. Writing in Grades 2-12 • Authentic writing aligned to the Texas English language proficiency standards (ELPS) and content-area TEKS • Avoid formulaic or school-wide writing assignments • Do not allow TELPAS writing days where all writing is done in 1 day

8. Holistic Rating Training Requirements

9. Training Requirements • NEW RATERS complete basic training in the fall; this allows them to use the PLDs to guide instruction and monitor progress throughout the year* • ALL raters complete online refresher in the spring to practice rating students prior to the TELPAS assessment • Reminder: • Vast majority of Level 1 training should occur in fall • In spring, Level 1 course is offered to cover extenuating circumstances

10. Spring 2010 Online Training • Level 1: Basic Training Course for K-1 • Level 2: Refresher Course for K-1 • Level 1: Basic Training Course for Grades 2-12 • Level 2: Refresher Course for Grades 2-12 • Assembling and Verifying Grades 2-12 Writing Collections

11. Who Takes Which Training • Level 1 online trainingNewly assigned K–12 raters who have not yet been trained and 2–12 raters previously trained but not yet qualified • Spring online qualificationFor grades 2-12 Level 1 trainees • Level 2online refresher For all other raters – the vast majority • K–1 previously trained raters (includes those trained in fall 2009) • 2–12 previously trained and qualified raters (includes those trained and qualified in fall 2009)

12. Key Dates for Holistically Rated Components https://texas.pearson.desire2learn.com/

13. End Dates for Holistic Rating Training • All refreshers must be completed by Monday, Feb. 22, 2010, 8:00 am. • Refresher Re-Taker supplemental training will be held Feb. 25, 2010. • March 8, 2010 is the deadline for new Qualifications and Refresher Re-takers. • Special Situations contact Courtney Hill chill@lcisd.org

14. New Level 2 Supplemental Training Requirements Beginning Spring 2010 • Testing coordinators to receive a copy of the qualification and/or refresher certificate and raters will initial roster of Raters. • For K–12 raters who do not rate students accurately enough on Level 2 (refresher course) practice activities • Supplemental Training – Feb. 25, 2010 (bring printout of training history) • Raters who are required to take the supplemental training will print the page of the supplemental training when complete. • Further info to come…

15. Supplemental Training Instructions • Raters who rate fewer than 5 of 7 students on the Level 2 refresher correctly in any domain are required to take the supplemental training in that domain. • Supplemental Training: Feb. 25 Refresher re-taker training • Raters will print 2 copies of the Supplemental training history completed at the Feb. 25 Refresher re-taker training. 1copy is for the campus coordinator and the other for their records. • A score of 4 out of 5 in the failed domain is required to be a rater. (Shown on copy of training history)

16. Careful with Confusion Between Supplemental Training and Additional Writing Practice Modules (Grades 2-12) • Level 2 refresher for grades 2–12 contains an additional writing practice module which raters who need supplemental training might do, by mistake, thinking it is their supplemental training • Raters will not get confused if they follow the instructions in the TELPAS Manual and online course

17. Options for Additional Writing Practice Module Campus Coordinators may encourage raters to do this module: • before Level 2 writing module • before Level 1 supplemental training if not successful on Level 2 • after Level 1 supplemental training if not successful (as part of rating support plan)

18. Individuals Not Successful on Supplemental Training Rating support will be provided for raters who are not successful on the supplemental training. The unsuccessful rater will still rate, but a qualified rater will rate behind them.

19. Bottom Line • Campus coordinators are responsible for overseeing whether raters: • complete necessary training • are adequately prepared to serve as independent raters

20. Monitoring Course Completions and Performance and Qualification Activities

21. Available Reports for Online Training and Qualification • TELPAS Confidential Course Completion Roster • List of users showing courses done, module results, and time in modules • TELPAS Course Completion Summary Report • Summarizes by course how many users are done or in progress • TELPAS Confidential Rater Qualification Roster • List of users showing qualification status and results • TELPAS Confidential Rater Qualification Summary Information • Summarizes how many attempted, successful, and unsuccessful qualifications • TELPAS At-A-Glance Training & Qualification Report • List of users with key course completion and qualification information https://texas.pearson.desire2learn.com/ Follow along through the next six slides

22. Running Reports and Report Formats • Reports are available by semester • Report can be viewed in 3 different formats • Web view • PDF file • CSV file • Data in CSV file format can be manipulated in a way that is most useful

23. TELPAS Confidential Course Completion Roster Use this report to monitor supplemental training

24. TELPAS Course Completion Summary Report

25. TELPAS Confidential Rater Qualification Report

26. TELPAS Confidential Rater Qualification Summary Information

27. TELPAS At-A-Glance Training & Qualification Report Can also be used to monitor supplemental training

28. Future Plans

29. Future Expanded Qualification Requirements • Beginning in 2010-2011, listening and speaking to be added to qualification component for raters of students in grade 2 or higher • Same number of students per qualification set –rating of writing, listening, and speaking within each set; qualification overall, not by domain • Raters already qualified won’t have to qualify again • Beginning in 2011-2012, qualification requirements to apply to K–1 raters too – all domains to be included

30. Resources • 2010 District and Campus Coordinator Manual • Includes the Fall 2009-Spring 2010 Coordinator’s User Guide for Online Training and Qualification • 2010 TELPAS Manual for Raters and Test Administrators • Includes the Fall 2009-Spring 2010 Rater’s User Guide for Online Training and Qualification • Assembling and Verifying Grades 2-12 Writing Collections online course • Found on the Texas TrainingCenter • 2009-2010 Writing Collection Overview PowerPoint training slides • Available on TEA’s Student Assessment Division website on the ELL Assessments page

31. Online TELPAS Reading Test

32. Online Testing Resources andeMeasurementSystem • Key guides • District and Campus Coordinator Manual • Technical User’s Guide • TELPAS Manual for Raters and Test Administrators • See page 207 of Coordinator Manual for comprehensive list of online testing resources http://ritter.tea.state.tx.us/student.assessment/resources/guides/coormanual/dccm_telpas.pdf

33. Electronic Submission Process

34. Electronic Data Submission Process Used to submit all information through the secure eMeasurement system: • student identification, demographic, and program information • holistic ratings • “do not score” information • rater information • testing accommodation categories

35. 3 Key Screens • Student Data: • View and Maintain Student Rosters: Add Student or Choose student and edit info. (Edit Student Screen) • Session Management: • Create a new session • Add/remove students, then choose a student from the Session Roster (Student Test Details Screen) • View Maintain Sessions • Student Test Details found by Clicking on a Session, Then choose a Student from the Roster (Student Test Details Screen) • View TELPAS Student Status • Generate List (TELPAS Student Status Screen) http://etest.pearson.com/Customers/Texas/eoct/txeoct/index.htm

36. Updated to reflect revised ethnicity/race field and clarified phrasing for Y in US schools. Edit Student Screen Edit Student screen is where student identification, demographic, and program information is found. Drop-down menus for identification and program information reflect the code values that are included in Appendix A of the 2010 District and Campus Coordinator Manual. JJAEP/DAEP student data is no longer being collected in Agency Use field.

37. Student Identification, Demographic, and Program Information • receive electronic (PDF) precoded list of students loaded in eMeasurement system at least 2 weeks prior to TELPAS window • Data loaded reflects PIEMS Submission Oct 30th • Lists to be verified against current information to update data if needed and add any new enrollees • Changes and additions to be entered on Edit Student screen • Instructions for adding students and updating data can be found in the Coordinator Manual

38. Student Test Details Screen • TELPAS manual for raters does contain instructions for this screen • The Campus coordinator will enter this information.

39. 2 Ways to Get to Student Test Details Screen • By clicking on student’s name in test session to which student is assigned • By clicking on first edit icon in row with student’s name on TELPAS Student Statuspage • This page allows students to be viewed in larger groupings than single test session

40. Creating Test Sessions 1. Getting to Student Test Details Screen Through Test Session Be careful to add the correct student to the correct test session.

41. Creating Test Sessions When setting up test session, remember that students should have all the time they need to complete the test.

42. Clicking on session takes user to a session roster.

43. Clicking on student name takes user to Student Test Details screen.

44. Student Test Details Screen Student Test Details screen is where holistic ratings, “do not score” designations, rater information, and accommodation categories are entered. For K-1 screen: The accommodation areas have been removed. Accommodations not collected for holistically rated domains.

45. Student Test Details Screen Listening drop-down menu shown here. For grades 2-12, the listening, speaking, and writing selections will be the same. For grades K-1, all four language domains will have the same selections. Only use the accommodations area if a student was allowed an accommodation on the reading test. If the student does not receive accommodations, fields should be left in “Select” position. After any selections are made, user must click Apply.

46. Reading Test Score Codes • There is no reading test score code on Student Test Details screen. For students who test, Reading field will simply be left in Select position, which is default setting • Every attempt must be made to test ALL students. However, should the extremely rare need arise, a “Do not score” code must be selected for students who do not test. These codes are: • Absent • Second Semester Immigrant Non-English Reader • ARD Decision • TEA-Approved Paper Administration • Other Student Not to Be Scored

47. Student Test Details Screen The rater info fields gather information about TELPAS raters and whether they collaborated with others in determining each student’s TELPAS ratings. “No Domains Rated” must be selected if a student is marked Extenuating Circumstances or ARD Decision in every domain

48. Mark Test Complete Button on Student Test Details Screen • Individuals entering ratings and rater information must not click Mark Test Complete button • It is to be used onlyfor grades 2–12 reading test if student’s test should not be scored (absent, ARD decision, etc.)or if a student didn’t submit a completed test • If button is selected, student cannot take reading test and Pearson will need to be contacted

50. 2. Getting to Student Test Details Screen Through TELPAS StudentStatus Page Clicking in first edit column takes user to Student Test Details screen.