American Institutes for Research February 2005

1. Assessment Policies Implementation and Monitoring American Institutes for Research February 2005

2. State Assessment Policy • States are required to have an assessment policy that describes: • Valid and reliable assessments, appropriate for adult literacy students • Standardized administration procedures • Scoring and reporting procedures • Training activities • DAEL verifies policy through data quality checklist and monitoring visits

3. Findings from Review of State Assessment Policies • Only 36 states submitted a complete assessment policy • Inappropriate tests still being used • “Informal classroom/teacher observation” still listed as assessment method • Allowed use of same form for pre- and post-test • Time for post-testing not specified or widely variant – few standards for post-testing specified

4. DAEL Monitoring Visits Also Found: • Lack of written assessment policies • Insufficient or incomplete policies in several areas • Use of inappropriate tests • Most common problems: • Time interval between pre- and post-testing was uneven, inappropriate or inconsistently implemented • Much information in data quality checklists could not be verified by the monitoring team.

5. Requirements for Assessments • Standards test, conforming to psychometric standards • Test designed to measure progress in basic and/or functional literacy skills • Multiple, parallel forms • Identify unacceptable assessment methods

6. Required Administration Procedures • Set times for pre- and post-testing followed by all programs • Number of hours or weeks of attendance • How to test and place special populations (e.g., unable to test) • Different forms used for pre- and post-tests • Correct test procedures used

7. Scoring and Reporting Requirements • Placement benchmarks for pre-test scores • Advancement benchmarks for post-test scores tied to assessments • Teachers should have access to scores

8. Effective Training • Local test administration staff trained on test administration and scoring procedures • Conducted by a qualified person • Sufficient number of staff trained in each program • Quality training materials used • Multiple and ongoing, follow-up training

9. Improved Pre-Post Test Rates • Without post-test no level gain can be reported • Improved post-testing leads to better overall performance • Table 4B data limited

11. Monitoring Assessment Procedures • Test scores recorded with: • Administration date • Test name and form used in database • Scale scores • Hard wire to EFL • Timely review of data -- on-site monitoring and review • Training: who trained, by whom, how many staff?

12. Discussion • Assessment policy issues • Monitoring and implementing assessment policies • Strategies for improving proportion of students post-tested • Other??