Teresa M. Gillespie, M.S., C.C.C. Bilingual Speech-Language Pathologist

1. MAST Use of CELApro and CELA Placement Results and Implications for Assessment Processes and Considerations, Appropriate Student Interventions, Appropriate Interpretations of Evaluation Results, and Student Peer Comparisons Teresa M. Gillespie, M.S., C.C.C. Bilingual Speech-Language Pathologist

2. Background Information • The Colorado English Language Acquisition Proficiency (CELApro) and CELA Placement assessments are administered to English Language Learner (ELL) students to obtain information about their English language competency skills in four domains: Listening, Speaking, Reading, and Writing. These skills are measured along a continuum of five proficiency levels: Beginning, EarlyIntermediate, Intermediate, Proficient, and Above Proficient. • During MAST speech-language assessments, the ELL students receive an informal evaluation of their English language proficiency skills in the following areas: length of utterance, types of utterances, accuracy of responses to simple and complex questions, and understanding and use of basic and complex vocabulary. These skills are compared to the different language skills at each level of the Stages of Second Language Acquisition: Silent and Receptive, Early Production, Speech Emergence, Intermediate Fluency, and Advanced Fluency.

3. Why is There a Difference Between Results? If the CELApro and CELA Placement assessments, and the informal evaluations, purport to assess the English language proficiency skills of ELL students, then why is there a difference between the results?

4. Possible Explanations • The difference between the results is due to the amount of time that has elapsed between administration of the CELApro or the CELA Placement assessments, and the informal evaluations of ELL students’ English language proficiency skills conducted during speech-language assessments. • The difference between the results is due to variables related to the content of the CELApro or CELA Placement assessments and the informal evaluations of ELL students’ English language proficiency skills conducted during speech-language assessments. • The difference between the results is due to how the ELL students’ performance is quantified and/or qualified on the CELApro or CELA Placement assessments and the informal evaluations of their English language proficiency skills conducted during speech-language assessments.

5. Searching for Answers • Both the CELApro and CELA Placement assessments are highly aligned with the English Language Development (ELD) standards for Colorado. • The Stages of Second Language Acquisition, which are used to compare the results of the informal evaluations of ELL students’ English language proficiency skills conducted during speech-language assessments with the different language skills at each stage, are based on languageresearch. • CELApro and CELA Placement scores have not yet been correlated with the scores of an external measure in order to help establish the extent to which the CELA test items reflect the knowledge actually required for proficient English language skills (content validity) and the extent to which the CELA assessments measure what they claim to measure (construct validity).

6. Searching for Answers • The extent to which two or more testers agree with each other (inter-rater reliability) and the degree of agreement among multiple administrations of a particular test conducted by a single tester (intra-rater reliability) are affected by how two of the CELApro domains are scored and how frequently the CELApro testers are trained. • The CELA Placement assessment provides limited information regarding ELL students’ English language competency skills. • In order to more specifically describe ELL students’ English language competency skills, the CELApro assessment provides ELD Level descriptors by Grade, which are aligned with the ELD standards for Colorado.

7. Searching for Answers • The content of an ELL student’s utterances obtained during the informal evaluation of English language proficiency skills conducted during the speech-language assessments is analyzed according to what has been published in research regarding English semantic, morphological, syntactical, grammatical, phonological, and pragmatic language development and milestones. • The individual CELApro and CELA Placement domain scores are more representative of an ELL student’s English language competency skills than the Overall, Oral, and Comprehension scores.

8. Conclusions • The difference between the results obtained on the CELApro or the CELA Placement assessments, and the informal evaluations of ELL students’ English language proficiency skills conducted during speech-language assessments, is not due to the amount of time that has elapsed between administration of these assessments. Sometimes, the period of time between administration of the CELA assessments, and the informal evaluation of the ELL students’ English language skills, varied by only a few months, and the ELL students exhibited a difference of at least two levels or stages of English language competence. • The difference between the results obtained on the CELA assessments and the informal evaluations of ELL students’ English language proficiency skills conducted during speech-language assessments is due to variables related to the content of these assessments.

9. Conclusions • The difference between the results obtained on the CELA assessments and the informal evaluations of ELL students’ English language proficiency skills conducted during speech-language assessmentsis due to how the ELL students’ performance is quantified and/or qualified on these assessments.

10. What Does All of This Mean? • It is best to use the individual CELApro domain scores when describing an ELL student’s English language competency skills for the purpose of conducting peer comparisons. • It is best to incorporate the corresponding ELD Level descriptor by Grade when reporting an ELL student’s individual CELApro domain scores. • It is best to use more than an ELL student’s CELApro scores when deciding whether or not to utilize an Interpreter for testing. • MAST members will need to discuss when and how to use ELL students’ CELA Placement scores. • Recommend that MAST Bilingual Psychologists and Special Educators/Educational Specialists replicate this research with their respective tests.

11. What Does All of This Mean? • Recommend that MAST/District Speech-Language Pathologists conduct a statistical analysis between CELApro and formal English speech-language test results. • Out of a random sample of 50 ELL students evaluated in Spanish in the area of speech-language, the CELApro Overall scores achieved by 4 of them overlapped with the “equivalent” Stage of Second Language Acquisition; and, the CELApro Speaking domain scores achieved by 8 of them overlapped with the “equivalent” Stage of Second Language Acquisition. The CELApro Overall, Oral, Comprehension, and individual domain scores are not interchangeable with the different Stages of Second Language Acquisition. • The CELApro individual domain scores are based on academic achievement; the different Stages of Second Language Development are based on language research.

12. Future Plans • Focus on the results obtained for the CELApro Speaking and Listening domains when making decisions regarding speech-language assessments and speech-language therapeutic objectives since these scores are more representative of an ELL students’ English language comprehension and production skills. • Include the corresponding ELD Level descriptors by Grade when incorporating the CELApro Speaking and Listening domain scores into my decision-making process regarding speech-language assessments and speech-language therapeutic objectives. • Continue to conduct informal evaluations of ELL students’ English language proficiency skills during speech-language assessments, and compare the results of the informal evaluations to the different language skills for each level of the Stages of Second Language Acquisition.