210 likes | 292 Views
Maryland’s Approach to Converting Assessment Data to OSEP Outcome Categories August 28, 2007 Deborah Metzger Dmetzger@msde.state.md.us. Birth to 3 Child Outcomes. Funded by IDEA General Supervision Enhancement Grant (GSEG) CFDA84.326X.
E N D
Maryland’s Approach to Converting Assessment Data to OSEP Outcome Categories August 28, 2007 Deborah Metzger Dmetzger@msde.state.md.us Birth to 3 Child Outcomes Funded by IDEA General Supervision Enhancement Grant (GSEG) CFDA84.326X
Outcome Data CollectedWhen the Child Enters and When the Child ExitsExample Child enters at 20 months Initial Present Levels of Development data are extracted from the IFSP database Child exits at 36 months Present Levels of Development data at Exit are extracted from the IFSP database and compared to entry data to determine progress Progress At Exit Data Status At Entry Data
How Do We Get The Data?(for Status At Entry and Progress at Exit)Data Extracted from Present Levels of Development and Electronically Linked to 3 Outcomes to Produce Answers Alignment of broad outcomes to Present Levels of Development Is the child’s acquisition and use of knowledge and skills (including earlylanguage/communication) at the level expected for his or her age? Are child’s social-emotional skills (including social relationships) at the level expected for his or her age? Does the child use appropriate behavior to meet his or her needs at the level expectedfor his or her age?
Protocols for Linking Age Levels/Age Ranges with Outcomes • For the outcome “acquisition and use of knowledge and skills (including early language/communication),” two domain categories (cognitive and communication) will be used. If both domains have quantitative data, the category that has the lowest range of data will be used. • When an age range has been entered, the midpoint of the range will be used.
Rationale for Maryland’s Approach • Decision to align outcome process with the IFSP process • Focus on improving evaluation and assessment practices (Online Tutorial) • Focus on ensuring data is collected in all domains (Monitoring) • Have a data system that collects Present Levels of Development (PLOD) • Could get started by generating electronic reports from data entered into PLOD • Response to local input
Most Commonly Used Toolsin Maryland • Early Learning Accomplishment Profile (E-LAP) • Early Intervention Developmental Profile (EIDP) • Hawaii Early Learning Profile (HELP) • Battelle Developmental Inventory II • Ages and Stages
How will Local Programs Measure and Report Status at Exit Data • Assessment at exit results will be entered on the IFSP form, Section II: Present Levels of Development (PLOD). The form will be revised so that providers can check off if the PLOD are from an entry, interim, or exit assessment. • The name of the assessment(s) that were used will be documented on the IFSP form. • The results of the assessments at exit will be entered into the new screens in the IFSP database.
Need to Validate the DataCompare Domain Results to Functional Results
Child Outcomes—ValidationWHY? Maryland has elected to use the information from domain-specificassessment results to determine the results of functional child outcomes. Therefore it is important that the electronic results be validated. In other words, we are asking the following: “Are the responses derived from the electronically-extracted domain data consistent with direct responses from providers about a child’s functioning in the three outcomes?”
How Will We Validate the Results? In December 2006, local Infants and Toddlers Programs began completing the Child Outcome Summary Form as soon as possible following initial evaluation and assessment. Local programs are completing COSFs at exit for children: •Who were referred since December 2006 •Who received services for at least six months, and •For whom a COSF was completed at entry COSF results will be entered into the IFSP database.
What if the “electronic results” and “validation results” are different? During 2007-2008, we will begin to conduct a validation study by comparing electronic results and COSF results, conducting focus groups with local programs, and discussing discrepancies in results with local provider teams. Based on the validation study results, we will determine how our approach should be modified or changed, whether we will continue our progress reporting by using Present Levels of Development data or through the COSF.
Measuring Progress Based on the Rate of Growth Between Entry and Exit •Working with Evaluation and Assessment Consultant to identify a methodology for measuring developmental gains during participation in early intervention •Testing child data using two existing indices: Intervention Efficacy Index Proportional Change Index
Intervention Efficacy Index Relates changes in child capabilities to time spent in program; describes individual and group progress in terms of developmental gains within and across domains for each month in an intervention program. (Bagnato & Neisworth)
Intervention Efficacy Index IEI = Developmental gain in months Time in intervention in months IEI = Exit DA - Entry DA Time in Intervention IEI = 34 months-20 months 12 months IEI = 1.17
Proportional Change Index •Controls for children’s developmental status before intervention •Calculation is a ratio of a child’s rate of development at pretesting to the rate of Development during intervention defined at posttesting (Wolery, 1983)
Proportional Change Index PCI = Exit DA-Entry DA / Entry DA Time in Intervention Entry CA PCI = 34months-20months/ 20 months 12 months 24 months PCI = 14 /.83 12 PCI = 1.40
Linking Results to OSEP Categories •Test both Indexes with real-child data •Determine numerical ranges for linkage to OSEP categories •Decide which index yields most meaningful results and most accurately matches the OSEP categories
Pros, Cons, Challenges! Pros: •Existing IFSP data and statewide database •Use of COSF for validation has generated good discussions of functional performance •Possibility of more meaningful results using factors such as time in intervention and relationship of chronological age to developmental age at entry
Pros, Cons, Challenges! Cons: •Using domain-based assessment results to Measure developmental progress in functional Outcomes •Using multiple assessment tools, rather than single or limited number of tools Challenges: •Making decisions thoughtfully with regard to Impact on local programs and families, but quickly enough to ensure meaningful data and analysis •Sleeping at night without outcome nightmares!
Ultimate Goal:Positive Results for Infants, Toddlers, and Families