1. Introduction

Less Is More? An Application of Propensity Score Stratification to First-Grade RetentionMieke Goos, Jan Van Damme, Patrick Onghena and Katja PetrySREE 2010

1. Introduction • Starting point: • Many young children struggle in elementary school • Countries deal with these early problems in a different way • Internationally frequently applied measure = grade retention

1. Introduction • Grade retention in Flanders: • Relatively high rate • For example: PISA 2003

1. Introduction • Relatively high rate … especially in Grade 1 • About 7% of Flemish children repeat Grade 1 • Socially approved by educators, policy makers and parents → being a grade retainee in Flanders has a different connotation than for example in the US (negative overtone) • No formal rules regarding grade promotion (no national/state standardized test procedures) → retention decision = joint decision by teacher and parents

1. Introduction • Research question: • Is Grade 1 retention an effective practice or not? • Focus of this study: • children’s psychosocial growth throughout elementary school

2. Method • Subjects: representative sample from the Flemish SiBO-project • 3624 first-graders, of which 298 were retained • 222 classes • 121 schools followed until Grade 6

2. Method • Instruments • Psychosocial growth: Teacher questionnaire • rated yearly by the teacher • items on a 1 to 6 point Likert scale • 7 subscales • Social skills • Popularity among classmates • Aggressive behavior • Hyperactive behavior • Asocial behavior • Dynamic-affective attitudes • and skills • Independent participation • School well-being • Self-confidence

2. Method • Instruments (continued) • Propensity of repeating Grade 1 • official records • achievement tests • Standard Progressive Matrices • teacher questionnaire about the child • parent questionnaire • teacher questionnaire about teacher didactics • school staff questionnaire • 68 prior student characteristics • 59 prior class characteristics • 42 prior school characteristics

2. Method • Analyses: 4-steps-procedure • Step 1: identification of ‘true’ confounders of Grade 1 retention • prior student, class and school characteristics • that are related to both treatment (i.e., Grade 1 retention) and outcome (i.e., children’s individual psychosocial growth) • Step 2: estimation of propensity scores based on these confounders • 3-level logistic regression analysis (students – classes – schools)

2. Method • Analyses: 4-steps-procedure (continued) • Step 3: decile stratification • 10 strata of equal size • Step 4: estimation of average psychosocial effects • 3-level curvilinear growth curve analyses (measurements – students – schools)

2. Method • Analyses: 2 comparison strategies • Same-grade approach = comparing retainees with their younger grade-mates • Same-age approach = comparing retainees with their age-mates who were promoted to a higher grade

2. Method Research year 1 (age 7) Research year 2 (age 8) Research year 3 (age 9) Cohort 1 3rd grade E Cohort 2 (not in SiBO dataset) Promotion 2nd grade C D H Grade retention Promotion Promotion Promotion 1st grade A B F G Grade retention Grade retention

2. Method Research year 1 (age 7) Research year 2 (age 8) Research year 3 (age 9) Cohort 1 3rd grade E Cohort 2 (not in SiBO dataset) Promotion 2nd grade C D H Grade retention Promotion Promotion Promotion 1st grade A B F G Grade retention Grade retention SAME-GRADE COMPARISON

2. Method Research year 1 (age 7) Research year 2 (age 8) Research year 3 (age 9) Cohort 1 3rd grade E Cohort 2 (not in SiBO dataset) Promotion 2nd grade C D H Grade retention Promotion Promotion Promotion 1st grade A B F G Grade retention Grade retention SAME-AGE COMPARISON

2. Method • Analyses: 2 comparison strategies (continued) • Why? ~ 2 different questions • How do Grade 1 repeaters, at the cost of one extra year of education, develop in comparison to younger children with whom they will eventually finish elementary school? SAME-GRADE APPROACH • How would Grade 1 retainees have developed, had they been promoted to Grade 2 instead? SAME-AGE APPROACH

3. Results • Propensity scores • based on 52 prior student characteristics Promoted students M = -4.47 Retained students M = -0.12

Propensity score stratification Cut-offs for strata based on overlap Division into 10 strata of equal size 3. Results

3. Results • Within-stratum balance in propensity score ≠ = ≠

3. Results • Within-stratum balance in 97% of the observed pre-retention student, class and school characteristics → Retained and promoted children within a certain stratum are equivalent (within sampling fluctuations) in terms of risk factors preceding retention

3. Results • Same-grade comparisons • On average: • during their retention year, Grade 1 retainees show a similar psychosocial functioning in comparison to younger grade-mates who are at similar risk of being retained • but … over time they (mostly) grow significantly slower  they end up showing more hyperactive behavior, feeling less well at school etc. ! One exception: popularity among classmates

3. Results

3. Results sign sign

3. Results

3. Results sign

3. Results ! One exception ! sign sign

3. Results • Same-age comparisons • On average: • Grade 1 repeaters would have developed a similar or even better psychosocial functioning, had they been promoted to Grade 2 instead, both in the short and long run

3. Results

3. Results sign

4. Conclusions and discussion • Overall, Grade 1 retainees do not seem to benefit much from their retention year • over time they grow slower compared to grade-mates, making them end up feeling less well at school etc. at the end of elementary school • while they would have developed a similar or even better psychosocial functioning, had they been promoted to Grade 2 instead →Practical implication: Our results call the practice of Grade 1 retention in Flanders into question

4. Conclusions and discussion • Future research is needed • Sensitivity analyses • Other outcome: growth in math and reading skills • Moderating effects: provision of additional support

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2ND BIENNIAL MEETING OF THE EARLI SPECIAL INTEREST GROUP 18 “Educational Effectiveness: Models, Methods and Applications” Leuven, Belgium 25-27 August 2010 http://www.sigee2010.org -------------------------------------------------------------------------------------------------------------------- Keynote lectures by Prof. Dr. Stephen Raudenbush, Prof. Dr. Robert E. Slavin, Prof. Dr. Jürgen Baumert and Prof. Dr. Jan-Eric Gustafsson

Thank you for your attention! Any suggestions or comments are welcome: mieke.goos@ped.kuleuven.be

1. Introduction

1. Introduction

Presentation Transcript

CH.1 Introduction 1-1 Introduction

1-1 Introduction

1. Introduction

1. Introduction.

1. INTRODUCTION

1. INTRODUCTION

INTRODUCTION 1

Introduction (1)

INTRODUCTION (1)

1. Introduction

1. Introduction

1. Introduction

1. Introduction

1) Introduction

1 Introduction

Introduction 1

1. INTRODUCTION

1. Introduction

Introduction (1)

1. Introduction

1. INTRODUCTION

Introduction 1