Learning

1. Learning Environment for Multilevel Methodology and Applications NCRM node based at University of BristolStaffKelvyn Jones Harvey Goldstein Edmond NGFiona Steele Sally Thomas Jon Rasbash

2. Talk content Review some general concepts and diagramswhich help with modelling complex populations Talk about one of the substantive research projects, the Geography of School Effects Outline other major substantive research projects under LEMMA Talk briefly about methodology developments Briefly about the structure of our proposed Virtual Learning Environment - training

3. Complex population structures – general concepts and diagrams In a multilevel model the population is defined as being made up of a number of levels or classifications. For example, students, households, teachers, schools. The relationship between classifications can take one of three forms Nested : for example students within schools Cross-classified: for example students are lie within a cross-classification of schools and area of residence Multiple membership: for example students attend more than one school These three elemental types of relationship between classifications can be combined to model very complex population structures.

4. Area A1 A 2 A3 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Schematic unit diagram for a nested structure With unit diagrams we draw the connections between individual units of each classification Pupils nested within schools, schools nested within area

5. School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 School S1 S2 S3 S4 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 Area A1 A2 A3 Schematic unit diagram cross-classified structure In this structure schools are not nested within areas. For example Pupils 1 and 2 attend school 1 but come from different areas Pupils 6 and 10 come from the same area but attend different schools Schools are not nested within areas and areas are not nested within schools. School and area are are cross-classified

6. P1 School S1 S2 S3 S4 Pupils P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 Area A1 A2 A3 Schematic unit diagram for a multiple membership model Lets take the cross-classified model of the last slide but suppose Pupil 1 moves in the course of the study from residential area 1 to 2 and from school 1 to 2 Now in addition to schools being crossed with residential areas pupils are multiple members of both areas and schools.

7. Classification diagrams Unit diagrams, with one node per unit, are useful but breakdown when we have very many classifications, each with their own set of units. A more useful and abstract diagrammatic tool for thinking about complex population structure is the classification diagram with one node per classification. Classifications connected by arrows are nested, the arrow indicates the direction of nesting Classifications not connected by arrows are crossed Classifications connected by double arrows indicate a multiple membership relation

8. area school area school area school pupil pupil pupil Examples of classification diagrams Crossed and nested Pupils nested within schools Pupils nested within areas Schools crossed with areas Multiple membership and crossed Pupils MM of schools Pupils MM of areas Schools crossed with areas Nested Pupils nested within schools Schools nested within areas

9. Geography of school effects We will be using the Pupil Level Annual School Census(PLASC) and Avon Longitudinal Study of Parents and Children(ALSPAC) data sets. Questions involve how do we properly model students attainment over time, taking account of complex structure including pupil mobility? School effectiveness models have assumed that schools are independent units. That is a child’s attainment,say, is not effected by what is happening in surrounding schools. Given we now have a “quasi-market” in education with schools in some sense competing is this assumption of independent school effects realistic? How might we model the non-independence? In the slides that follow I will refer to the ALSPAC data but similar issues and structures arise with the PLASC data. The slides are not a definitive statement of the research we plan to do-rather an exemplar showing some of the issues and structures that occur across all the LEMMA substantive research projects.

10. Alspac data All the children born in the Avon area in 1990 followed up longitudinally Many measurements made including educational attainment measures Children span 3 school year cohorts(say 1994,1995,1996) Suppose we wish to model development of numeracy over the schooling period. We may have the following attainment measures on a child : m1 m2 m3 m4 m5 m6 m7 m8 primary school secondary school

11. Structure for primary schools Primary school P School Cohort Area Pupil P. Teacher M. Occasion • Measurement occasions within pupils • At each occasion there may be a different teacher • Pupils are nested within primary school cohorts • All this structure is nested within primary school • Pupils are nested within residential areas

12. A mixture of nested and crossed relationships Primary school P School Cohort Area Pupil P. Teacher M. occasions Nodes directly connected by a single arrow are nested, otherwise nodes are cross-classified. For example, measurement occasions are nested within pupils. However, cohort are cross-classified with primary teachers, that is teachers teach more than one cohort and a cohort is taught by more than one teacher.

13. Multiple membership Primary school P School Cohort Area Pupil P. Teacher m1 m2 m3 m4 M. occasions t1 t2 t3 t4 It is reasonable to suppose the attainment of a child in a particular year is influenced not only by the current teacher, but also by teachers in previous years. That is measurements occasions are “multiple members” of teachers. We represent this in the classification diagram by using a double arrow.

14. What happens if pupils move area? Primary school Area P School Cohort P. Teacher Primary school Pupil P School Cohort P. Teacher M. occasions Area Pupil M. occasions Classification diagram without pupils moving residential areas If pupils move area, then pupils are no longer nested within areas. Pupils and areas are cross-classified. Also it is reasonable to suppose that pupils measured attainments are effected by the areas they have previously lived in. So measurement occasions are multiple members of areas Classification diagram where pupils move between residential areas BUT…

15. If pupils move area they will also move schools Primary school Primary school P School Cohort P. Teacher P School Cohort P. Teacher Area Pupil Area Pupil M. occasions M. occasions Classification diagram where pupils move between areas but not schools If pupils move schools they are no longer nested within primary school or primary school cohort. Also we can expect, for the mobile pupils, both their previous and current cohort and school to effect measured attainments Classification diagram where pupils move between schools and areas

16. Non-independence of school effects Schools competing in a quasi-market and increased mobility and priority on educational attainment of middle-class parents can lead to strongly differentiated school intakes. This has been modelled in terms of value added and compositional effects in school effectiveness studies. The question arises are there competition effects after compositional and value added effects have been modelled.

17. School B School B School A School A attainment attainment Prior ability Prior ability Differential school intake –value added effects The model for child attainment is conditional on baseline measures prior to school entry. The school effects therefore relate to progress during schooling not gross effects. Value added school effect taking account of differential school intake Gross school effects

18. A very able student in terms of intake ability, will on average do 1sd better in their final attainment when educated in high ability peer group rather than a low ability peer group Differential school intake - Compositional effects We often find a strong peer ability group effects. School means for intake ability were ranked and then categorised into low(bottom 25%), mid(25%-75%) and high(top 25%). The graph below shows the relationship between pupil level prior ability and predicted pupil attainment for low ability peer groups and high ability peer groups.

19. Modelling non-independence of school effects After adjusting for the value added and compositional effects of differential school intake how would we model any residual competitive effects between schools. One place to start would be to allow a covariance between schools with overlapping catchments. cov(uj, uk) = catchment(j,k) where uj is the school effect for school j and uk is the school effect for school k. catchment(j,k) is 1 if schools j and k have overlapping catchments, 0 otherwise If  is estimated as positive this means schools with overlapping catchments show a positive correlation. If  estimated as negative this means schools with overlapping catchments show a positive correlation.

20. Positive correlation: two schools sharing a catchment. Positive correlation: two schools sharing a catchment. Negative correlation: two schools sharing a catchment. school effect 0 school rank Interpretation of spatially correlated school effects An overall positive correlation between schools with overlapping catchments means that if school j has a positive school effect and it shares a catchment with school k, then school k will tend to have a positive effect. Likewise if school j has a negative effect, school k will tend to have a negative effect. An overall negative correlation means between pairs of schools with overlapping catchments means if one school has a positive school effect then the other school will tend to have a negative school effect.

21. An over simplistic model? Fitting a single parameter to schools the correlation between schools with overlapping catchments may be over simplistic. There may be structural attributes that vary across school pairs with overlapping catchments that cause some pairs to be negatively correlated and some pairs positively correlated resulting in an average correlation of zero. In which case we would wrongly conclude, with the single parameter model, that there were no “competitive” effects. For example, school pairs with overlapping catchments with similar intake compositions may be positively correlated. While pairs with different intake compositions may be negatively correlated, due to the lower intake school being demoralised due to its proximity to its “affluent” neighbour.

22. cov(uj, uk) 0 abs([mean intake for school j]-[mean intake for school k]) Modelling the effect of differential school intake on spatially correlated school effects cov(uj, uk) = 1catchment(j,k)+ 2intake(j,k)catchment(j,k) where uj is the school effect for school j and uk is the school effect for school k. catchment(j,k) is 1 if schools j and k have overlapping catchments, 0 ortherwise intake(j,k) = abs([mean intake for school j]-[mean intake for school k]) Only allowing correlation betweenschools with overlapping catchments may also be too simplistic and more complex distance functions may be needed.

23. Other LEMMA substantive projects • Modelling the duration of episodes in hospital (Steele, Jones) • Data set :Hospital Episodes Statistics • Multilevel, multiple membership event duration models for length of stay in hospital • Voting choice (Johnston, Jones, Rasbash) • Data set:BHPS • Individual, household and neighbourhood determinants of voting abstention and party choice. Multilevel binary and multinomial repeated measures with latent categorical random effects at the individual and household levels. • Mental health and psychosocial development (Rasbash, Lewis, Propper, Jones) • Data sets:BHPS and ABSS • Comparing continuous and discrete random effects for models of psychosocial development. • Modelling group diversity (Goldstein, Burgess, Gordon) • Data set PLASC • Obtaining less biased estimates of diversity indexes, which can then be used as predictor variables.

24. Methodological developments There has recently been a lot of interest in latent categorical random effects. Where variation between units is modelled by unit membership of a set of discrete categories rather than continuous Normally distributed random effects. The main application has been in developmental trajectories, where individual’s growth curves are classified into a set of discrete patterns. MPLUS(Muthen), PROC TRAJ(Nagin), Latent Gold(Vermeuth), GLAMM(Rabe-Heskith et al) We will develop estimation procedures for these models and allow mixtures of continuous and discrete latent random effects for populations structured by multiple classifications To be implemented in the MlwiN software developed by the team.

25. Developing Multilevel Models for Realistically Complex Social Science Data(Goldstein, Rasbash, Steele, Plewis) Missing Data for Multilevel Models(Carpenter, Kenwood,Goldstein) Multilevel Multiprocess Models for Partnership and Childbearing Event Histories(Steele, Goldstein, Joshi) Sample size, MCMC efficiency and identifiability(Browne) Methodologies for Studying Families and Family Effects(O’Connor, Rasbash) The Impact of Local Areas Influences on Individual Outcomes (Burgess, Jones, Propper, Johnson) Related methodological and substantive input from other ESRC projects

26. Multilevel virtual learning environment Built collaboratively by cross-disciplinary team of statisticians, social scientists, ICT professionals and software engineers. How do we provide support to the social science community to deliver the NCRM’s required “step change” in methodological capacity? The team has an extensive track record in running training workshops. The problem is how to provide the follow up to allow workshop participants to convert awareness gained at workshops to improved methodological skills that they routinely use in their research. Not enough expert mentors to go around Can the LEMMA MVLE provide a platform to foster self-sustaining research communities? The LEMMA MVLE aims to facilitate both solo and group learning

27. MVLE Level 2 Online workshops Course management system MVLE Level 3 Research network building Research collaboration tools MVLE Level 1 Training repository Materials for solo learning or group work at levels 2+3 • Notice boards • Asynchronous discussion groups • Synchronous chat with archiving • Calendar/diary-web log • Task lists • Work groups • Content management and version control • Collaborative writing tools • Learning log • Screen sharing • On-line questionnaire/survey tools • Videoconferencing • Data sets and exemplar analyses • FAQs • Self-tests and quizzes • Conceptual modules • Check list for ethics • Links to other online resources • Reviews of software packages for multilevel modelling • Annotated archived multilevel modelling discussions from levels 2 and 3 of the MVLE • Annotated bibliographies • Notice boards • Asynchronous discussion groups • Synchronous chat with archiving • Calendar • Task lists • Content management • Learning log • Screen sharing • Videoconferencing LEMMA MVLE – Learning Architecture