On-line Learning Environment for Multilevel Modelling Fiona Steele and Sacha Brostoff

1. On-line Learning Environment for Multilevel Modelling Fiona Steele and Sacha Brostoff Centre for Multilevel Modelling University of Bristol

2. The LEMMA Project • A node of the ESRC-funded National Centre for Research Methods • LEMMA – Learning Environment for Multilevel Methodology and Applications • www.cmm.bris.ac.uk • Research and training components

3. LEMMA Training • Capacity building in the analysis of data with complex structure • Ultimate goal is to move learners to “take-off”, i.e. conducting and publishing multilevel analyses • Different modes of delivery • Face-to-face workshops (3-day + 5-day with time allocated to analysis of participants’ data) • Face-to-face workshop followed by on-line mentoring • Training for established networks (e.g. university departments) • Web-based materials in a virtual learning environment

4. Lessons Learnt • Participants need to be motivated and have time to learn • Best motivated are those with data and research questions that can be addressed through MLM • Experience of on-line follow-up and targeting established groups disappointing • Participants often do not possess pre-requisites for MLM (good understanding of multiple regression) • In practical sessions, tendency to focus on mechanics of using software rather than interpretation

5. Basic Principles • Accessible to anyone with basic statistics training (up to simple regression) • Modules to have 2 integrated components: concepts and practice • Facility for learner’s self-evaluation • Pre-requisite quiz, and regular quizzes throughout materials • Collect data to evaluate materials and inform future training initiatives • Basic user profile information collected on registration • Quiz responses,webstats on patterns of use • Design materials so they can be easily modified by other trainers

6. Types of Material For each module: • 5-minute video giving overview of content • Prerequisites with links to other online resources • 2 linked documents: (i) Concepts and methods, and (ii) practice (MLwiN instructions with interpretation of output) • Quiz questions • Further reading (published research and other online resources) • Glossary

7. Structure of Linked Documents • Concepts • 30-40 pages, split into lessons • Illustrative examples from mix of disciplines • Draw links between fitted model equations, graphs of predictions and verbal interpretation • No reference to software • Expect other trainers to use with little change • Practice • Each Concepts lesson followed by exercises in MLwiN • Thorough analysis and interpretation of one dataset • Trainers can rewrite for other datasets and software

8. Core Materials • Types of variable • Introduction to statistical modelling • Multiple regression (single-level) • Data structures • Multilevel modelling of continuous data • Logistic regression (single-level) • Multilevel logistic regression

9. Future Materials • Substantive examples linked to core materials • Models for other types of outcome • Nominal, ordinal, counts, duration • Models for non-hierarchical structures

10. The LEMMA VLE • Moodle • Open source, widely used • Open University is involved in it’s development • Has a good licensing model • Avoids annual fees • No restrictions on how many users • Sustainable • Does everything we need

11. Demonstrating the prototype • Encouraging engagement • Video overviews • Quizzes • Straw polls • ML Driver’s Licence

12. Further work • Improving navigation • Pilot testing • Link in to the follow up to Athens • Link administration with MLwiN UK free version • More question types • Certainty based marking