Helen Walmsley-Smith Staffordshire University

1. Comparing Distance and Blended Learning Designs with retention and attainment data to improve practice Helen Walmsley-Smith Staffordshire University

2. Online Distance Learning Retention • Open University research suggests graduation rates may be as low as 22% (Simpson, 2013)

3. What impacts on retention? • What is said to impact on retention? • External factors • Academic and social integration (Tinto 1987) adapted for online by Rovai (2003) • Transactional Distance (Moore 1993) • Course design, level and subject • Internal factors • Motivation (Bekele 2010), e.g. ARCS theory (Keller 2008) • Previous qualification, IT skills etc , (Harrell II and Bower 2011) • Satisfaction, learning community, support and communication etc, (Hart 2012) • What Methodologies have been used? • Student surveys • Case studies • Data analysis Terminology

4. Impact of Learning Design • Interaction improves satisfaction and retention (Croxton 2014) • Feedback improves learning (Hattie 2012) • No easy way to represent amount/type of interaction or feedback in learning design • Lesson plan? • Little agreement on terminology Terminology

5. Learning design instrument

6. Online distance learning courses • Four convenience courses used for pilot: • LLM Business Law – International Trade and Commerce (level 7) • Physical Activity and Mental Health (level 7) • Money Advice Practice (DEMSA) (level 4) • The Management of Database systems (level 7)

7. Sample Learning Activities Sample activities: Read through the "lecture" slides and make notes Tutorial – exercise e.g., given this information, answer the following: The table shows five transactions T1-T5 which act on database variables A-E. For each operation show the log entry and the values of the variables on disk. [In answer for your portfolio] discuss how each transaction recovers from failure. Refer to the model answers Discuss in the forum the purpose of "intercoms" and how a FAS contract works. How does a FAS contract differ from a FOB contract?

8. Content Analysis • Raters were trained on how to use the instrument • A sample activity completed and reviewed • Raters completed categorisation of 4 courses • Data analysed with Cohen’s Kappa to see level of rater agreement

9. Inter-rater reliability • During training and discussions it became clear that what were assumed to be ‘clear’ concepts were difficult to apply to the actual learning design. • There was little inter-rater agreement • The terminology was changed several times in response to these discussions

10. Learning Design Instrument • Activity: • In a pair/small group, review the sample activities and try to identify any: • Interaction – what type? • Feedback – what type? • Other • Comment on where you disagree!

11. Discussion • Online learning designs are ‘messy’ • Representing learning designs is challenging • Learning design instrument shows potential to describe and represent learning designs quantitatively • Learning design instrument allows for comparison to retention and attainment data as well as to other designs • Learning design does not necessarily show what students /tutors actually do • Final learning design instrument to be used to review a number of courses and the inter-rater reliability evaluated