Neil Haigh, (AUT University) 2007

1. Research on the relationship between academic development and student learning: Challenging realities ? Neil Haigh, (AUT University) 2007

2. FORCES Clearly and precisely define role and associated performance accountabilities + performance evaluated robustly, including through research I can’t provide compelling evidence that I do work effectively: that through my work, student learning is enhanced. I can’t readily define my role Some Academic Developers

3. Status of research-based evidence Prebble et al (2004) NZ Ministry of Education The ultimate purpose of academic development practice is the enhancement of student learning outcomes. Research-based evidence that academic development can contribute to the achievement of that purpose is limited in both quantity and quality. the synthesis highlights the paucity of New Zealand research in this area, with only three New Zealand based studies exploring the link between academic staff development and the outcomes for teachers or students and no studies that address Maori or Pasifika contexts Trowler and Bamber (2005) There is little research which clearly links effective student learning with improvements stemming from lecturer training. What studies there have been are either inconclusive, making no claims to generalizability or reliability ….or are small scale….and makeno attempt to link apparently positive outcomes for course participants to the learning outcomes of their students

4. National project, academic developers in all NZ Universities. • Academic development support for teachers and its • impact on learning of students in first year courses. • Academic Developers and teachers collaboratively develop • teaching and learning enhancement initiatives (TLEIs). • Questions: • How can academic developers and teachers work together to enhance students’ learning experiences and achievement? (ways of engaging in academic development) • What impact do teaching and learning enhancement initiatives (TLEIs) developed by academic developers and teachers have on students’ learning experiences and achievement in large first year classes?

5. How can the impact of academic developers on student learning be determined? Other influences Academic Developers Development Activities Teacher’s Thoughts and Actions Teaching and Learning Enhancement Initiatives (TLEIs) Student Learning Impacts Other influences

6. Why? • Guskey 1997 • Confusion/inconsistency over effectiveness criteria • Limited use of student learning as key impact criterion – mainly teacher • impacts. • Misguided focus on ‘main effects’ – inadequate account of subtle • variations in complex array of factors associated with any AD initiative. • Relative neglect of qualitative data and indicators. • Trowler and Bamba 2005 • AD agenda mainly focused on individual teachers. Limited concurrent attention on organizational environments which can limit teacher ‘elbow room’.

7. Why? • Realities of teaching – learning contexts (Haigh and Katterns, 1984) • Factors that may help or hinder teacher • and student learning: • invariably numerous • interact in very complex ways • often unknown, unpredictable • often uncontrollable • have probabilistic relationships with • learning. • Can raise or lower the odds, but no certainties – no recipes.

8. But – literature provides guidelines • For academic developers: • Deepen pedagogical content knowledge vs content knowledge • and generic teaching practices. • Continuous, coherent, extended engagement vs one-off, • disconnected encounters. • Iterative cycles of development activity. • Encounters/activities woven into daily life of teacher. • Align closely with teacher’s immediate goals, interests, needs.

9. But – literature provides guidelines • Academic developers ctd. • Research evidence informs AD and teachers directed to research. • Teacher inquiries into questions/issues encouraged and supported. • ADs model such inquiry. • Data about student responsesgathered, responded to – underway. • Take account of/align changes withother initiatives within • department, faculty, institution.

10. But – literature provides guidelines • And, for researchers • Action research strategy- appropriate given interacting purposes of • change in practice and research on practice; complexities, • ambiguity and changeable nature of the situation; participation of • practitioners as co-researchers. • In-depth case study approach - to recognize the realities of • everyday academic development, teaching, learning. • Multiple case studies - may allow generalizability/situation • specificity to be assessed. • Comprehensive, rich data from multiple sources – to enable • potential influences on teacher and student behaviour to • be teased out and effects traced. • Maximize and triangulate data - as data will vary in validity and • reliability. Some weak in isolation.

11. But – literature provides guidelines • Researchers ctd. • On-going extended data gathering - to acknowledge timeframes for • change (encounter, try-out, evaluate, practise, fine tune, embed). • Timing for data gathering and analysis - must take into account • relevance to on-going teacher and academic development • decision-making, as well as research agenda • Teachers’ first-hand observations and accounts – use when • defensible as well as significant data • Student learning data - needs to take into account the purposes of • specific TLEIs as well as general learning goals • Feasibility and appropriateness of data gathering given need to • maintain authenticity of teaching and academic development – are • important design considerations.

12. AUT University Case Study • Introduction to Hospitality Management: • a large, part one core paper • Concerns about overall learning results • (54% successful completion) • A new teacher assigned to the paper (Chris) • Two academic developers from Centre for Educational and • Professional Development CEPD) working with Chris (Amanda • and Brendan). • One member of CEPD overseeing research dimension of project, • limited contribution to academic development. • We are all are co-researchers as well as practitioners.

13. Academic Developer Influence?? New teacher TLEIs implemented New teacher TLEIs implemented

14. The Data and Data Sources

15. The Data - Features • Influences • external, internal; macro, meso, micro • vary in stability, extent and timing of impact • interactions - complex, cumulative, concurrent • some dissonance likely – response may be compromise • Inner mental life • Insights into • influences aware of • factors determining how influential particular influences are • thoughts influenced (e.g. meanings, explanations) • response to dissonance when experienced • decisions/plans made (incl. TLEIs) • Actions,Interactions (AD, T, S; AD – T, T – S) • actual actions – what, attributes • Material Artefacts • available, created • Outcomes • changed AD/T/S thoughts, plans, actions • teacher learning, student learning, academic developer learning

16. Data Analysis: Language TLEIs • About: • learning and teaching of • discipline/subject and • industry vocabulary • meeting needs of EAL • students

17. Language-Related TLEIs - Analysis

18. Continuing Challenges • Acknowledging longstanding influences and time between potential • and actual influence - requires a life history approach. • Can differentiate the beating wings, but weighting their influence/impact • difficult, if not impossible. • TLEIs can have interacting purposes and impacts (scaffolding, and • language support). Not discrete. • Impact of some TLEIs (revised learning outcomes) not reflected in • specific achievement gains. • Accepting incompleteness of data • (avoid overloading, maintain authenticity) • Accuracy of delayed interpretations • Handling the mass of data