George TH Ellison PhD DSc Division of Epidemiology and Biostatistics

1. George TH Ellison PhD DSc Division of Epidemiology and Biostatistics Leeds Institute of Genetics, Health and Therapeutics g.t.h.ellison@leeds.ac.uk Wendy Harrison (Leeds) and Graham Law (Leeds) Johannes Textor (Utrecht) Teaching DAGs to support MBChB students design, analyze and critically appraise clinical research

2.  DAGs help us distinguish between: - nonparametric theoretical models of causality; and - optimal parametric statistical models for testing these  DAGs can be used at every stage of quantitative research: - optimising the number of variables measured (design) - optimising adjustment for confounding (analysis) - evaluating published statistical models (critical appraisal) Why teach statistical modeling in MBChB? • Most clinical research/audit uses an observational design • Most observational research is poorly/implicitly modelled Why DAGs?

3. What is a DAG (Directed Acyclic Graph)?  A type of ‘causal path diagram’ with: unidirectional (‘causal’) arrows linking variables; and no circular paths

4. Challenges facing the application of DAGs • Algorithms are tedious and time-consuming to apply • DAGs with more than a handful of variables are complex DAGitty.net applies algorithms automatically

5. Cross-tabulation might help as variables  causes one above  caused by one above  no causal relationship

6. Comparing three ways of drawing DAGs • Three one-hour tutorials using three approaches: • (i) ‘graphical’; (ii) ‘cross-tabulation’; and (iii) ‘relational’ •  Each approach evaluated based on: • - how many variables were included in the DAG • - mediators/confounders correctly identified* • - student feedback on ease of use and interpretation •  All participants were third year MBChB students who had • completed a year-long critical appraisal course •  The context was a published paper on an accessible topic: • ‘determinants of pregnancy-associated weight gain’

9. Handouts contained 10, 20 and 30 variables:

10. Group using ‘graphical’ approach:

11. Group using ‘cross-tabulation’ approach:

12. Group using ‘cross-tabulation’ approach:

13. Group using ‘relational’ approach:

14. Group using ‘relational’ approach:

15. What do I (now) think the DAG should be?

16. Focusing on the ‘relational’ • None of the students were able to attempt including more • than 10 variables in their cross-tabulation • 86% correctly identified covariates that should have been • classified as ‘mediators’ by Harris et al. 1999... • Fewer than 5% correctly identified the only covariate that • is likely to have acted as ‘confounder’ (maternal age) • A disproportionate use of ‘competing exposure’ as a • classification for covariates that are likely to have been • ‘mediators’ suggests students were reluctant to identify • ‘exposure’ as a potential/likely/theoretical cause

18. Focusing on the ‘relational’ • Most students found it ‘Difficult’... •  Why? • - understanding DAGs and DAG-related terminology • - “Time consuming” debating/agreeing links and directions • - “...so much depends on variation and opinion”

19. Summary • It is feasible to teach DAGs to MBChB students •  Most students are capable of distinguishing between • ‘confounders’, ‘mediators’ and ‘competing exposures’ •  ‘Cross-tabulation’ and ‘relational’ were slower to apply • but less likely to result in errors •  Suggestions for future development: • - include a quiz to strengthen initial knowledge • - (perhaps) avoid group work (at least initially) • - reward recognition of ‘subjective causality’ • - explore an approach that involves removing rather than • including causal paths (‘arcs’)