Getting the most out of a biostatistics consultation

1. Getting the most out of a biostatistics consultation John Pearson University of Otago, Christchurch

2. Themes • Who we are • What we do • How we work

3. Full time consultants Assoc Prof Elisabeth Wells Department of Public Health and General PracticeEmail elisabeth.wells@otago.ac.nz Research Interests Psychiatric epidemiology Survey methods Dr John Pearson Department of Pathology Email john.pearson@otago.ac.nz Research Interests Microarrays Genetics Bioinformatics Survey Methods Statistical Computing

4. How to find us • http://www.chmeds.ac.nz/departments/pubhealth/biostat.htm • Stephen Sharp (admin, ground floor) can make bookings • stephen.sharp@otago.ac.nz • Tel:  378 6026 • Supervisors should attend the first meeting

5. Two-way communication issues in consulting Client: Be patient with a biostatistician who doesn’t know your research area and is struggling to understand enough to advise well Biostatistician: Be respectful and try to communicate with non-statisticians, taking account of the amount of statistics they know and trying to translate, if required

6. Biostatistical consulting UOC biostatistics webpage http://www.uoc.otago.ac.nz/departments/pubhealth/biostat.htm says: “Biostatistical consultancy involves advice on study design, methodology, computer software, data analysis and preparation, and revision of articles and reports. The biostatisticians can also act as collaborators during research analysis or on long term studies, and provide training in statistical methods.”

7. Biostatistics webpage (ctd) “There is no charge for consultation services provided by biostatistical consultancy staff for students and members of staff of the School and the Canterbury District Health Board. However biostatistical work should be costed as part of research grant applications.” WE ARE FREE BUT IN SHORT SUPPLY

8. Process • Think first (preferably think lots) • Read the literature • Consult a biostatistician, well in advance of an ethics or grant application In this sequence each step may result in your going back to an earlier step and revising

9. Reasons for revising Examples: • Literature may show your project has been done or that you need to measure other variables • The biostatistician’s questions may require you to re-read articles or search again • There may be ways of analysing the data which you had not thought of so you may revise your aims and objectives

10. Looping back and revising • Looping back and revising is part of the usual process of setting up a research project • The more you can think through things in advance the better • Nonetheless revision is often required as you think further, and in more detail, and is nothing to be ashamed of

11. What to bring to a biostatistician • Aims/objectives/hypotheses • Suggested study design • Expected size of effect based on the literature/clinical experience orthe desired precision of some estimate • Any key papers • Draft data collection instrument (if available)

12. Study design • Observational studies:- cross-sectional studies- cohort studies- case-control studies • Intervention/experimental studies:- for human trials usually only one or two interventions are used- for laboratory studies there will usually be a dose variable with several levels and often other experimental conditions as well

13. Study design Within the main study designs there will be options: • Is a case-control study unmatched or matched (individually or frequency matched) • In a treatment trial are there parallel groups or a cross-over design • In follow-up of patients is it better to study more patients or fewer more often?

14. What sample size do I need • This is sometimes asked with no context • That is like asking a travel agent ‘What does it cost to travel?’ • The answer depends on what you want to do.

15. Effect size or single estimate When planning a study you need to decide if you are: • Estimating the size of an effect (eg the difference between Tx A and Tx B) and testing the hypothesis that there is an effect • Estimating a single quantity (eg the proportion of the population living below the poverty line) – no hypothesis here

16. Effect sizes expected • Use the literature as a guide to the size of effect you might find (similar studies, similar treatment for different problem, size of effect that would change clinical practice or policy) • Consider differences in proportions, odds ratios, or differences in means (try to find out the standard deviations if possible)

17. Effect size and sample size • For comparisons, expected effect sizes have to be thought through before a statistician can come up with a sample size • Various scenarios can be looked at before deciding one is best • If sample size is fixed, then it is possible to see what size of effect could be detected

18. Power and sample size • Ethics applications mention the power of the study. This is the probability of obtaining a statistically significant result in your study if the world is as you suppose it (ie there is an effect of the size you propose). • Power is often set at 80% (odds of 4:1) but sometimes at 90% or 95% for definitive studies.

19. Power (ctd) • If the power is less than 80% then there is not much point in doing the study • Biostatisticians don’t like multiple small studies which are too small to detect the likely effects of different treatments

20. Hypothesis testing vs confidence intervals • In the mid 1980s biostatisticians tried to move health research away from significance testing (results reported as significant or not) to confidence intervals around an estimate • The estimate shows the most likely effect and the confidence interval shows the interval within which the true value is likely to fall

21. Precision of single estimates • Sometimes there is no ‘size of effect’ but just the precision of a single estimate. • This is often how sample sizes for national surveys are set but it also can apply to small surveys • You may want to know if only a small, moderate or high proportion of patients had a 12 month follow-up outpatient visit • However if the survey result is 40% you want to know the precision (“margin of error”); 10%-70% is a different estimate from 35%-45%

22. Planning for precision Planning for precision is based on the size of the confidence interval expected: • If 4 out of 20 children have asthma then the prevalence (P) is 20%, with a 95% confidence interval (CI) of (8%, 42%) • For 20/100, P=20%, 95%CI=13%, 29% • For 40/200, P=20%, 95%CI=15%, 26% Agresti, A., and Coull, B. Approximate is better than 'exact' for interval estimation of binomial proportions. The American Statistician 52: 119-126, 1998

23. Planning for precision Planning for precision is based on the size of the confidence interval expected:

24. Planning for precision Precision depends on P so it is necessary to look at plausible values for P • N=100 P=50% 95%CI=40%, 60% • N=100 P=5% 95%CI=2%, 11% What precision do you want? What precision can you afford?

25. Diminishing returns • Precision and power depend on the square root of N, not N • To 1/2 a confidence interval you need to 4x the sample size, (not 2x it) • Costs increase linearly with sample size • There are diminishing returns from larger and larger samples

26. Data collection Biostatisticians can advise on data collection: • This should be planned well in advance of starting collection • A database or EXCEL spreadsheet should be set up • Procedures for checking data are required

27. Questionnaires • Most biostatisticians have experience in questionnaire design • Biostatisticians will think about data entry issues, coding and analysis of the data, not just the questions themselves • Questionnaires may look easy but there are many traps

28. Computing • Almost no research is done without computers now • Biostatisticians can advise on what software packages to use (but we use only some of them so if you choose another then the support will be limited.)

29. Analysis Whether you carry out your own analyses or whether a biostatistician does them depends on: • How much you know • Whether or not you are a student (students have to learn to do their own analyses) • How complicated the analyses are

30. Writing up the results Biostatisticians can help with this: • For papers or reports biostatisticians often write up the statistical methods and may write some or much of the results • Students have to write their own results but will receive guidance from their supervisor(s) and may check with a biostatistician

31. Responding to referee criticism The point of referees is to find weaknesses in studies (as well as to praise good points). Depending on the issues raised biostatisticians may be involved in: • Rebutting criticism or making design changes suggested by grant reviewers • Rebutting criticism or carrying out additional analyses from journal or report reviewers

32. Summary • Biostatisticians can be involved with all stages of a project from design to publication • Sometimes they are required only for technical advice at some part (but fixing problems which occurred because advice was not sought earlier is not liked) • Sometimes they are part of the team

33. Take home messages • Check statistics early • Statisticians are mere mortals