Design for Learning: Improving Patient Safety

1. Design for Learning:Improving Patient Safety Lou Fogg, PhD Project Statistician February 1, 2012 Phase II Site Coordinator Meeting

2. Who Am I? • Lou Fogg, Ace Statistician • I was trained as a mathematical psychologist • I have been working in Nursing research since 1987 • I enjoy working with ‘messy’ data

3. What is our Design? • The study we are conducting is called a ‘randomized cluster design’ • This means that we collect data from a relatively large number of organizations, and we assign them at random to either the intervention group, or a control group

4. Stochastic Matching • In addition, there were a set of three variables (such as the number of new nursing hires expected each year) that we wanted to keep roughly equivalent across the two treatment groups • In order to achieve this initial balance, we used a technique called ‘Stochastic Matching’

5. Stochastic Matching-2 • Stochastic matching is done by generating many different random assignments for these organizations, and then selecting the subset of these randomization schemes that leave the key variables roughly equivalent between our two groups • The fear is that if we let these variables vary freely, there will be a large difference among the groups • This is called a failure of randomization

6. Failure of randomization • Failure of randomization is a serious problem, and is especially serious in randomized cluster designs because the overall sample size (of organizations) is fairly small, and randomization failures occur more often in small samples • So, this makes it more likely that we will be able to make valid comparisons between the two groups

7. Missing Data • Missing data is a serious problem in nursing research • While there are algorithms for ‘imputing’ (statistician-talk for ‘making up’) missing values, the confidence in your results drops the more missing data we are forced to make up…Oops!, I mean impute…

8. Missing data • So while I know that it can be very difficult to collect complete data for a study like this, it is very important for our ability to understand how well the intervention actually works • And the less able we are to make this determination, the less useful these data will be for understanding 1) if the intervention works, and 2) what we can do to make it work better

9. Finally • Some of you will be assigned to the control group and some to the intervention group today. • But you are all precious to a statistician, we need as much of both groups’ data as we can possibly get, in order to obtain valid results. • So, don’t think that because you are in the control group, that we value your contribution any less. It ain’t so…

10. Working with Messy Data • I like to work with ‘messy’ data, but I don’t court disaster • It is more difficult to draw useful inferences from data that is too ‘messy’ • So, I am asking for all of you to help me out and make this the ‘neatest’ (least messy) study that we can

11. Questions? • Questions? • Comments? • Suggestions?

12. And Thank You for being a nice audience • And I’ll leave you with one of my favorite Chicago pictures—the Chicago lighthouse