Statistical Consulting – Asking Good Questions

1. Statistical Consulting – Asking Good Questions Presented by Brendan Lehman

2. General Knowledge around the Statistical Problem • It is important to gain context around the statistical problem you are asked to solve • First understand the substantive problem rather than the statistical issues • The statistical issues may be from the consultees misunderstanding • Avoid trying to immediately jump into solving the statistical issue of the consultee without working through the substantive problems • Statistical tests may go by a different name in other fields • It is important to ask about your consultees’ level of statistical expertise • Can determine the method by which you share your thinking about the problem • Particularly important in checking if you are on the same page as the consultee • Make sure to avoid jargon based on the consultees level of expertise

3. Types of Questions • Leading questions: avoid asking questions where the “desired” answer is implied in the question • Example: Were you able to get a random sample of this population? • Leading questions often give desired answer regardless of the correct answer • Forced choice questions: a multiple choice question that includes all possible responses • Example: Are you able to change the order in which different animals receive the treatments, or must the order be the same for each animal? • Leaving out options may imply only those options included are acceptable

4. Types of Questions • Open probe questions: questions to get general information regarding the problem • Example: What factors are likely to affect the response? • You can also check if there is other relevant information about a topic before moving on • Closed probe questions: questions that ask for specific information • Example: Are there any missing data? • Generally give a yes or no answer to obtain very specific information

5. Important Comments on Asking Questions • Asking questions may expose weaknesses in one’s thinking • Naturally, this isn’t a very enjoyable process • Important to ask questions diplomatically or not frame these questions as questions at all • Important to guide the questions toward understanding rather than exposing weaknesses in the consultees understanding/thinking • Remember this is a consultation, not an interrogation • Consultee should feel comfortable to admit ignorance • The general process is 1) client’s question 2) statistical framework 3) statistical solution and 4) client’s solution as an iterative process

6. Works Cited • Flom, Peter. Do You Have A Statistics Question, Statistical Analysis Consulting, 20 Feb. 2010, www.statisticalanalysisconsulting.com/how-to-ask-a-statistics-question/. • "Thoughts on Statistical Consulting." Statistics, Genetics, Programming, Academics, 2 Apr. 2013, kbroman.wordpress.com/2013/04/02/thoughts-on-statistical-consulting/. • Rendahl, Aaron. Asking Good Questions, University of Minnesota School of Statistics, 10 Feb. 2010, users.stat.umn.edu/~rend0020/Teaching/STAT8801-2010Spring/slides/06-clients2-questions/questions-handout.pdf. • Rindskopf, David. "Rindskopf’s Rules for Statistical Consulting." Statistical Modeling, Causal Inference, and Social Science, 25 Jan. 2008, statmodeling.stat.columbia.edu/2008/01/25/rindskopfs_rule/.