Welcome to the December TLC Chat!

1. Welcome to the December TLC Chat! A Conversation on Qualitative Research Your Hosts: Kelly and Roberta

2. Agenda • Definitions of qualitative research • TLC Fellowship Report-out • Tips and Tools • Example of current TRG work using qualitative research methods • Current research in the field: A comment on employee surveys

3. Getting Started: Speed Connecting • Find someone you don’t know that well or haven’t had the chance to connect with recently. • With your partner, discuss the following questions: What is a recent experience that you have had working on a TRG task/assignment/project? What did you learn from that experience?

4. Getting Started “Qualitative researchers aim to gather an in-depth understanding of human behavior and the reasons that govern such behavior. The qualitative method investigates the why and how of decision making, not just what, where, when.”

5. What is Qualitative Research? • Investigative method which involves exploring issues, phenomena and answering questions • In-depth and holistic • Used in organizations to problem solve, understand ways of doing work, team/interpersonal/organizational dynamics • People-centered • Realistic • Researcher-driven • Subjective

6. How Does This Apply to TRG’s Work? • 2011-2012 TLC Fellowship Topic • Interviewed 5 TRGers- Trainers, PAs and informal conversations with others • Primarily looked at four methods: Interviews, Focus Groups, Storytelling and Online Communication • Developed Tips and Tools 1.0

7. What We Learned • TRG is using qualitative research in client systems like EPA, USAID, PSI and IFC • Even though we rely heavily on interviews, we also use quantitative methods for surveys, course monitoring and evaluation which strengthen our work and credibility with clients • Our own emotions, perceptions and judgments inform how we analyze the data (selecting themes, areas for concern) and what we choose to highlight as important • We have lots of questions about ways to strengthen our research methodologies

8. Some Qualitative Feedback • “As interviewers, we definitely interpret data in our own specific ways. This comes out in terms of how much or little we paraphrase, how we understand the emotions and voice tone of interviewees, etc. This can pose a challenge when we are working on teams on the same project/assignment.” • “Qualitative research enhances the quantitative and vice-versa. As much as we rely on qualitative methods, our clients sometimes like the numbers as well. There are ways that I have taken surveys to see how many times words were repeated and have made charts and graphs to quantify that data for our clients.”

9. CGIAR Assessment of Gender Research Capabilities • Lead Researcher: Roberta, OD/TS: Kelly, PM: Linda, PA: Nina • Assessing CGIAR’s environment for inclusion of gender analysis in agricultural research • Qualitative Methods Employed: Focus Groups with Gender Experts, Interviews with Respected Researchers, Survey • Using both Qualitative and Quantitative Methodologies

10. A comment on employee surveys

11. At your table • read the results from an USOTAT employee survey • as a table, come up with a quick summary that you would present to the client

12. A comment on employee surveys: negativity bias in open-ended questions a study by Poncheri, Lindberg, Thompson, and Surface North Carolina State University 2008

13. Research on organizational climate surveys

14. Hypotheses • Open-ended comments produced in response to an employee survey will be disproportionately negative in tone. • The length of comments will increase as they become more negative in tone. • The satisfaction level (i.e., positive, negative, or neutral) individuals express in their open-ended comments will be positively related to the satisfaction level they express in response to closed-ended questions. • The closed-ended rating produced by respondents who make comments will be more negative than the rating produced by respondents who do not make comments.

15. In terms of comment tone, open-ended responses tended to echo the closed- ended satisfaction ratings of the commenter. Hypothesis 3

16. Results revealed that relatively dissatisfied employees were more likely to provide comments than their more satisfied counterparts. Hypothesis 4

17. Length of comments increased as the comments became more negative in tone. Hypothesis 2

18. Open ended comments were disproportionately negative in tone.. Hypothesis 1

19. What does this mean to our work?

20. Authors suggest • Recipients of survey feedback reports should be warned about the negative tone of open-ended comments so that they do not place undue emphasis on the comments • It is important to note that automatically presenting managers with all of the opened-ended comments provided by survey respondents may mask important group differences and trends.

21. authors suggest • Other survey types (e.g., 360), may not be predominantly negative but instead may be primarily positive as a result of leniency error.

22. Need more research on • wording • placement • mandatory/optional

23. a copy of the article is posted on the VLC.