Mastering Research Design: Practical Guide for Effective Methods

1. Research Methods II Method Design "When you change how you see the world, your whole world changes.”Aman Motwane

2. Game Plan • Designing your research • Quantitative • Qualitative • Collecting Data • Primary and Secondary Data • Sampling • Measurement • Qualitative Data Collection • Interviews • Surveys • Observation

3. Method design in the thesis process Formulate and clarify your research question Chapter 1 Review the Literature/Theory Chapter 2 Formulate your research design Chapter 3 Answering research question Sampling Measurement Analyze your data Chapter 4 Qualitative Methods Quantitative Methods Write your Results and Discussion Chapter 5

4. The methods process • Research Design • How are research questions answered? • Ex: A mixed method study investigating the relationship between cultural acuity and practitioner effectiveness • Data Collection / Sampling / Setting/ • Who did you study and how did you select them? • Ex: Population is all employees of Intel – sample is Finance Department • Measurement • How are numbers assigned to concepts? • Ex: Quality of Work Life – measured by the QOL index • Data Analysis • How are data manipulated and interpreted? • Ex: Regression Analysis

5. Alternative Research DesignsCreswell (2003) Your design will drive your data collection

6. The Research Onion Time horizons Choices Strategies Approaches Philosophies Techniques and procedures Positivism Realism Interpretivism Deductive Experiment Survey Objectivism Case study Cross- sectional Mono-method Subjectivism Action research Data collection and data analysis Mixed methods Grounded theory Multi-method Longitudinal Pragmatism Ethnography Archival research Functionalist Inductive Interpretive Constructivism Radical humanist

7. Constructionism Knowledge is subjective It’s the product of the context in which its constructed (ex.  Copernicus) Scientific knowledge is not a “direct representation of the natural world” It’s based on systems of thought that are culturally and historically bound Positivism All knowledge is derived from experience whether of the mind or of the senses. Evidence by what we can see, hear, touch, taste, or smell Is the basis of “The Scientific Method” Emphasizes the importance of testability, reliability, validity, and repeatability Asserts that the researcher is “bias-free” Claims “objectivity” Positivism versus Constructionism Your theoretical perspective leads to your research design

8. Quantitative Tests or verifies theories or explanations Identifies variables to study Relates variables Observes and measures information numerically Uses unbiased approaches Uses statistical procedures Largely deductive reasoning Qualitative Takes place in the natural setting Uses multiple methods that are interactive and humanistic Is emergent Fundamentally interpretive Researcher views social phenomena holistically Researcher reflects on who he or she is in the inquiry and is sensitive to how that shapes the study Largely inductive reasoning Qualitative vs Quantitative Design

9. Collecting Primary Data • Survey Research • Interviews • Observation Methods • Experimental Research

10. Collecting Secondary Data • Consider who would have collected data • Governments, NGOs, Association or interest group etc • Good Sources • Statistical Universe • Statistical Abstracts • Historical Statistics • WDI – World Development Indicators • IMF – International Financial Statistics • United Nations – UN Stat • Economic Report of the President - Historical tables Secondary data: data that has been collected for some other purpose

11. Sampling: who AND how many to ask when you can’t study the entire population • You are conducting a study of professional success factors among CC alumni who are external consultants. What is your best approach? • Two key questions: • Is the sample biased? Is the size adequate? • Sampling is often a major limitation • Failure to identify the population, sample of convenience, volunteerism, • Sample size – bigger is not always better – “it all depends…”

12. Sampling Techniques Sampling Probability Non-Probability Simple Random Stratified Random Quota Snowball Convenience Systematic Cluster Purposive Self-Selection Your goal is to produce valid and credible sample data and statistics that match the precision needed for the study

14. Non-probabilistic Sampling

15. How many interviews, surveys? • It all depends • Research Question • Variance of the population • Survey • Minimum of 30 for most statistical tests • More is better – to a point! • Interviews • Ask until you start to get repetition Remember – the intent in qualitative research is not to generalize, but to elucidate the particular, the specific

16. Measurement: Variable definition

17. Measurement: Research questions and variables • Q1: What is the relationship between faculty sleep deprivation and clarity of faculty communication in a thesis workshop? • Dependent variable => clarity of communication • Independent variable => sleep deprivation • Q2: What is the relationship between clarity of faculty communication and student learning during a thesis workshop? • Dependent variable => student learning • Independent variable => clarity of communication

18. Faculty sleep deprivation, communication clarity and student learning

19. Measuring Variables: Scale construction • Scale – any measuring instrument (questionnaire, interview, test) composed of one or more items (questions, observations, etc…) that have logical relationships with one another. • Standardized measures – don’t reinvent the wheel. Source for standardized measures – • Journal articles, books, other thesis or dissertation projects • If you have to create your own • KISS • Measure twice and cut once… the value of beta testing your approach

20. Scale Example: Job Characteristics Hackman and Oldham, 1975 • Measures the degree to which a job is designed to be motivating (the motivating potential of a job) • Five dimensions: skill variety, task identity, task significance, autonomy and feedback • My job requires me to use a variety of skills. • My job allows me to see the finished products of my work. • Doing my job well affects others in some important way. • My job is designed so that I know when I have performed well. • My job allows me freedom to work with minimum supervision.

21. Scale Examples • Participation/Involvement Scale • I feel free to suggest new and better ways of doing things. • I am asked how we can improve the way my work group operates. • Sufficient effort is made to get the opinions and ideas of people in this work unit. • Suggestions made by unit personnel are implemented in our daily work activities. • Satisfaction • In general, I am satisfied with my job. • I have a sense of personal fulfillment at the end of the day. • I am a valued member of my unit. • I would recommend an assignment in my unit to a friend.

22. Validity and Reliability • Validity – does it measure what it is supposed to measure? • Many types of validity that you need to consider • Reliability – is it consistent? • Increase reliability by: • Sample size • Multiple items • Multiple raters • Multiple measures

23. Qualitative Data Collection • Interviews • Surveys • Documents • Audiovisual materials • Observations

24. Surveys / Interviews • Interviews • Telephone Interviews • Group Interviews/ Focus Groups • Self-Administered Questionnaires Mail, Fax, Internet • Survey Monkey

25. Interview Question Design: Good Questions • Sound conversational • Use words participants would use • Are easy to say • Are clear • Are short • Are open-ended • Are one-dimensional

26. Question Design: Probing Questions • How do you mean that? • Would you give me an example of what you mean? • Tell me more about that… • I don’t understand • Anything else?

27. Example of a Questioning Route • Opening Questions • Introductory Questions • Transition Questions • Key Questions • Ending Questions

28. Other questioning techniques • Listing things • Rating things • Picture sort • Drawing a picture

29. Questioning Pitfalls • Avoid asking “why”? • Can put respondents on the defensive • Be cautious about giving examples • Double-barreled questions • Abbreviations / Jargon / Slang • Double-negatives

30. Interview template • Discusses setting • Transcribes interview • Adds reflection notes where • appropriate

31. Survey Monkey Zoomerang InstantSurvey CC has a Survey Monkey account. See Library for account name and password. Major Decisions in Questionnaire Design What should be asked? How should each question be phrased? In what sequence should the questions be arranged? What questionnaire layout will best serve the research objectives? How should the questionnaire be pre-tested? A survey is only as good as the questions it asks Designing Questionnaires/Surveys

32. Phrasing Questions • Simple-Dichotomy • Determinant Choice • Frequency determination • Attitude rating scale • Checklist

33. What can be observed? Human behavior and action Verbal behavior Expressive behavior Spatial relations Temporal patterns Physical objects Observational Studies

34. Observational Studies • Content of Field Notes • Space • Actor • Act • Activity • Objects • Time • Goal • Feeling

35. Observation Field notes

36. Observation Exercise • Partner with someone in class • Decide on what you could observe to answer a question about the CC experience • Go to the place for 5 minutes • Observe and take notes For purposes of this exercise, please do not disturb other classes 2 people can go into room 117 or room 128if you would like to observe a classroom

37. Observation Follow-up • General comments • Most challenging? awkward? • Easy to record? • Impact of observer • How could you manage that?

38. Documents • E-mails, Letters, Memos, reports and minutes of committees • Annual reports • Could ask participants to keep journals (case studies and narrative research)

39. Administration • No class tomorrow – Aju and I will be available to meet with you (Julie after 10:30) • Research Question presentation Monday. Please bring your presentation on a memory stick.