Introduction to Research Methodology

1. 1 Introduction to Research Methodology January 2007

2. 2 Today and Tomorrow�s Agenda What is Methodology? Types of Research Studies Formulating Research Questions and Objectives Research Designs Research Designs cont�d Sampling Strategies Sampling and Non-Sampling Errors Methodology Limitations Measurement Scales Designing Data Collection Tools Basic Qualitative Analysis Writing Research Proposals

3. 3 What is research methodology?Why should you care about it?

4. 4 Methodology A set of procedures for the purpose of answering a research question(s) that describes: How study participants will be selected How you will analyze the data How and when you will gather data from participants

5. 5 Types of Research Studies

6. 6 Types of Research

7. 7 Exploratory Research Conducted as the first step in determining appropriate action; Helps clearly outline the information needed as part of any future research; Tends to rely on qualitative research techniques such as in-depth case studies, one-on-one interviews, and focus groups. 

8. 8 What are some of the exploratory research studies that you have conducted?

9. 9 Types of Exploratory Research Baseline study (CSR) Needs assessment (DOC)

10. 10 Conclusive Research Conclusive research tends to be quantitative research It can further be sub-divided into two major categories: descriptive and causal.

11. 11 Conclusive Research Descriptive Research Provides data (usually quantitative) about the population being studied. It can only describe the situation, not what caused it.

12. 12 Conclusive Research Causal Research To determine whether there is a cause and effect relationship between variables To determine whether a specific independent variable is producing an effect on another dependent variable.

13. 13 Causal Research There are two types of causal research: Experimental Observational (quasi-experimental)

14. 14 Causal Research There are two types of causal research: Experimental Observational (quasi-experimental) Experimental and observational studies try to demonstrate a causal relationship between two variables.

15. 15 Causal Research Experimental Research: In experimental studies, units (people, etc.) are put into control or exposure groups by the researcher.

16. 16 Causal Research Observational Research: In an observational study, members of the control group are pre-determined. They can be matched according to demographic information to a member of the exposure group.

17. 17 Causal Research Think of some examples of causal research. Are they experimental or observational?

18. 18 Causal Research Examples of causal research: A drug trial for a new medication that has not yet been approved by the FDA. A study testing the long-term health effects of exposure to high levels of radiation. A study comparing asthma rates among children who live on farms with those living in urban areas.

19. 19 Types of Research

20. 20 What you gather data on: Research Questions and Objectives

21. 21 Steps in Formulating a Research Question Steps: Formulate initial research question Literature review based on initial question Refine research question

22. 22 Literature Review What is the purpose of a literature review?

23. 23 Literature Review Purpose provide a context for the research justify the research ensure the research hasn't been done before show where the research fits into the existing body of knowledge outline gaps in previous research key to refining, focusing or changing the research question(s)

24. 24 Conducting a Literature Review What are the major health concerns of Berkeley�s homeless women? If this is your research question, what would you do a search on to conduct a literature review?

25. 25 Refining Research Questions: Considerations beyond the Literature Review How will you define (operationalize) the components of the research question? If the definition is too broad, should you change non-specific variables into specific variables that can be more easily measured?

26. 26 Refining Research Questions: Considerations beyond the Literature Review Does the scope of your research question need to be limited? If necessary, break down the research question into sub-problems.

27. 27 Refining Research Questions Define (operationalize) the components of the following research question: What is the attitude of Berkeley residents towards gambling?

28. 28 Refining Research Questions Operationally define the components of the research question Berkeley residents (permanent residence within Berkeley city limits) Gambling (permanent casinos, bingo halls, lotteries)

29. 29 Refining Research Questions Change: What is the attitude of Berkeley residents towards gambling? To: What is the attitude of Berkeley residents towards the legalized bingo halls and lotteries? What is the attitude of Berkeley residents between the ages of 18 and 25 towards on-line gambling? What is the attitude of residents of North Berkeley towards the legalization of gambling within city limits?

30. 30 Writing Research Objectives Research objectives are simply research questions rewritten in statement form. �How does age, education level and gender influence individual attitudes towards legalized gambling?� What would be the corresponding research objective?

31. 31 How and when you gather data: Research Designs

32. 32 Research Design Primary vs. Secondary Data Quantitative vs. Qualitative Longitudinal vs. Non-Longitudinal Using Control / Comparison Groups

33. 33 Research Design What�s the difference between primary and secondary data?

34. 34 Data Sources Primary Data Observations Direct communication with subjects (surveys, interviews, etc.) Secondary Data Existing data sources collected for some other purpose than the proposed study (reports, databases, results of past studies or surveys).

35. 35 Qualitative vs. Quantitative Research

36. 36 What are the advantages and disadvantages of qualitative and quantitative research?When should you use quantitative or qualitative research?

37. 37 Quantitative vs. Qualitative Quantitative research: Structured research instruments Larger sample size Results easily replicated Information about how often or how many Less in-depth, flexible Qualitative research: Less structured instruments Smaller sample size Results difficult to replicate Information about why and how More in-depth, flexible

38. 38 Quantitative vs. Qualitative Quantitative research: Researcher should know clearly what he / she is looking for Statistical analysis Qualitative research: Researcher may only know roughly what he / she is looking for Analysis is subjective Can be used to determine the focus of follow-up, quantitative inquiries

39. 39 Qualitative or Quantitative? How did you develop your strategic plan? Who participated in the strategy development process? What are the major achievements since this plan was written? What suggestions do you have for improving either the strategy itself or the strategic planning process?

40. 40 Qualitative or Quantitative? How satisfied were you with the service you received today? Completely dissatisfied Somewhat dissatisfied Neither satisfied nor dissatisfied Somewhat satisfied Completely satisfied

41. 41 Qualitative or Quantitative? How regularly do you review progress against the plan and make adjustments as needed? Once a month 2-3 times a year Annually Every other year Never

42. 42 Qualitative or Quantitative? Do you like your job?

43. 43 Qualitative or Quantitative? Why do you like your job?

44. 44 Qualitative or Quantitative? Have you or a family member ever attended Disneyland before? Yes No Don�t Know

45. 45 Qualitative or Quantitative? Is there anything else that you would like the management to know about the service that you received today?

46. 46 Qualitative vs. Quantitative What would be most appropriate? Community needs assessment Study of the frequency of various reported health issues among homeless women Impact evaluation Population census

47. 47 When you gather data: Longitudinal vs. Non-Longitudinal Studies

48. 48 What are longitudinal studies?When should they be used?

49. 49 Longitudinal vs. Non-Longitudinal Best way to test changes in a population over time is to conduct a longitudinal study Data is gathered multiple times over a period of time (multiple time-series design)

50. 50 Longitudinal Studies X = data gathering O = intervention X O X X O X X X X O X X

51. 51 Non-Longitudinal Studies X = data gathering O = intervention O X X

52. 52 Longitudinal vs. Non-Longitudinal What would be most appropriate? Community needs assessment Study of the frequency of various reported health issues among homeless women Study in changes of the health status of homeless women during their first year of homelessness Impact evaluation Drug trial Study of changes in attitudes among Yerevan residents towards racially mixed marriages

53. 53 Who you gather data from: Control and Comparison Groups

54. 54 Control and Comparison Groups What is the purpose of using a control or a comparison group in your research study?

55. 55 Control and Comparison Groups Used to better determine the impact of an intervention on a particular population.

56. 56 Longitudinal Study w/ Control Group X = data gathering O = intervention Control Group: X X Exposure Group: X O X

57. 57 Non-Longitudinal Study w/ Control Group X = data gathering O = intervention Control Group: X Exposure Group: O X

58. 58 With or Without a Control Group? What would be best? Community needs assessment Study of the frequency of various reported health issues among homeless women Study on how changes in the health status of homeless women compare to the changes in the health status of housed women over the course of a year Impact evaluation Drug trial Study of changes in attitudes among Yerevan residents towards racially mixed marriages

59. 59 Control and Comparison Groups Assigning Units / Individuals Units / Individuals may be randomly assigned to a control group or they may be assigned based on whether they have been exposed to a particular intervention or not.

60. 60 What Kind of Research Design? Qualitative vs. Quantitative Longitudinal vs. Non-Longitudinal With or Without a Control Group (who assigns?) Community needs assessment Study of the frequency of various reported health issues among homeless women Study on whether exposure during childhood to excessive sun leads to increased risk of skin cancer

61. 61 What Kind of Research Design? Qualitative vs. Quantitative Longitudinal vs. Non-Longitudinal With or Without a Control Group (who assigns?) Study of whether cross-border community development projects result in changes of perception among community members of the �other� ethnic group Clinical drug trial Study of changes in attitudes among Yerevan residents towards racially mixed marriages

62. 62 What Kind of Research Design? Qualitative vs. Quantitative Longitudinal vs. Non-Longitudinal With or Without a Control Group (who assigns?) Study of the long-term health effects of exposure to nuclear fallout Study to try to determine whether there is a link between childhood immunizations and mental disability DOC Program Design

63. 63 From whom you gather your data: Samples and Sampling Strategies

64. 64

65. 65 What is a Sample? A sample is a finite part of a population whose properties are studied to gain information about the whole.

66. 66 Samples and Sampling Strategy The degree to which your sample mirrors the population from which it comes will depend to a large extent on your sampling strategy. Why is this important?

67. 67 Sampling Strategy The sampling strategy is the way in which you select units from the population for inclusion into your study.

68. 68 Sampling Frame A list of all the individuals (units) in the population from which the sample is taken. What are some examples of sampling frames?

69. 69 Sampling Frame Examples of Sampling Frames: List of businesses registered with the Chamber of Commerce The phone book List of clients served by a resource center List of labor migrants registered with authorities in a particular city

70. 70 Sampling Frame What could you use as a sampling frame? A study on CSR practices in Armenia A study on the health of homeless women A study of the reading habits of children between the ages of 6 and 8 in Yerevan A study of the attitudes of residents of Armenian border villages towards their Georgian neighbors

71. 71 Probability Samples

72. 72 Probability vs. Non-Probability Sampling What�s the primary difference between probability and non-probability sampling?

73. 73 Probability Sampling Types: Simple Random: Units are randomly chosen from the sampling frame Stratified Random: Random sampling of units within categories (strata) that are assumed to exist within a population Systemic Random: Number units within the sampling frame and select every 5th, 10th, etc. Cluster Sample: Clusters (each with multiple units) within a sampling frame are randomly selected.

74. 74 Probability Sampling Cluster Sample: If you want to conduct interviews with hotel managers in NYC about their training needs, you could decide that each hotel in the city represents one cluster, and then randomly select a small number. You could then contact the managers at these properties for interviews. If the subjects to be interviewed are selected randomly within the selected clusters, it is call "two-stage cluster sampling".

75. 75 Probability Sampling Stratified Random Sample: If you want to conduct interviews with businesses in NYC about their SI practices, you could categorize your list of businesses into small, medium and large. Within each strata you could then randomly select a small number.

76. 76 Non-Probability Samples

77. 77 Non-Probability Sampling Types: Convenience sampling: selection based on availability or ease of inclusion Purposive sampling: selection of individuals from whom you may be inclined to get more data Quota sampling: selection on the basis of categories that are assumed to exist within a population What are some examples of these?

78. 78 Non-Probability Sampling Types: Convenience sampling: selecting individuals who happen to be walking down the street Purposive sampling: selecting resource center clients that use many services Quota sampling: selecting businesses for a survey that fall into the categories of small, medium and large

79. 79 Sample Size Quantitative Research: A function of the variability or variance one expects to find in the population (standard deviation), and the statistical level of confidence (usually 95%) one wishes to use. Qualitative Research: As big as possible No definite rules to be followed

80. 80 Sampling Strategy Quiz

81. 81 Sampling and Non-Sampling Errors: Threats to Reliability and Internal Validity

82. 82 Reliability The extent to which results are consistent A study has high reliability if its results hold true across different settings and participants. If a study is reliable its results will generalize to the larger population.

83. 83 Internal Validity Your research has high internal validity when it has successfully measured what it set out to measure.

84. 84 Sampling & Non-Sampling Error The reliability and internal validity of your research can be effected by: Sampling errors (Bias) Non-sampling errors

85. 85 Sampling Error (Bias) Sampling error occurs when an abnormally large number of units with unusual characteristics are chosen from a given population.

86. 86 Sampling Error (Bias) Most often the result of: Flawed sampling frame Poor sampling strategy Small sample size

87. 87 Sampling Error (Bias) Protecting against sampling error: Use a large sample size Probability sampling Most representative sampling frame available

88. 88 Other Non-Sampling Threats to Internal Validity & Reliability Historical Change Participant Maturation Participant Mortality Testing Effects Interviewer Effect Knowing the Study Purpose Poorly designed tool for data gathering High Refusal Rate

89. 89 Threats to Validity & Reliability What might threaten the study�s internal validity & reliability? An interviewer with strong personal feelings about abortion and birth control interviews a teenaged woman about the types of birth control she uses. A five-year study evaluating the impact of a particular teaching method on the thinking skills of children. A donor agency conducts an assessment of community health needs.

90. 90 Threats to Validity & Reliability What might threaten the study�s internal validity & reliability? A longitudinal study on changes in attitudes towards Muslims is conducted w/ data gathered before and after Sept 11th. A year-long longitudinal study on the health status of homeless women. A questionnaire mailed to 100 randomly selected graduate students to measure levels of stress.

91. 91 Methodology Limitations

92. 92 Methodology Limitations Any aspect of the methodology that effects the accuracy (internal validity) or generalizability (reliability) of the study results.

93. 93 Methodology Limitations Which describe limitations to methodology? A baseline study of SI practices: The underdeveloped state of SI in the country. A baseline study of SI practices: Respondents are unwilling or unable to provide detailed information on the level of their companies� SI activities. A baseline study of SI practices: Mentality of the respondents.

94. 94 Methodology Limitations Which describe limitations to methodology? A study of the career choices of adult men: The sample consisted of men between the ages of 20 and 40. A study of the health status of a particular population: Secondary data was used, which was two years old. A study on trends in SI practices: Poor economic growth in Armenia over the past 10 years. A survey of the health of very low-income families: The phonebook was used as a sampling frame.

95. 95 Mara�s Research What might threaten the study�s internal validity & reliability?

96. 96 How you gather data: Measurement Scales

97. 97 What is a Measurement Scale? A scale used to measure responses to closed-ended questions (quantitative data)

98. 98 Measurement Scales Types: Nominal Scale Interval Scale Ordinal Scale Ratio Scale

99. 99 Nominal Scales Categorizes events, attributes or characteristics. Does not express any values or relationships between variables.

100. 100 Nominal Scales What is your sex? Male Female Unsure / Neither Labeling men as "1" and women as "2" does not mean that women are �twice something" when compared to men. Nor does it suggest that 1 is somehow better than 2.

101. 101 Ordinal Scales Categories have a logical or ordered relationship to each other. The specific amount of difference between points on the scale can not be measured. Very common in marketing, satisfaction and attitudinal research.

102. 102 Ordinal Scales Example: How would you rate the service of our wait-staff? (1) Very Good (2) Good (3) Fair (4) Poor (5) Very Poor

103. 103 Interval Scales Rank items in order The distance between points on the scale are equal. No starting or ending points

104. 104 Interval Scales Example: The Fahrenheit scale is an interval scale since the distance between each degree is equal but there is no absolute zero point.

105. 105 Ratio Scales Items are ranked in order The scale consists of equidistant points and has a meaningful zero point Each category should be the same size Categories should never overlap

106. 106 Ratio Scales Examples: age, income, years of participation, etc. What is your age? 0-15 16-30 31-45 46-60 61+

107. 107 Measurement Scales & Variables The type of scale used to measure a particular variable (gender, age, temperature, etc.) also tells you the variable name. For example, gender is measured using a nominal scale so it is called a nominal variable.

108. 108 Measurement Scales Quiz

109. 109 How you gather data: Designing Tools for Gathering Primary Data

110. 110 Data Gathering Tools What tools are you familiar with for gathering primary data?

111. 111 Data Gathering Tools What do you understand by these? Survey Confidential Anonymous

112. 112 Designing Questionnaires Question Order?

113. 113 Designing Questionnaires Orient the respondent! First, briefly describe the purpose of the research study, explain how data gathered by the survey will be used, and by whom it will be used. Is the survey confidential or anonymous?

114. 114 Designing Questionnaires First Questions The first several questions should be relevant to the study itself so that the respondent quickly understands what the survey is about and becomes engaged. The first questions should be straightforward with relatively few categories of response.

115. 115 Designing Questionnaires First Questions Do not place sensitive questions too early! They can lead the respondent to abandon the survey and result in a high non-response rate for the whole survey.

116. 116 Designing Questionnaires Middle Questions Respondents should be eased into sensitive topics by asking them what they think is important or what they prefer. Do not first ask respondents to agree or disagree with a position or sensitive issue.

117. 117 Designing Questionnaires Final Questions Put demographic questions at the end of the questionnaire. Since they are easy to answer, they are much better at the end when respondents are getting tired.

118. 118 Designing Questionnaires Sequence your questions logically! Questions should be grouped in sections Each section should have a logical sequence Avoid making the respondent jump around mentally Help respondents shift gears by introducing a new section. For example, �Now, we would like to get your opinion on some related areas".

119. 119 Pre-Testing Questionnaires Step 1 Administer the tool to a small group of people, who know little or nothing about the research.

120. 120 Pre-Testing Questionnaires Can they clearly understand what is being asked? Does the flow of the questions make sense? Will other people have difficulty? Which questions in particular might pose problems?

121. 121 Pre-Testing Questionnaires Step 2 Test your tool with a small number of people from your sampling frame

122. 122 Pre-Testing Questionnaires Are there too many "neutral", "don�t know" or "don�t remember" responses? Do you need additional questions relevant to the research? Do you need to provide more space for written responses? Did respondents respond appropriately to open-ended questions? Will other people have difficulty? Which questions in particular might pose problems?

123. 123 Pre-Testing Exercise

124. 124 Qualitative Data Analysis

125. 125 Qualitative Data Analysis Steps of Qualitative Data Analysis: Data transcription Organizing data & numbering Familiarization Initial coding Combining and renaming codes Re-coding Generating & summarizing themes

126. 126 Coding A code is a name given to the ideas or concepts that emerge from the qualitative data gathered. �Open coding� is the process of developing codes as you review the transcripts or notes.

127. 127 Coding How did the heart attack impact your life? �I became afraid to stay at home alone.� �I decided to start taking care of my health.� �I made a decision to spend more time with my family.� �I am always afraid that I will have another.� �I try to continue with my life as before, but it�s hard because I�m scared of ending up in the hospital again.�

128. 128 Coding Preliminary Codes: �Fear� and �Change� �I became afraid to stay at home alone.� �I decided to start taking care of my health.� �I made a decision to spend more time with my family.� �I am always afraid that I will have another.� �I try to continue with my life as before, but it�s hard because I�m scared of ending up in the hospital again.�

129. 129 Qualitative Analysis Exercise