300 likes | 326 Views
Methodology Matters: Doing Research in the Behavioral and Social Sciences. ICS 205 Ha Nguyen Chad Ata. The way we do research matters. Because the meaning of the discovered evidence is based on the way it is obtained 3 domains are always involved when doing research
E N D
Methodology Matters:Doing Research in the Behavioral and Social Sciences ICS 205 Ha Nguyen Chad Ata
The way we do research matters • Because the meaning of the discovered evidence is based on the way it is obtained • 3 domains are always involved when doing research • Substantive content of interest • Conceptual ideas that give meaning to the content • Methodological techniques for conducting research • 3 important keywords • Relations which are between elements within a context or embedding system
The Substantive & Conceptual Domains • Substantive Domain • Elements = Phenomena • Relations = Patterns of phenomena • Studying the “actors behaving towards objects in context” • Conceptual Domain • Elements = Properties of the actors’ behaviors... • Relations = Interconnections between elements
The Methodological Domain • Elements = methods = Modes of Treatment • Includes techniques for • Measuring • Manipulating • Giving instruction to participants • Imposing constraints on the environment • Selecting materials for use • Giving feedback about prior performances • Using experimental confederates • Controlling • Distributing impact
… Methodological Domain… • Relations = Comparison Techniques • Involve 3 features of the system under study • Dependent variables or features • These are measured • Independent variables or features • These are manipulated • All of the rest of the system’s features • These are relevant but not the focus of the study • Comparison performed to assess covariation or association between dependents and independents
Research Methods Are Like Tools • Each method offers opportunities not available by other means • Each method also has inherent limitations • Therefore you should use multiple methods in order to counterbalance the flaws in each individual method • Make sure to chose methods such that one strength can compensate for another’s weakness
The Research Methods Abstract Precision Maximum II III I Obtrusive Unobtrusive IV Realism Maximum Generalizability Maximum Concrete
Quadrant I: Field Strategies Abstract Precision Maximum II III Obtrusive Unobtrusive IV Realism Maximum Generalizability Maximum Concrete
Quadrant I: Field Strategies • Field Study • Natural & unobtrusive • Field Experiment • Manipulating one variable in an otherwise natural system decrease in realism • A bit obtrusive
Quadrant II: Experimental Strategies Abstract Precision Maximum III I Obtrusive Unobtrusive IV Realism Maximum Generalizability Maximum Concrete
Quadrant II: Experimental Strategies • Laboratory Experiment • Concocted system • Increased precision • Decreased realism & generalizability • Experimental Simulation • Similar to lab experiment, but • strives to increase realism
Quadrant III: Respondent Strategies Abstract Precision Maximum II I Obtrusive Unobtrusive IV Realism Maximum Generalizability Maximum Concrete
Quadrant III: Respondent Strategies • Sample Survey • Sampling of the population high generalizability • Low precision • Judgment Study • Here the focus is on the stimulus material, rather than respondents’ attributes High precision • “actors of convenience” Low generalizability over the population • Both often done in neutral settings low realism
Quadrant IV: Theoretical Strategies Abstract Precision Maximum II III I Obtrusive Unobtrusive Realism Maximum Generalizability Maximum Concrete
Quadrant IV: Theoretical Strategies • Formal Theory • Formulating general relations between variables of interest high generalizability • No concrete system involved low in realism • No observation low precision • Computer Simulation • Like experimental simulation, except human participants are not included • Modeling the real world high realism • Outcomes are predictable low precision • Modeling a particular system low generalizability
Questions to keep in mind… • Does the presented material coincide with the strengths & weaknesses of the research strategies that were used? • Is the research evidence well supported by a single or by multiple research strategies? • Are the strategies’ weaknesses and strengths offset such that they counterbalance one another?
Study Design • For every empirical study, observations have to be gathered, combined, and compared • Ex: Given a pair of properties, X and Y, can we tell if X caused Y? • X is independent variable, Y is dependent variable • What techniques are available for comparison? • Baserates • Correlation • Difference
Comparison Technique #1 • Baserates • How often (at what rate, or what proportion of time) does Y occur? • If I do not know how often Y occurs in the general case, then I cannot decide whether the rate of Y in the study is significant (particularly high or low).
Comparison Technique #2 • Difference Question • Is Y present under conditions where X is present? Is Y absent when X is absent?
Comparison Technique #3 • Correlational Question • Do the values of property X covary with the values of property Y? • If X is high, is it likely that Y will also be high? • If X is low, is it likely that Y will also be low? • High positive correlation, high negative correlation, little or no correlation • This technique tells you whether X and Y go together but it does not help in determining causality.
Randomization • Randomization – using a random assignment procedure to allocate “cases” to “conditions” • Each case must be equally likely to end up in any given condition • This procedure does not guarantee to rule out all other factors that could affect the results.
Validity of Findings • 4 types of validity • Internal validity • How close can you come to asserting that the presence of X caused the altered level of Y? • Statistical conclusion validity • Is the given result due to chance? • Construct validity • How well defined are the theoretical ideas in the study? • External validity • How well will the results hold up for replication and how generalizable are the results?
Measure Techniques • Categorized by two questions • Who makes the record of the behavior? • Is the participant aware that his/her behavior is being recorded for research purposes?
Strengths and Weaknesses of the Measures • Self-reports • Strength - Low costs, does not require a lot of time to create and use • Weakness - Reactive • Trace measures • Strength – unobtrusive • Weakness – not very versatile, time consuming to gather
Strengths and Weaknesses of the Measures • Observations • Strength – relatively versatile • Weakness – can only be used on overt behavior, vulnerable to observer errors, can be reactive for visible observers, possible ethical issues for hidden observers • Archival records • Strength – good for researching the past, studies for extensive periods of time • Weakness – public records can be reactive, versatility is limited
Techniques for Manipulating Variables • Selection – selecting cases with desired values and assigning them to appropriate conditions • Direct Intervention – manipulating the specific variable directly
Techniques for Manipulating Variables • Inductions • 3 types • Misleading instructions • False feedback • Experimental confederates • Ethical issues
Concluding Comments • Results depend on methods. All methods have limitations. Therefore, any set of results is limited. • It is not possible to maximize all desirable features of different methods in any one study. • Each set of results must be interpreted in relation to other evidence relating to the same questions.