Lecture 2 What is a Theoretical Contribution?

1. Research MethodsDoctoral Program, National Research University Higher School of Economics Dr C S LeonardJune 2011 Lecture 2 What is a Theoretical Contribution?

2. Outline • What is a theoretical contribution? • What is a concept? • A variable is a concept • Validity • External, Internal Research Methods

3. Theory Publishing (Evaluative) Criteria Research Methods

4. Practical guidelines • Choose a hypothesis important in literature but for which no systematic study exists—decide in favour or against the hypothesis • Choose a theory you suspect is false and investigate if it is, indeed, and what alternate might replace it • Design research to illuminate unquestioned assumptions in the literature • Argue that a topic of importance has been overlooked • Show relation of theory from one literature to separate problem from another Research Methods

5. Revision of formal theories? • More than just adding a variable • Showing how it changes perception of the problem • Producing data that are inconsistent with a theory Research Methods

6. Borrow from other disciplines? • Shed new light on old theories • Investigate qualitative boundaries of a theory, not just quantitative ones • Show why a theory will not work in a new application, not just that it does not • Focus on multiple elements of a theory, not just one inconsistency Research Methods

7. Evidence required for revising a theory • Marshal compelling evidence. • This evidence can be logical (e.g., the theory is not internally consistent), • empirical (its predictions are inconsistent with the data accumulated from several studies), or • epistemological (its assumptions are invalid-given information from another field). Research Methods

8. Provide remedies, alternatives • Is the original actually inferior, • Or simply the best we can do? Research Methods

9. Is it publishable? • How radically new is the idea? • Is it linked to evidence, will it change practice? • Why so? Are the underlying logic and sup-porting evidence compelling? Are the author's assumptions explicit? Are the author's views be-lievable? • Well rounded, broad, deep understanding? • At professional standards? • Who cares? Why now? Research Methods

10. What theory is not: (1) references • For example: • "This pattern is consistent with findings that industrialization produces growth (Marx 1880, Solow 1956) This sentence lists publications that contain conceptual arguments (and some findings). But there is no theory because no logic is presented to explain why industrialization produces growth Research Methods

11. What theory is not:(2) Data • Introduced to show what patterns are there • Not why they are there • This approach relies on brute empiricism, • where hypotheses are motivated by prior data • And not the relevance of theory Research Methods

12. (3) Lists of variables and definitions • Lists of concepts, similarly, are not theory • Need connections between variables explained Research Methods

13. (4) Diagrams alone • What is needed is causal inference • Time horizon and change over time • Diagrams as Stage props, not the performance • Logic needs to be spelled out, verbally Research Methods

14. (5) Hypotheses: what is expected to occur Well-crafted conceptual argument includes hypotheses They serve as crucial bridges between theory and data Making explicit how the variables and relationships that follow from a logical argument will be operationalized. These are statements about what is expected to occur, not why it is expected to Research Methods

15. Ideas and concepts • A theory generally stems from a small set of research ideas, not a list of testable hypotheses Research Methods

16. Qualitative/Quantitative Distinctions Qualitative: Research Questions Quantitative: Set of hypotheses (nul or directional) Research Methods

17. Evidence for your hypotheses/testing theory When theories are particularly interesting or important, empirical support can be partial: • a small set of interviews, • a demonstration, • experiment, • a pilot survey, • archival data Research Methods

18. Evidence for your hypotheses May point to why a particular process might be true. Subsequent research indicated Mayshow whether the theoretical statements hold up, or whether they can repeatedly be falsified Research Methods

19. External Validity Economics Research Methods

20. Two concepts of external validity, as applied to theory 1: Replicability, that the study would hold for other persons, settings, times, or places. Only one form of validity when the objective of research is to test theory Research Methods

21. External validity 2: Applicability It arises primarily through severe and rigorous tests of theory rather than by attempts to incorporate "real world' variables into individual studies designed to test theory. Such variables only become important in the context of evaluating interventions based on theory. Research Methods

22. Construct validity One needs to make a distinction between (1) the construct validity of a concept, as reflected in the convergence (and discrimination) of some particular set of operationalizations of it, and (2) the construct validity of a relation between two concepts, as reflected in the "fit" of that relation within some nomological network. The fit is linked to considerations of external validity. Research Methods

23. Sampling is critical for validity Four major sampling strategies that might be adopted visa vis any one aspect or facet of the events under study • Sampling homogeneously over the entire study • Sampling several subsets, each homogeneous within subset on the facet but differing on it between subsets, so that all the subsets together span the whole range Research Methods

24. Sampling 3. Sampling heterogeneously. but in a way that yields an overall distribution of the facet among the cases within the study that reflects (is representative of) the distribution of the facet among cases "in nature” 4. Sampling heterogeneously on the facet but without regard to representativeness. Research Methods

25. Sampling threats to validity • These four strategies offer different opportunities for—and pose different threats to—the exploration of the external validity of any given set of findings with respect to the facet in question. • Define external validity: • the deliberate and systematic search, on a number of facets, for both the scope and the limits over which that given set of findings does and does not hold. Research Methods

26. Reporting • Validity as robustnesss • Where are the boundaries? • Can findings be replicated? Research Methods

27. tests for validity STAGE 1 STAGE 2 • Design: Instrument validity instrument use validity Comparison validity execution validity • Hypothesis: Experiment: Construct validity Operational Nomalogical validity Predictive Research Methods

28. Checking Validity • Observation: State validity Attribute Pattern validity Process Research Methods

29. Theory What is a good theory? Research Methods

30. A theory does not come first in research • Theory: • A reasoned and precise speculation about the answer to a research question, including a statement about why the proposed answer is correct • Implies several specific descriptive or causal hypotheses • Is consistent with prior evidence about a research question Research Methods

31. Choosing a theory • Possibly, it will be wrong • It must be falsifiable-what evidence would convince us? • Requires observable implications (many) • Observe the principle of parsimony (a judgment about the nature of the world, which is assumed to be simple) • These rules assume you have not yet collected the data, which can be used afterwards, to modify your theory Research Methods

32. After your research • What to do with a theory • Expand it, dropping a condition or variable, to, say, all countries, all regions • Make it embrace a larger range of phenomena • Do not do the opposite • If the theory does not work for some of your observations, do not squeeze new theory out of a revised and qualified research base • Ie, make it less restrictive, but not more restrictive—unless you go back and collect more data and observations beginning with the new hypothesis Research Methods

33. Data • Ensure reliability • Maximize leverage • Explain as much as possible with as little as possible • Increase numbers of observable implications with confirmation • Improve the theory, improve the data • From the beginning, list all the possible implications of your hypotheses—outcomes, responses, interviews, at all levels Research Methods

34. Scepticism • Report negative findings • Report alternative hypotheses Research Methods

35. Theory Concept How to build a concept Research Methods

36. Concept Formation • Rigorous approach to concept formation (Osigweh) • Defining meaning boundaries (what the concepts do not include) • Minimizing concept misuse, confusion Research Methods

37. Concept precision • General, yet precise • Moving a concept from low to high levels of abstraction • Economic onceptsmust span several contexts • Yet they must provide a precise guide to what is not included Research Methods

38. Concept Imprecision • Useless: new contexts too easily fit • New topics can be suspect, vulnerable to herding by scholars (CR impact on performance) Research Methods

39. Concepts • Can be decomposed taxonomically Research Methods

40. Variables A variable is a special kind of concept. It is a classification into two or more mutually exclusive and totally inclusive categories (e.g., Hage, 1972; Smith, 1975). Research Methods

41. Concepts Stretching and Travelling Research Methods

42. Concept travelling: Positive contribution Concept travelling, in this sense, means that the concept is precise enough to allow researchers to define it in the same way, and so to test it in a wide range of situations—that is, that the concept is a universal. Research Methods

43. Example • Puzzle -- Each use creates essentially the same image anywhere: • (a) a maze of activities that cannot be correctly solved outside settings or mental frameworks that prescribe following certain paths or specific combination of paths;(jigsaw?Rubik’s cube)? • (b) an exercise of figures, numbers, or behaviours that cannot be engaged in or successfully disengaged in, except through a specific order or steps, procedures, or systems of thought processes; • (c) activities that when completed cannot fail to yield exact answers, known solutions, or ultimates that could be exactly predicted beforehand. Research Methods

44. Concept Stretching (to be avoided) Multi level governance when refracted through the lens of lived political experiences-- notions of policy transfer and of the complex politics of scale of interventions in time, place and space--needed Research Methods

45. Too vague (stretching) • Decentralization • Would be misapplied in different societies • Too broad Research Methods

46. Too narrow, as well as vague: observational concepts • Can be moved up or down the abstract ladder • Communication puzzle, group, decision, problem, conflict, and participation. Research Methods

47. Research Methods

48. Research Methods

49. Research Methods

50. Moving to the upper left corner • Move up the ladder by negating • Concepts with negation are precise • Those without are indeterminant Research Methods