RESEARCH METHODOLOGY

1. RESEARCH METHODOLOGY IndawanSyahri Graduate Program in Language Education PGRI University

2. THE ‘NUTS AND BOLTS’ OF RESEARCH The tools of Research Model, typology and Ideal types Paradigm Concept Key Terms

3. THE TOOLS OF RESEARCH • A good way to introduce key concepts used in research is to suggest a continuum of research tools that have as their intention ‘explaining’ social phenomena at one end ‘describing’ social phenomena at the other. Explaining Describing Model Ideal type Concept Theory Typology Paradigm Notes: An ‘explaining’ is not to be understood as being better or more significant that a ‘description of an event

4. Theory • Theory is a scheme or system of ideas or statements held as an explanation or account of a group of facts or phenomena. • Theory is a statement of what are held to be general laws, principles, or causes of something known or observed. • Theories are understood as abstract nations which assert specific relationships between concepts. • The abstract ideas and propositions contained in theory are either tested in the field by the collection of data or derived from the data itself. • Theory ought to be ‘falsifiable’; i.e., it must be possible to falsify, then it by definition not a theory. • A good theory will be generalizable – and if possible, predictive – and able to be employed in different contexts to the original.

5. Model • A model is an abstraction from reality that serves the purpose of ordering and simplifying our view of reality while still presenting its essential characteristics. • A model is a representation of reality; it delineates certain aspects of the real world as being relevant to the problem under investigation. • A model makes explicit the significant relationships among the aspects. • A model enables the formulation of empirically testable propositions regarding the nature of these relationships.

6. Typology • Typologies (also called ‘taxonomies’) are similar systems of classification to ideal types. • Typologies consist of a system of categories constructed to fit empirical observations so that relationships among categories can be describes. • These devices can be seen as loose framework within which to organize and systematize our observations. • Typologies do not provide us with explanations; rather they describe empirical phenomena by fitting them into a set of categories.

7. Ideal type • The ideal type is a construct that represents an intellectual description of a phenomenon in its abstract from and does not provide us with explanation. • An ideal type should not be understood as an ‘ideal’ standard in the sense of being perfects, but rather that it is ‘ideal’ in sense of an intellectual construct that may never exist in the real world. • An ideal type is a conceptualization, e.g., the ‘working class’, with which the researcher can compare reality on the ground. • An ideal type does not posit relationships among variables.

8. Paradigm • A paradigm is an institutionalization of intellectual activity which, in effect, socializes students into their respective scientific community. • A paradigm is used in research when students and researchers describe crude and broad grouping of certain approaches to the study of a specific topic; ‘top-down’ and ‘bottom-up’ research paradigm. • A paradigm is used for describing broad approaches to research; ‘positivist’ and ‘interpretivist’ paradigms.

9. Paradigm (continued) • Paradigm: this should be reserved for broad definitions, e.g., the positivist paradigm of research (a realist; patterns and regularities, causes and consequences, explanation based, objectivity, etc.) • Discipline: this usually applies to ‘traditional’ academic disciplines like economics, history, linguistics, etc. • Perspective: an academic perspective pertains to (a) certain approaches within a disciplines, e.g., communicative approach in language teaching, and (b) approaches that transcend narrow academic disciplines, e.g., a ‘interlanguage’ perspective in language teaching.

10. Concepts • Concepts are the building blocks of theory, hypotheses, explanation and prediction. • A concept can be seen as an idea or notions expressed and compressed into one or more words. • A concept carries with it a certain perspective and certain built-in assumptions or ways of looking at empirical phenomena and can be seen as a agreed-upon term among scholars. • A concept is an abstraction of empirical phenomena based on certain assumptions and can be used as a type of shorthand in explanation.

11. KEY TERMS IN RESEARCH Method • All the root of research lies what the ancient Greeks term methodos meaning ‘the path towards knowledge’ and ‘reflections on the quest for knowledge-gathering’. • Research method can be seen as the techniques or procedures used to collate and analyze data. • The method (M) chosen for a research project are inextricably linked to the research questions (QS) posed and to the sources (S) of data collected (RQ M S) • Methods: quantitative research (quantity and quantifying) and qualitative research (interpreting the subjective experiences)

12. Methodology • Methodology is concerned with the logic of scientific enquiry; in particular with investigating the potentialities and limitations of particular techniques or procedures. • It pertains to the science and study of methods and the assumptions about the ways in which knowledge is produced. • A Project’s methodology in concerned with the discussion of how a particular piece of research should be undertaken and so can best be understood as the critical study of research methods and their use.

13. The Interrelationship between the Building Blocks of Research Ontology Epistemology Methodology Methods Sources What’s out there to know? What and how to know about it How to go about acquiring that knowledge Which precise procedures used to acquire it Which data to collect

14. ASKING APPROPRIATE QUESTIONS • Own interest and curiosity. “I wonder if…” or “I wonder why…” • Sources of Research questions • Experiences • Journals • Articles • “critical friends” • The more specific the area, the easier the question should be to formulate clearly. • Looking at previous research you will have better prepared to ask relevant questions.

15. DEFINING THE RESEARCH QUESTION (RQ) • What is Research? • Asking Appropriate Questions • Stating RQ and Hypothesis • The Highlights of Formulating Your Research • What is “a systematic may”? • Internal Validity • External Validity • Feasibility of Research

16. WHAT IS RESEARCH? • A systematic approach to searching for answers to questions. • Asking questions, using systematic approach and end with answers to the questions. • Asking appropriate questions, selecting the best and optimally (the shortest) ways to find answers, and to interpret the findings in a way that we can justify.

17. APPROPRIATE QUESTIONS (continued) Research questions should: • interest us • promise new information or confirm old information in new ways • have reasonable scope • have key terms that are clearly defined and operationalized

18. Stating Research questions and Hypotheses • You should be able to state a more precise research question and have possible answers to the questions as well (the possible answers = hypotheses) • A hypothesis is a tentative statement about the outcome of the research. • While the research problem in a question form, the hypotheses are generally made in form of statements. • The hypothesis may be stated as a null hypothesis (Ho) and as an alternative hypothesis (H1). E.g. Ho: There is no order of acquisition of English spelling patterns. H1: There is an order of acquisition of English spelling patterns.

19. THE HIGHLIGHTS OF FORMULATING YOUR RESEARCH: • IDENTIFY A RESEARCH PROBLEM. • NARROW THE TOPIC DOWN AS MUCH AS POSSIBLE. • REVIEW THE LITERATURE ON THE TOPIC AS COMPLETELY AS POSSIBLE. • STATE THE PROBLEM IN A QUESTION FORM. • STATE HYPOTHESES ABOUT THE EXPECTED OUTCOME OF THE RESEARCH.

20. WHAT IS “A SYSTEMATIC WAY”? • It simply means that the researcher should follow established principles. • This is a matter of avoiding ad hoc solution during the investigation. • By clearly outlining your procedure and maintaining consistency, you can reduce the effect of your personal preferences as well as other extraneous factors which might influence the outcome of the research. • The researcher should consider which approach, e.g., longitudinal or cross-sectional approach will be most efficient in giving answers to the research questions. • If the procedures are flawed, then neither the results nor the interpretation of the result can be valid.

21. INTERNAL VALIDITY • The extent to which the outcome is a function of the factor you have selected rather than other factors you have not controlled. • There are many factors which can influence the internal validity of the research study; among them are maturation, test effect, subject selection, and the history factor. • Maturation refers to the general developmental changes in subjects during the course of the research. • Test effect is another influencing the internal validity of research, e.g., one may have better score in posttest due to learning something in pretest.

22. INTERNAL VALIDITY (continued) • Subject selection may influence the results of research studies. History factor may happen when something else was happening at the same time as the research study being conducted. • Many possibilities are available to researchers to help obtain internal validity; among them are using random procedures and statistical procedures.

23. EXTERNAL VALIDITY • The extent that the outcome of any research study would apply to other similar situations in the real world. • Selection of subjects is a factor influencing external validity. • A random sample can help researcher obtain external validity.

24. FEASIBILITY OF RESEARCH • HOW MUCH TIME YOU HAVE TO INVEST? • HOW CAN YOU GAIN ACCESS? (QUANTITY AND QUALITY OF ACCESS) • HOW MUCH DOES THE PROJECT COST?

25. DESCRIBING VARIABLES • Research Variables Variable vs. Level • Measurement of Variables Nominal Scale Variables Interval Scale Variables Frequency Data Vs. Score Data • Functions of Variables Dependent & Independent Variables Moderator & Control Variables Other Intervening Independent Variables

26. Research Variables • A Variable can be defined as an attribute of a person, a piece of text, or an object which varies from person to person, text to text, object to object, or from time to time. • Variables can be very broad or very narrow. • The broader the variable, the more difficult it may be to define, locate, and measure accurately. • Variables can be assigned to groups of people as well as individuals.

27. Variable vs. Level • We may wish to look at levels within a variable. • E.G., ESL student as a variable may be divided into two levels; foreign students and immigrant students. • We can limit the range or expand them, e.g., bilingual as a variable; either is or is not bilingual (two levels) – yes/no levels. • Proficiency as a variable can be expanded into three levels; advanced, intermediate, and elementary

28. Measurement of Variables • When variables are of the all-or-nothing, yes/no sort, we cannot measure how much of the variable to attribute to the person, text, or object. So how? • Variables will be quantified in different ways depending on whether we want to know how often an attribute is present or how much of the variables to attribute to the person, text, or text.

29. Nominal Scale Variables • A nominal variable names an attribute or category and classifies the data according to presence or absence of the attribute. • The classification numbers have no arithmetic value, e.g., in yes/no notation, 1 represents yes and 2, no. and in south Sumatra dialects, 1=Palembang, 2=Lahat, 3=Komring, 4=others. • There is no reliable method of measuring the degrees, e.g., of how much happiness one possesses at any moment.

30. Ordinal Scale Variables • An ordinal scale variable shows how much the variable attributes a person, text, or object in term of degree. • E.g., the variable happy can be measured in term of degree of happiness; very unhappy—unhappy—happy– very happy • Most researchers prefer to use a 5-point, 7-point or 9-point scale • You may use a Likert-type scale to tap students attitudes: The lesson were boring 1 2 3 4 5 6 7

31. Interval Scale Variables • An interval scale tell us how much of the variable to attribute a person, text or object which more precise measurement. • The intervals of measurement can be described. • Each interval unit has the same value so that units can be added or subtracted. • E.g., the difference in intervals between 1 and 10 in a test of 100 items.

32. Frequency Data vs. Score Data • Another way to think about difference in measurement is to consider whether the study measures how much on an interval or ordinal scale or whether it measures how often something occurs– the frequency of nominal measurement. • This distinction is important because it will determine the appropriate statistical analysis to use with the data.

33. Functions of Variables • To understand how the variables in a study relate to one another, we need to be able to identify their functions. • Functions grow out of the RQ. • Functions depend on the connection we believe exists between the variables we have chosen to study. • One variables are expected to be related to one another or one affects the others.

34. Dependent Variable (DV) • The dependent variable is the major variable that will be measured in the research. • E.g., the construct communicative competence (CC) of a group of students, the dependent variable is the construct and it might be operationalized as the students’ scores or ratings on some measure of CC. • We expect the performance on the dependent variable will be influenced by other variables.

35. Independent Variable (IV) • An independent variable is a variable that the researcher suspects may relate or influence the dependent variable. • E.g., if we want to know something about CC of students, the dependent variable is the score for CC. We might believe that the male and females students differ on this variable. So we can assign gender as the independent variable.

36. Moderator Variable (MV) • Some researchers makes a distinction between IV and MV, others call them both IVs since they influence variability in the DV. • Specifying variables as “independent” and “moderator” helps us study how moderating variables mediate or moderate the relationship between the IV and DV. • E.g., we believe that gender as IV is the most important variable to look at L2 compliments, but we observe that length of residence suspected to influence the DV can be considered as MV

37. Control Variable (CV) • CV is a variable that not of central concern in a particular research project but which might affect the outcome. • It is controlled by neutralizing its potential effect on the DV. • E.g., if you assume that handedness can affect the Ss’ respond in many task, you can institute a control by including only right-handed Ss in your study • Remember when you control in this way, you limit the generalizability of your study.

38. Other Intervening Independent Variables Young adults Additional Education Increased Income Older adults Intervening variable

39. CONSTRUCTING RESEARCH DESIGNS • Determining the design The dependent vs. independent variables Repeated-measures vs. Between-groups designs Mixed designs (split-plot designs) • Practice in drawing design boxes • Classification of designs Studies with intact groups One-shot design One-group pretest-posttest design Intact group—single control Time-series designs Experimental studies Random assignment posttest Control group pretest-posttest Ex post facto designs

40. Determining the Design • In addition to • determining the scope of the RQ, • stating the RQ as clearly as possible, • giving operational definitions to key terms, • identifying variables, • understanding the roles variables will play in the research • how those variables will be observed, we need to plan the overall design of the project. • This is important for it will help us to determine how the data will be analyzed.

41. Dependent vs. independent variables Common questions asked by advisors or consultants: • Tell me about your research. • What are you trying to find out? • So X is your dependent variables and A, B, and C are you independent variables, right? If the RQ is not clearly stated, it may take time to determine this first crucial piece of information: which variables in the research are which.

42. Dependent vs. independent (cont.) • The conditions that we ourselves vary are called independent variables. • Those whose response we are measuring are dependent variables.

43. Repeated-measures vs. Between-groups designs Repeated-measures, e.g.: • A is asked to teach literature to the students of English study program. He is not sure just how appropriate his selection of short stories might be. Five different themes are presented in the stories. Since he will teach the course again in the next term, he wants to be sure of his choices. He asks the Ss to rate the stories on a number of factors. Each S’s ratings of the stories within a theme is totaled as that S’s score for the theme

44. Between-groups • You are comparing native and nonnative teachers of English teaching English in term of their success in teaching. There two groups of Ss you observe, i.e., those who are taught by NT and NNT. NT NNT

45. Mixed designs (split-plot designs) • If we survey all the Ss in English Study program regarding the value they placed language vs. literature courses in their university training, then we have only one group of Ss and we draw comparisons for the courses by repeated measures of that one group. Ratings Literature language

46. Practice in drawing design boxes Decide whether the following studies involve a repeated-measures design or a between-groups design, or both and draw a design box. • In a cross-cultural analysis, you ask Ss to watch a video and judge the appropriateness of small-talk behavior in each of five different episodes. The Ss are Indonesian, Chinese, and Indian. • Using a 30-item questionnaire, you ask native speakers of English and non-native Ss to decide whether a single verb or a two-word verb would be more appropriate in a given context. Each question contains a context and then a choice between a single verb (such as telephoned) and a two-word verb (such called up). Fifteen items appear in a formal context and 15 in an informal context. Does context type influence verb type choice?

47. Classification of Designs • Studies with Intact Groups • The subjects/samples have been assigned on the basis of some principles. • It is impossible randomly to select Ss to begin with. • Ss may self-select a section according to their timetables. • In intact group studies, we are unable randomly to select or randomly to assign Ss for research purposes. • Such designs will not allow us to make causal (cause-effect) statements about the findings.

48. 2. One-Shot Designs • In many teaching programs, teachers want to know whether Ss meet the objective set for the course. • At the end of the courses (and whatever happened during the course) Ss take a test. T - X • This is a very simple design but the results must be interpreted with great care. • The results may not be valid and you cannot generalize from them with confidence.

49. 3. One-Group Pretest-Posttest Design • You give a pretest on the first day of class and a posttest at the end of the course. X1 – T – X2 • By giving the pretest, you can ensure yourself that Ss did not already know the material tested on the posttest. • The pretest-posttest design for one group has many drawbacks that you can consider in your study group.

50. 4. Intact Group-Single Control • If you are working with a class and there are Ss in other sections of that same course, it is possible to establish a control group for the research. • Ss have not been randomly selected for the course nor randomly assigned to sections of the course. • However, you could randomly assign the special treatment to the sections by flip of a coin. G1 (intact) – T – X G2 (intact) – O – X • This design in an improvement over the previous design.