Conducting an empirical investigation

1. Conducting an empirical investigation

2. Goals of scientific research • Description of the data • Explanation of the data • Predictions of future data

3. Criteria of scientific research • Objectivity • Repeatability • Reliability • Internal validity • External validity • Occam’s Razor

4. Example: Positiong of ADV-clauses (1) a. When we arrived in Berlin it was dark. b. It was dark when we arrived in Berlin. (2) a. Because he didn’t sleep much Peter is tired. b. Peter is tired because he didn’t sleep much. (3) a. If it doesn’t rain we'll go to the party. b. We'll go to the party if it doesn’t rain.

5. Literature research • Search in your library. • Search in special bibliographies. • Search for recent publications on your topic. • Homepage of researchers who have worked on the topic. • Random search on the internet.

6. Impressionistic inspection of the data From file <adam43.cha> *CHI: oh # Mommy # I think I have +... *CHI: I think I've [=? I] finished with that . *CHI: I think I've [=? I] finished # Mommy . *CHI: think Robin uses those . *CHI: I think we finished with all dose . *CHI: I think we finished . *CHI: Mom # I think we really [?] finished . *CHI: I think I got it . *CHI: I don't think I have it in de right way . *CHI: I think I do it the other # way . *CHI: I think I finished playing with this . *CHI: I think this bag has a top . *CHI: I think I use this for paper . *CHI: I think we use some different colors # alright ? *CHI: I think it is # but it is on a part .

7. Informal pretest If you want to conduct a questionnaire study or an experiment, you should always start with a pilot study.

8. Informal formulation of your theory Linguistic structures are motivated by underlying cognitive and functional pressures. Since there are always multiple pressures (e.g. economy, expressivity, processing efficiency), there is usually competition between factors. - The positioning of adverbial clauses is motivated by competing cognitive and functional factors:

9. Informal formulation of your theory Possible factors influencing the positioning of ADV-clauses: • Meaning: conditional > temporal > causal • Length: short-initial vs. long-final • Information: old-initial vs. new-final • Subject: same subject-initial vs. different-subject-final • Iconicity: matching order of events and clauses • Genre: scientific > fictional > colloquial discourse

10. Is your theory falsifiable? Some theories are not falsifiable: (1) Language change is progress (or decay). (2) Cognitive and linguistic development are related / are not related.

11. Is your theory compatible with other theories/findings? Many linguists assume that ordering patterns are determined by competing functional and cognitive pressures; but in some languages adverbial clauses almost always precede the main clause (e.g. Japanese). Are the Japanese data compatible with your hypothesis about the positioning of adverbial clauses in English?

12. Statement of hypothesis There are two types of empirical studies: 1. Studies testing a given hypothesis 2. Explorative studies Statistical tests may only be used for hypothesis testing; i.e. do not use statistical tests to test data from an explorative study.

13. Statement of hypothesis State your hypothesis informally in terms of a conditional sentence: If x then y The more x the more y X and y are called variables: x is the independent variable, and y is the dependent variable.

14. Statement of hypothesis: Meaning i. If the adverbial clause has conditional meaning, it predominantly precedes the main clause. ii. If the adverbial clause has causal meaning, it predominantly follows the main clause. iii. If the adverbial clause has temporal meaning, it is about equally frequent before and after the main clause.

15. Statement of hypothesis: Length The longer the adverbial clause, the more often it follows the main clause.

16. Statement of hypothesis: Information i. If the adverbial clause resumes information from the preceding discourse it tends to precede the main clause. ii. If the adverbial clause expresses new information it tends to follow the main clause.

17. Statement of hypothesis: Subject If the adverbial clauses includes the same subject as the associated main clause it precede the main clause more often than if it includes a different subject.

18. Statement of hypothesis: Iconicity If the adverbial clauses expresses an event prior to the event expressed in the main clause it precedes the main clause more often than if the adverbial clauses expresses an event posterior to the event expressed in the main clause.

19. Statement of hypothesis: Genre i. In scientific texts, adverbial clauses precede the main clause more often than in fictional texts. ii. In fictional texts, adverbial clauses precede the main clause more often than in colloquial discourse.

20. Experimental hypothesis vs. Null hypothesis Every hypothesis has an alternative. Experimental hypothesis: The longer the adverbial clause, the more often it follows the main clause. Null hypothesis: There is no relationship between the length and positioning of the adverbial clause.

21. Operationalization of variables Meaning • You look up the meaning in a dictionnary. • You analyze each sentence in its discourse context. • You ask people to judge the meaning (directly or indirectly).

22. Operationalization of variables Length • Number of words / syllables • Milliseconds

23. Operationalization of variables Information status • Consider each sentence in its context. Is the information status categorical or scalar?

24. Operationalization of variables Subject • Direct coreference (e.g. Peter … if he …) • Associative link (e.g. The house … if the door …

25. Operationalization of variables Iconicity Is iconicity relevant to conditional and causal clauses? (1) The accident happened because the driver lost control of his vehicle. How do you treat sentences with overlapping events? (2) When we arrived in Jena, it was raining.

26. Selection of statistical measure/test • Type of data (nominal, ordinal, interval) • Type of hypothesis (difference between groups, correlation) • Experimental design (related vs. independent) • Data distribution (normal vs. not normal) • Number of items and number of subjects

27. Structure of an empirical research paper

28. Introduction Background and preview • Why do you conduct the study? • What makes the study interesting? • What is the goal/purpose of your study?

29. Introduction Linguistic phenomenon • Introduce the phenomenon you want to investigate (e.g. relative clauses, acquisition of verb-particle constructions). • Characterize the most important properties of the phenomenon (notably the properties that are relevant to your hypothesis). • Provide examples.

30. Introduction Literature review If there are previous studies, summarize their main findings and state what you intend to do in your study: 1. Look at a new aspect 2. Challenge a hypothesis 3. Test/replicate a previous study If there are no previous studies, make that clear: “This is the first study to investigate ….”

31. Introduction Preview / explicit hypothesis • Introduce the phenomenon you want to investigate (e.g. relative clauses, acquisition of verb-particle constructions). • Characterize the most important properties of the phenomenon (notably the properties that are relevant to your hypothesis). • Provide examples.

32. Method Corpus study • Characterize the corpus (size, type, etc.) • Characterize the data you have compiled from the corpus (total number of ADV-clauses, number of children/subjects) • Describe the treatment of problematic cases and cases you have excluded (e.g. ADV-clauses inserted in the main clause) • Describe coding and provide examples of each category

33. Method Experimental /questionnaire study • Characterize subjects (e.g. how many, selection procedure) • Describe the questionnaire • Describe materials you have used in your study (e.g. toy animals) • Describe the procedure (i.e. how did you collect the data) • Describe coding • Describe treatment of problematic cases

34. Results Descriptive summary • Present tables and figures • Describe the results in tables and figures

35. Results Inferential statistics • Say what type of tests you have used and present the relevant measures (e.g. p-value, F-value, degrees of freedom, effect size, confidence intervals). • If necessary, explain your choice of test. • State what the result of the statistical analysis suggests.

36. Discussion • Provide a short summary of your results. • If possible consider your paper from a broader theoretical perspective and mention implications of your study for related questions. • Future direction of research: Mention open questions: What should be done in the next step? Ideas for an experiment. etc.

37. Appendix If the data are too comprehensive to be included in the text, include them in the appendix. If the data are very comprehensive, you might only present samples of your data in the appendix.