Quantitative and Qualitative Methods

1. Quantitative and Qualitative Methods Sebastian M. Rasinger Research Methods in Linguistics. An Introduction. Second Edition London: Bloomsbury S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

2. Agenda • Pros and cons of questionnaire use • Design • Coding S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

3. Questionnaires - Basics • Theoretically suitable for both quantitative and qualitative data • But practically a lot more useful for quantitative approaches • large amounts of data with relatively little effort? • feedback/return rate • reliability • extensive piloting of usually more than 2 versions before final run S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

4. Basics: know what you want… • once the questionnaire is completed, there’s no way back  needs to be ‘perfect’ • clear aim/hypothesis • is questionnaire the best methodology for your research? It very often isn’t! • Questionnaire versus (semi-)structured interviews S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

5. Questions: Phrasing • think about what exactly you want to know (again!) • age or year of birth? i.e. age or cohort effects? • Avoids coding-nightmare afterwards • definite point in time or period of time? • ‘since when’ vs. ‘for how long’ • sex or gender? • Biological ‘setup’ vs cultural construct? • children: young? pre-school? Post-comp.ed? general? daughters? sons? • Unique anonymous identifier S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

6. Question phrasing: avoid… • Ambiguous terms and questions: be clear what you want. You know what you mean, the respondent doesn’t • Long, syntactically complex questions • Double-barrelled questions: Do you like your job and the working conditions? 2 questions – ask 2 questions. • Very general questions: Do you think X is a good idea? usually tell us very little S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

7. Question phrasing: avoid… (2) • Leading questions: Don’t you agree that Mitt Romney would have made an awful president? • Questions which include negatives • Most people skim read • Technical terms: you’re an expert, your respondents most likely not • Does the respondent have the requisite knowledge? Is your respondent literate in the language of the questionnaire? S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

8. Open vs closed questions • Closed: • Quick to respond to and easy to code/process • But: limited scope of info; precise phrasing • Open • Larger scope with regard to info obtained • But: tedious to complete; better interviews instead? (NB: no feedback mechanism with questionnaires!) S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

9. Closed questions – answer options • Should be clear, precise and not overlap E.g., age group: 18-20 21-2426-30 18-2020-24 25-30 • Have you learnt English before or after puberty? Yes – No S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

10. Closed questions: multiple items • Should include most/all possible answers • Closely linked to the question • Thorough research at design stage! • Open ended option for ‘emergencies’ only ‘What are your thought about this workshop? – tick one box only’  complete waste of time  generally provides useful information  is fascinating  I couldn't live without it  other (please specify) _________________ S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

11. Multiple items – layout • Layout – which box belongs to which answer?  Strongly agree  agree  undecided  disagree  strongly disagree vs. • Strongly agree • Agree • Undecided • Disagree • Strongly agree S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

12. Scales and semantic differentials • E.g. Likert scales • Measuring attitudes/opinions/agreement/ disagreement • Even or odd number of items? • Neutral option or not? • 3, 5 or 7 point? • Depends on degree of accuracy/detail required: to few  not detailed enough; too many: tedious S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

13. Scales (cont’d) ‘Research methods is the most important module of my MA’ S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

14. Number and sequencing • As many questions as you need… but not more • E.g., age and DOB together often unnecessary • Simple, easy and uncontroversial qs first • Thematic blocks (but beware of response sets!) • Give clear instructions throughout: • ‘If “yes”, go to q.14. If “no”, continue with q.3’ S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

15. Problems… • Response sets: • Respondent’s tendency to agree of disagree • Acquiescence responses/social desirability responses • Do not reflect respondent’s actual opinion but what respondent thinks it the ‘right’ or desirable answer  Reliability and validity issues S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

16. … and ways to solve them • Bi-directional questions: • Measure opinion one way • And then the other way round ‘Research methods modules are completely unnecessary’ (yes/no) ‘Research methods modules are important to develop key skills.’ (yes/no) S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

17. Coding • =translation of data into numerical values so it can be used for quantitative analysis • Open questions: either qualitative approach or development of detailed coding scheme • Closed questions: • Data that is already numerical (e.g. DOB, age)  can be used straightaway • Data that is not numerical S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

18. Coding (cont’d) • Assign particular numerical value to particular answer option e.g. gender: male=1 female=2 or vice versa e.g. age groups (not age!) 18-20 =1 21-24 =2 25+ =3 S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

19. Coding (cont’d) • E.g., multiple items:  complete waste of time =1  generally provides useful information =2  is fascinating =3  I couldn't live without =4  other (please specify) __________ (additional coding) S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

20. Coding – scales • ‘self coding’ • if you want to combine questions/variables into an overall index, make sure all coding runs into the same direction, e.g., all ‘agrees’ are 5 and all disagrees are 1 S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

21. Coding of scales (cont’d) • 5 point scales ‘Research methods are completely unnecessary’ 5=strongly agree ‘Research methods are important to develop key skills.’ 5=strongly DISagree S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.

22. Piloting • …and debugging • For MA, full pilot (e.g. trial run with smaller sample) usually unnecessary • BUT: always a good idea to test whether they work • Phrasing (and spelling) • All answer options included? • Etc… • Use friends, partners, children, supervisors… S.M.Rasinger. 2013. Quantitative Research in Linguistics. 2e. Bloomsbury.