Introducing Quantitative Approaches to Research

Introducing Quantitative Approaches to Research Mike Griffiths m.griffiths@gold.ac.uk

What I will cover • A. General introduction • Basic concepts • Critiques and responses • B. Data gathering • Overview of some common approaches • Validity issues • C. Presenting results • Overview

A. General Introduction • Variables • Operational definitions • Relationships between variable • Critiques of the quantitative approach

Variables • Quantitative research concerns itself with variables. • Which of the following are variables? • Height • Gender • Response to the question: on a scale of 1 to 10, how satisfied are you with your social worker?

What is a variable? • A variable is anything that varies • Over time, or between people, etc • E.g. hours of television watched per week does both • Varies between people • For a given person, varies over time

Operational definitions (1) • Suppose a researcher wants to know what clients think of their social worker • She might ask them ‘On a scale of 1 to 10, how satisfied are you with your social worker?’ • Turning a broad concept (e.g. opinion of social worker) into a specific variable that can be measured is known as making an operational definition or operationalising the concept

Operational definitions (2) • Notice that she could have chosen different operational definitions, e.g. • She could have used a different question, e.g. ‘How good is your social worker?’ • Or she could have used a different response scale, e.g. asked them to rate from 1 to 5 • To do quantitative research we often have to work with an operational definition • Obviously the closer it is to what we really want to study, the better

Information about variables • Sometimes, it is useful to have information about just one variable • Especially in new areas of research, or mixed method research • E.g. • What proportion of people in Lewisham are homeless? • What proportion of people in Croydon are diagnosed with various kinds of ailments? • Amongst counsellors attached to GPS’ surgeries, what problems do their clients present with, and how often?

Relationships between variables • But often in quantitative research, we are interested in relationships between variables, e.g. • Are people with poor general health more likely to be homeless?

Independent and dependent variables • And even better, we like investigating whether one variable affects another • E.g. does poor general health contribute to homelessness? • In which case we talk about the • Independent variable (IV) • the variable which does the affecting • Dependent variable (DV) • the variable which is affected • And we have a hypothesis: • that the IV affects the DV

Summing up the quantitative world view • Everything can be expressed as variables • Whatever we want to study can be summed up in a manageable number of such variables • There are relationships between these variables, which we can study • … Or at least, that is close enough to be useful (“postpositivism?”)

Critiques of quantitative research (1) • Some of the important critiques • Its epistemology (the philosophy of how knowledge is acquired) • Quantitative research tends to be associated with positivism, with certain implied values, e.g. • Knowledge is only scientific if it can be observed • Research should be value-free and objective • An unachievable ideal, hence misleading? • Qualitative researchers often say that their research better acknowledges its subjective element

Critiques of quantitative research (2) • A practical issue: simplifying the world • If we are going to investigate things quantitatively, we have to presume it is useful to describe them in terms of a manageable number of variables • E.g. personality is often reduced to 5 variables (plus one for intelligence) • Often, we have to presume that variables can be reduced to a small number of categories • E.g. race

A brief answer to critiques • All research methods have their strengths and weaknesses • Be aware that quantitative methods often simplify the world • And they often do involve subjectivity • e.g. operationalising variables • But they are often convenient in practice, as long as we recognise these limitations • Every research method, whether quantitative or qualitative, has its strengths and weaknesses and should be chosen to suit the research being done.

Discussion • What would be the arguments for studying the weather: • Qualitatively? • Quantitatively?

Questions so far?

B. Data gathering • Sampling: populations and samples • Some ways of gathering data

Populations and samples • The population is all the people or things you want your study to generalise to • e.g. postgraduate students at English universities • single parent households in England • A sample is the the people or things you actually studied • e.g. the students in this class; • 20 single parent households in Deptford

Quantitative samples • In quantitative research we want the sample to represent the population • In such a way that we can use statistics about the sample to represent the population

Quantitative samples • E.g. we might take the mean age of single parents in our sample, and want to use that as an estimate of the mean age of single parents in general • Or at least of single parents in the area we studied • Often in practice we look at our sample and ask what is the population we can extrapolate it to • Or, researchers ignore the issue and leave it to the reader to decide how far the results will generalise

Exercise • How might you go about obtaining a representative sample of 100 residents of Salisbury?

Sampling in practice • In practice, samples might be • Volunteers • A convenience sample, e.g. your colleagues or clients • Say how you recruited your sample • In quantitative research, we like large samples • At least 20 (per group when there is more than one group)

Some ways of gathering data

Some ways of gathering data • Evaluating an intervention • Questionnaires • Analysing pre-existing data, e.g. client or personnel records

Evaluating an intervention (1) • E.g. Savage et al. (2008) • Health status of homeless patients at a nurse-managed clinic • After 2 months’ use of the clinic, patients improved on measures such as substance use and mental health

Examining an intervention (2): with a comparison group • For example: Do parenting classes reduce truanting? • You might • Give half the parents a truanting class • Give the other half no intervention • And ask: Afterwards, is there a (significant) difference in the amount of truanting in the two groups?

Questionnaires: pre-existing • For many things (e.g. depression), there are pre-existing questionnaires • Make sure you find a manual or journal article that explains how to use it, and discusses its validity

Questionnaires: creating your own • There is plenty of advice in books about how to devise a questionnaire • E.g. Robson, C. (2011). Real world research (3rd ed.), chapter 10. • Before you send it out: • Decide its purpose • Develop appropriate questions • Think about how you will analyse it • Include instructions: including whether the participant needs to answer all the questions

Developing appropriate questions • Good books (e.g. Robson) have a checklist of things to watch out for • E.g. understandable, not offensive (think the way that your respondents will think) • Pilot your questionnaire, preferably involving members of the target group • Include a cover sheet or covering letter

Using questionnaires • We might just want the numbers/ percentages of our sample who do, or think, certain things • We might look at relationships between variables (e.g. are the people in our sample who are parents more likely to be depressed than people who are not) • We might use a questionnaire in conjunction with an intervention (e.g. how depressed are people before/ after using our services)

Using pre-existing data • Check with your supervisor whether this is a substantial enough study! • Ethical and legal issues • The Data Protection Act, or other concerns, may limit the uses that records can be put to

Quality of research: Validity • Does your research show what you say it does? • Useful concept for • Improving the quality of your research • Looking for flaws in other people’s research • There are two main kinds of validity • Internal validity • Did the study show a genuine effect? • External validity • Do the results generalise to other places, people and times?

Internal validity • Did the study show a genuine effect? • Differences in one variable may be associated with another • Or (equivalently) there may be a difference between groups, or before/after an intervention • But not for the reasons we claim • Examples follow …

Internal validity in correlational studies (1) • If we simply measure variables, this is called a correlational study • E.g. to find out whether homelessness affects general health, we ask people about their housing status and their health • Suppose we find a relationship • You have probably heard it said: Correlation is not causation

Internal validity in correlational studies (2) • Maybe poor general health is a cause of homelessness (reverse causation) • Or some other factor (e.g. personality) is a cause of both (a confounding variable) • Of course in this case, all three may be true • In general, try to bring in other evidence and/or arguments to your write-up

Internal validity in experiments (1) • If we manipulate one of the variables (the independent variable, IV), this is called an experiment • E.g. (1) Savage et al., above • IV was attending the clinic • DV was health status • E.g. (2) My hypothetical example • IV was attending parenting classes • DV was truanting

Internal validity in experiments (2) • An experiment aims to rule out reverse causation • But there can still be confounding variables • If we just look at before/after (as in Savage et al.), how do we know people wouldn’t have just got better anyway, e.g. due to the passage of time?

Internal validity in experiments (3) • So in experiments, we like there to be a comparison group • As in the example with parenting classes • We like the participants to be allocated at random into the groups • If they are not (e.g. one school gets the classes and another does not), the groups may not be the same as each other (causing a confounding variable)

Internal validity in experiments (4) • Even if the groups are allocated at random, any difference in the way they are treated can lead to a confounding variable, e.g. • Simply going to the place where the classes are held provides social contact • Knowing they are getting the classes, leads to • Enthusiasm in the group that get the classes • Demotivation in the group that do not get the classes • Do your best to design out and acknowledge such threats

External validity: some threats • How far does the study generalise? • E.g. if we read research about the relationship between health and homelessness in New York, how far can we apply those results to inner London? • A particular kind of external validity is ecological validity: whether the results reflect the real world • E.g. if we show people in the laboratory a film about anger management and they score less on an anger questionnaire, does that really mean the film will help people who watch the film at home?

Some responses to these threats (1) • Do what you can to eliminate or reduce the threats, especially in experiments, e.g. • Allocate people to groups at random • You may not be able to withhold treatment altogether from one group, but can you put them on a waiting list? • Or can you give two different treatments (e.g. an old version of parenting classes versus a new version)? • Check that the DV (e.g. truanting rate) does not differ before you start, and/or measure the change instead of the final rate

Some responses to these threats (2) • Accept you live in the real world • At least in quantitative research the threats to validity tend to be reasonably obvious

C. Presenting results • Kinds of variable • Descriptive statistics • Inferential statistics

Kinds of variables • Two main kinds • Categorical: variables that are divided into categories, e.g. male/ female • Variables that use numbers, e.g. temperature, height, weight, scores on a questionnaire • I will lump these together as “continuous” variables • The type of variable obviously affects the information we give about them

Descriptive statistics: categorical variables • Counts, percentages, and/or pie charts, e.g.

Descriptive statistics: ‘continuous’ variables • E.g. Age (of the participants in a study) • There were 12 participants, with a mean age of 28.3 (Standard Deviation = 2.8).

Means and standard deviations • Mean: the most common way of giving a typical value • add up all the values and divide by the number of cases (i.e. here, add up the ages and divide by the number of people). • In our sample, the mean age is 28.3 • Standard deviation: the most common way of saying how tightly the values cluster round the mean • More complicated formula • In our sample, the standard deviation of age is 2.8 years

When there are two variables (or different groups/ situations) • Categorical variable (e.g. before/after) and a continuous variable (e.g. mental health) • E.g. mental health before and after treatment • Two categorical variables • E.g. whether people drop litter, with or without existing litter • Two continuous variables • E.g. is child wellbeing related to child poverty?

Different groups / situations • Means and standard deviations for each group, or each time • E.g. Savage et al. (2008) • Do people score differently on a mental health questionnaire after a nursing intervention? • They gave the means and standard deviations at each time

Different groups/ situations • Means and standard deviations

Introducing Quantitative Approaches to Research