Problems in Research

1. Problems in Research • Poor design decisions • Researcher bias • Researcher effects on participants • Things participants may do regardless psychlotron.org.uk

2. Poor Design • Problems that stem from a badly designed study • Poor operationalisation of variables • Inadequate control of variables • Threaten the internal validity of the study • Cause reliability problems psychlotron.org.uk

3. Poor Design • Poor operationalisation • Insufficient care when translating unobservable psychological processes into observable behavioural ones • Can end up measuring the wrong thing • Can generate false positives or false negatives psychlotron.org.uk

4. Poor Design • Reliability – the extent to which a study or measuring tool produces consistent results • Test-retest reliability • Inter-observer reliability • Reliability is maximised by (1) careful operationalisation; (2) adequate control of variables • NB: a reliable study can still be invalid psychlotron.org.uk

5. Poor Design • Inadequate controls • Extraneous variables should be held constant • Most are not a problem (random effect) • Those that have a systematic effect (e.g. affect one condition more than another) become confounding variables psychlotron.org.uk

6. Researcher Bias • Emphasising some things at the expense of others • Self-confirmatory bias – tendency for researchers to pay more attention to data that confirm their expectations • Reduce the need for subjective judgements by (1) careful operationalisation; (2) use of double blind psychlotron.org.uk

7. Researchers & Participants • Researcher effects - behaviour of researcher can affect that of participants: • Intimidate (anxiety) • Insult (motivation) • Cue PPs to behave in particular ways • Can be reduced through standardisation of scripts and procedures psychlotron.org.uk

8. Researchers & Participants • Interviewer effects – characteristics of the researcher can affect how PP relates to them; hence their responses • Gender • Social class • Age • Ethnicity… • Difficult to predict, impossible to eliminate psychlotron.org.uk

9. Participants… • …are people too; people try to make sense of their situation…how they do this is not always predictable. • Attempts to please (or displease) the researcher • Management of self-presentation psychlotron.org.uk

10. Participants • Respond to the fact that they are being studied • Evaluation anxiety • Feelings about the study & the researcher • Effects on motivation – please the researcher or confound the results; depends on how researcher is perceived psychlotron.org.uk

11. Participants • May respond to demand characteristics (cues); these can enter the research: • Through poor design e.g. allowing PPs to be aware of both conditions in an expt • Through researcher behaviour e.g. unwittingly indicating desired behaviour • Can be reduced through careful choice of design; standardisation; double blind psychlotron.org.uk

12. Participants • Generally try to manage how they are perceived by others (we all do); can lead to: • Socially desirable responses • Other distortions of truth in order to manage impressions (how depends on who and why) • Can be reduced through anonymity, confidentiality but rarely eliminated psychlotron.org.uk

13. General Precautions • Start with the aim, not the procedure • Pay attention to choice of design, controls, operationalisation etc. • Pilot all procedures and materials • Use double blind procedures where appropriate & possible psychlotron.org.uk