UNIT 1: RESEARCH METHODOLOGY

1. UNIT 1:RESEARCH METHODOLOGY

2. ILLUSIONS • “We live in a world of illusion…” Steve Miller band • In their book The Invisible Gorilla,Chabris & Simons details some of these including the illusions of attention, memory, confidence, knowledge, cause, and potential. • Rate yourself: On a scale of 1-5 How attentive are you? How good of a driver are you? How good are you at multi-tasking?

3. Forming a scientific attitude • “To believe with certainty we must begin by doubting.” Polish proverb • Scientific attitude: • Be skeptical but not cynical • Be open but not gullible • Be a critical thinker by developing the following skills:

4. CRITICAL THINKING SKILLSAs taken from Ch. 1 of “Challenging Your Preconceptions” • Be flexible—tolerate ambiguity & uncertainty • Identify bias—accept its existence, be on the lookout, try to prevent it • Be skeptical…but not cynical. • Distinguish between FACT & OPINION • Don’t oversimplify • Use logical inference processes • Look at the evidence and then draw conclusions.

5. Forming a scientific attitude • To put the scientific attitude to practice requires skepticism and humility. At times we have to reject our own ideas. • Example: The Mozart Effect: Research was published several years ago that showed a relationship between listening to Mozart and improved problem skills. This became the phenomenon known as the Mozart Effect. This was a founding principal behind “Baby Einstein” products. • http://en.wikipedia.org/wiki/Mozart_effect • http://www.nytimes.com/2009/10/24/education/24baby.html?_r=0

6. BIAS—Something to recognize and avoid Bias: a particular tendency or inclination, especially one the prevents unprejudiced consideration of a question; In can present itself in the experimenter or the research participant or both. Experimenter bias: movie clip Ghostbusters Can lead to self-fulfilling prophecy, false consensus effect, overconfidence…

7. TYPES OF EXPERIMENTER BIAS • SELF-FULFILLING PROPHECY: Bias that occurs when the “experimenter” has expectations about a behavior and then acts in certain ways to carry out those expectations. Consider the Rosenthal study… • FALSE CONSENSUS EFFECT: The tendency of the experimenter to overestimate other’s agreement with us because we are most often surrounded by people who share similar opinions and perceptions as ourselves. (Example: The idea that “alternative” music is most preferred by teens simply because you and your friends prefer it…)

8. TYPES OF EXPERIMENTER BIAS • To prevent the self-fulfilling prophecy, it is important to use “blind” tests. This is where the researchers are unaware of characteristics or labels that apply to the participants in order to minimized biased treatment. • To prevent the false consensus effect, use random sampling. Don’t just sample from people you know or from people who are like you.

9. Random Sampling: a sample that fairly represents the population because each member had equal change of inclusion • You cannot survey everyone. • Select a random sample. • Not just family, friends, people of same age, gender or ethnicity. Otherwise you possibly end up with a false consensus effect. • Random sampling is where every person has an equal chance of being selected for participation. • Basic statistical rule of thumb is 10% of your population is an appropriate size for a sample.

10. TYPES OF EXPERIMENTER BIAS • OVERCONFIDENCE: In has been scientifically proven that it is human nature to think we know more than we do. Overconfidence will often limit the preliminary research we think we need to do…leading to bias such as confirmation bias. • HINDSIGHT BIAS: Bias that occurs when we find out something happened and then perceive it as inevitable. “Anything seems common place once explain,” Dr. Watson.

11. TYPES OF EXPERIMENTER BIAS • To prevent overconfidence you must first acknowledge your tendency to exhibit it. Students are notoriously bad at predicting how long it will take them to complete homework, projects, papers, etc. WE all do, so in the future, double you estimated time! • To avoid hindsight bias state a hypothesis. If you make your predictions known prior to the research then you will never have to worry about hindsight bias. Who do you predict will win the game? You have to say it upfront before the event.

12. BIAS ON THE PART OF THE PARTICIPANT • PLACEBO EFFECT: The desire on the part of the participant to want to see or feel something that may or may not be really there but is due to the belief in some variable such as medication or therapy.

13. BIAS ON THE PART OF THE PARTICIPANT • HAWTHORNE EFFECT: The effect seen when test subjects observed behavior changes simply because they are involved in an experiment an are receiving special attention. • http://www.economist.com/node/13788427

14. Practice Questions 1 • “Monday morning quarterbacks” rarely act surprised by the outcome of the weekend football games. Their tendency to believe they knew how the game would turn out is best explained by A. Overconfidence. D. Illusory correlation. B. Hindsight bias. E. Random sampling. C. Intuition. • Aaron, who is participating in an experiment on the effects of alcohol on perception, is truthfully told by the experimenter that he has been assigned to the “high-dose condition.” What is wrong with this experiment? A. There is no control group. B. There is no experimental group. C. Aaron’s expectations concerning the effects of “high doses” of alcohol on perception may influence his performance. D. Knowing that Aaron is a part of the “high doses” group may influence the experimenter’s interpretation of his performance. E. Both C & D are problems with this study.

15. Follow the steps to good scientific methodology… • Make observations that lead to the proposal of a problem • Propose a researched based hypothesis • Design and conduct an experiment • Gather and analyze data • Conclude and replicate, replicate, replicate!

16. Operational definitions • A statement of procedures (operations) used to define research variables. For example, intelligence may be operationally defined in a project by the exact tool used to measure it. • If you wanted to measure how “emotional” a population is, how exactly would you measure “emotional?” • When Levine & Norenzayan (1999) measured the “pace of life” in 31 countries they defined “pace” by walking speed, speed at which postal clerks completed a task, and accuracy of public clocks.

17. Operational definitions(additional examples) • If you want to improve your grades by being more “organized,” how would you measure your organization in order to know if you are more or less? • As you visit college campuses you may have a “friendliness” scale you use to make judgments about that campus. What is on your scale? How do you determine if A&M is “friendlier” than “UT?” • Who told you this class was “easy?” Did you ask them to operationally define that…

18. Ethics, Code of Conduct • Psychologists strive to benefit those with whom they work and take care to do no harm. • We uphold professional standards of conduct… • We seek to promote accuracy, honesty, and truthfulness in science.

19. Ethics, Code of Conduct • In situations in which deception may be ethically justifiable, we strive to maximize benefits and minimize harm. • We exercise reasonable judgment to prevent unjust practices. • We respect the dignity and worth of all people, and the rights of individuals to privacy, confidentiality, and self-determination.

20. TYPES OF STUDIES • CASE STUDIES: study of an individual in great depth in hopes to reveal universal truths. They suggestion direction for further study, but atypical cases can be misleading. Ex: early brain research • LONGITUDINAL STUDIES: Studies that occur over a long period of time. Typical of developmental studies and some drug studies. Many problems: time consuming, expensive, subject drop out, etc. • CROSS-SECTIONAL STUDIES: Alternative to the longitudinal study where subjects from different age groups are tested at the same time. Usually not as in-depth (ex: National testing data)

21. TYPES OF STUDIES • NATURALISTIC OBSERVATIONS: watching and recording behavior of organisms in their natural environment; this only describes behavior, it does not explain it. • SURVEYS: Surveys look at many individual cases but not in depth like a case study. Design of the tool is important….

22. WHEN DEVELOPING SURVEYS… • Wording of surveys: Slight word changes can have a significant impact, • “not allowing” vs “forbidding” or “censoring” • “in favor of” vs “not opposed to” As a result you want to be very cautious in your wording when developing surveys. Also be critical of survey results. Consider your thoughts in relation to call in voting shows result such at American Idol or Dancing W/Stars… Random sampling important.

23. Correlation • Often when conducting research, we seek to find a relationship among measured variables. • How similar are personality test scores for identical twins? Fraternal twins? • Do SAT scores predict college GPA’s? • Are more babies born on nights of a full moon? • Is stress and disease related?

24. Correlation • Most studies designed to answered the above questions are seeking a relationship between the two variables. • Often the results are re-constructed from “correlation” to “causation.” • YOU CANNOT ASSUME CAUSATION FROM A CORRELATION! • Just because a relationship has been found between watching violence and being violent, one cannot assume watching violence makes someone perform violent acts.

25. Correlation • Low self-esteem correlates, and therefore predicts, depression. • Does that mean low self-esteem causes depression? • No, depression might cause low self-esteem. • OR some other related variable may be contributing to both low self-esteem AND depression. • Though a causal relationship may exist, a correlation doesn’t guarantee this.

26. Correlations • Correlations can be positive: • As study time increases, test scores increase Correlations can be negative: As time spent sleeping decreases, hallucinations increase.

27. CorrelationsIdentify these as either positive or negative: • The more young children watch tv, the less they read. (Kaiser, 2003) • The more sexual content teens see on tv, the more likely they are to have sex. (Collins, et. al., 2004) • The longer children are breast-fed, the greater their later academic achievement. (Horwood & Fergusson, 1998) • The more often adolescents eat breakfast, the lower their body mass (Timlin, et al, 2008)

28. Correlation • Correlation coefficient: statistical measure of relationship that reveals how closely 2 things vary together and thus how well either can predict the other. Correlation coefficients range from -1.0 to +1.0. The closer to 1, the stronger the correlation. • Illusory correlation: perceived correlation that does not exist. Ex: Adoption followed by conception stories… When premonitions are reinforced, we are more likely to believe them.

29. Practice Questions 2 • The correlation between two observed variables is -0.84. From this, it can be concluded that A. As one variable increases, the other is likely to increase. B. As one variable increases, the other is likely to decrease. C. The two variables are unrelated. D. One variable causes the other variable to occur. • One variable causes the other variable to not occur. • A researcher was interested in determining whether her students’ test performance could be predicted from their proximity to the front of the classroom. So she matched her students’ scores on a math test with their seating position. This study is an example of A. Experimentation B. Correlational research C. A survey D. Naturalistic observation E. A case study

30. Experimentation is use to determine cause & effect relationships. • Make observations that lead to the proposal of a problem • Propose a researched based hypothesis • Design and conduct an experiment • Gather and analyze data • Conclude and replicate, replicate, replicate!

31. Experimentation is use to determine cause & effect relationships. • To conduct an experiment, you must vary the independent variable (IV) and keep all other variables constant. An independent variable is the variable in an experiment that is being implemented and its effect is being measured. • You must have a control group (does not received the IV)and an experimental group (does receive the IV). • You will measure the outcome which is the dependent variable. • Experiment simulation

32. Conducting experiments • Researcher should randomly assign participants to either the control group or the experimental group. • With some experiments, particularly drug tests, it is ideal to have a double-blind test. • A double blind test is when neither the participant nor the researcher knows whether someone is in the control group of the experimental group.

33. Experimentation • The experimental group gets the drug and the control group receives a placebo, an inactive drug. • The purpose of this is to reduce the self-fulfilling prophecy on the part of both experimenter and participant. Often a placebo effect is documented in studies where participants simple believe the drug will have an effect and therefore they report experiencing those effects.

34. CONFOUNDING VARIABLES • Sometimes not all variables can be held constant, and these out of control variables may impact your research. • These are known as CONFOUNDING VARIABLES. • Examples could include in our research with music/no music, people’s height in relation to the table on the smart board.

35. QUASI-EXPERIMENT • This occurs when you cannot randomly assign your participants to the experimental and the control groups. • Sometimes random assignment isn’t possible; therefore, one must recognize their experiment as “quasi.” • In education, this is common when student may be exposed to different teaching methods by class.

36. Practice Questions 3 • In an experiment to determine the effects of exercise on motivation, exercise is the A. Control condition B. Intervening variable C. Independent variable D. Dependent variable • Illusory correlation • In drug studies, your participants are randomly assigned to one of two groups, the control group and the experimental group. The control group would experience A. receiving the real drug being tested. B. receiving a placebo. C. receiving no intervention. D. receiving an active drug that is already approved for use. E. receiving an alternative therapy that is not a drug.

37. Data Analysis & Statistics • Measures of central tendency: mean, median, mode. One number represents the whole set. • Mean: average • Median: exact middle number • Mode: most frequently occurring number Example: 67, 94, 72, 88, 88, 95, 100, 88, 72 Mean = 84.9, Median = 88, Mode = 88

38. Data analysis • Central tendency numbers are effected by extreme scores and lop-sided or skewed distributions; therefore, measures of variation must also be considered: range and standard deviation • A large standard deviation means that scores are very spread out from the mean. • A small standard deviation means that scores are relatively close to the mean.

39. Data analysis • Positive skew is associated with a tail to the right. • A negative skew is associated with a tail to the left.

40. Standard Deviation of normally distributed scores 2% 13.5% 34% 34% 13.5% 2%

41. Data Analysis • Descriptive statistics: describe an entire population, ex: teacher’s gradebook • Inferential statistics: data collected from sampling, and if a representative sample is collected, data can infer information about the entire population • Be cautious of big, rounded, undocumented statistics, often stats are misquoted b/c gen. population doesn’t understand them!

42. Statistical significance • Statistical significance: the difference we observe is probably not due to chance variations between samples. • In research, depending on a statistical test, one may claim their findings are “statistically significant.” • Be cautious of significance, it needs to be validated by replication of studies. Also, statistical significance doesn’t mean the findings are relevant or important it is just another statistic.

43. Statistical caution • "The secret language of statistics, so appealing in a fact-minded culture, is employed to sensationalize, inflate, confuse, and oversimplify," warns Darrell Huff, author of How to Lie with Statistics.

44. Practice Questions 4 • What is the median of the following scores: 10, 7, 5, 11, 8, 6, 9? A. 6 B. 7 C. 8 D. 9 • 10 • What statistic would best describe a set of scores that are very spread out from the mean? A. A large standard deviation B. A small standard deviation C. A large range D. A small range • Any measure of central tendency • When a difference between two groups is “statistically significant,” this means A. The difference is statistically real but of little practical importance. B. The difference is probably the result of sampling variation. C. The difference is not likely due to chance. D. The difference is of importance to the world of research. E. The difference found is probably a fluke.

45. Practice Questions Answers • Set 1: B, E • Set 2: B, B • Set 3: C, B • Set 4: C, A, C • Sample quiz answers: 1. d, 2. e, 3. a, 4. d, 5. c, 6. b, 7. a, 8. a, 9. b, 10. a, 11. c, 12. d, 13. b, 14. d, 15. d