Experimental Psychology PSY 433

1. Experimental PsychologyPSY 433 APA Format Reports: Results, Discussion, References

2. Exam on Thursday • Based on the textbook: • Covers all chapters and pages noted on syllabus (Ch 1-5 plus Appendix A & pgs noted) • No questions from labs • No questions on APA format details • Powerpoints provide an outline of what I considered important enough to discuss in class – not a substitute for reading the book.

3. Projects Due in This Course • Proposal – similar to the kinds of proposals submitted to granting agencies. • Contains parts of a full APA report plus extra info needed by the agency. • Written in future tense. • Final Report – similar to the manuscripts submitted to journals for publication. • Written in APA format. • Written in past tense because it describes what happened.

4. Contents of Proposal • Required for the assignment due 5/10: • Title Page • Introduction • Methods (written in future tense) • References • Proposals to granting agencies also include: • Bios of the researchers • Budgets and performance timetables • Lots of forms

5. Goals of a Research Proposal • Convince the reader that the question is important and needs to be explored. • Convince the reader that you are qualified to do the research (not part of the class project). • Describe what research has occurred previously and what the competing theories are. • Describe your plan for research in detail. • Demonstrate that you have the resources to carry out the plan.

6. Introduction • Start by stating your research question. • Be specific, no general introductory remarks. • Next, review the literature by summarizing previous research. • At least 5 peer-reviewed sources required. • Do not describe each article sequentially – give an overview emphasizing what they found out. • Conclude with an overview of your study and a prediction about the outcome of your own study stated in terms of theory.

7. Methods • Include subsections (with subheads), as described in the APA Manual: • Subjects (participants) • Materials or apparatus (describe computer, your stimuli, any questionnaires). • Procedures • Describe design • Tell what happened to the subject in chronological order • Leave out things that happen in every experiment

8. Goals of the Final Report • Communicate to the scientific community. • Clearly describe your project in sufficient detail to permit replication. • Convince readers that your findings support your conclusions. • How strong is the evidence? • Does it justify your statements about theory? • Summarize your contribution to the ongoing debate on an important question. • Pay special attention to your abstract!

9. Contents of Final Report • Must contain all sections listed in the APA Publication Manual, including: • Title page • Abstract • Introduction • Methods • Results • Discussion • References • Tables and Figures

10. You Are Telling a Story • Introduction -- state your research question, review the literature, make your predictions (hypotheses). • Methods – describe how you explored the question in sufficient detail to permit replication. • Results – describe your findings and test your hypotheses using statistics. • Discussion – analyze your results and put them back into the context of your question.

11. Abstract • This may be the only part of your paper that most people read, so make it count! • Write this last. • Tell the story of your study, with one sentence per report section. • Do not exceed 120 words.

12. Use of Tenses • For the final report, revise the sections that were written for your proposal because they will be graded again. • Your proposal was written in the future tense (e.g., “subjects will…”), but for the final report… • Put the Methods section in the past tense. • Report your results in the past tense.

13. Reporting Results • Only include the results that are relevant to your research question, not all data collected. • Go from the general to the specific. • Provide tables for: • Multiple analyses. • Complex experiments (factorial designs). • ANOVA • Organize your results section around your hypotheses, testing one at a time.

14. Describing Data • Don’t forget to include descriptive statistics (means, SDs). • “The mean number of words recalled was calculated for each group. The means and the standard deviations for each group are shown in Figure 1.” • “Recall was higher for the drug group (M = 15, SD = 5.43) than for the placebo group (M = 10, SD = 4.98).”

15. Reporting Inferential Statistics • “The data were analyzed using an independent t-test. The t-test showed no significant difference between the mean of the placebo group and the mean of the drug group, t(34) = 1.35, p = .782. • “Using one-way ANOVA, gender differences were found to be significant, with females scoring higher on the average than males, F(1, 23) = 23.89, p =.025.” • Show more complex analyses in a table.

16. Report Exact p-Values • The old approach simply tested results against a standard of p<.05 by looking up the critical value in a table. • Significance was an all-or-nothing judgment. • Only the critical value (cutoff) was known, not the exact p-value for your statistic. • Today, SPSS gives exact p-values for every result. Report those exact values (p=.031). • NEVER report p > .05 for a non-significant result. It implies use of p > .05 as a standard.

17. Ethics of Reporting Statistics • Don’t change your hypotheses (prediction) to fit what you actually discovered. Instead say you were surprised by your results. • Decide how many subjects to test in advance. • Don’t stop collecting data because you already have significant results. • Don’t add more subjects because your results are almost significant and would become so with a few more subjects. • State your reason for ending data collection.

18. Avoid “p-Hacking” • p-hacking is the practice of trying different approaches to data analysis until you find one that gives significant results. It is unethical. • Collect at least 20 observations per condition. • Report all experimental conditions, even failed manipulations (studies that didn’t work). • List all variables collected in a study, even if they are not analyzed in your paper. • If there is any doubt, report results with and without excluded subjects, covariates.

19. Changes in Reporting • The internet is making possible different approaches to report writing. • Because journals are no longer limited in space, authors can supply complete data sets, stimuli (materials) and alternative analyses. • This represents a movement toward greater transparency. • Exact, not conceptual replications are needed results are marginal.

20. References • Format varies depending on the type of material being referenced (e.g., book, article, web site). • Only list the sources actually mentioned in the text of your report. • Everything listed in the references must be cited in text • Everything cited in text must be listed in the references. • When you mention a source referenced in another paper say: “as cited by…” and cite the source you actually read, not the original quoted by someone else.

21. Tables and Figures • Tables go first – always use APA format. • Tables contain numbers or words. • Figures are pictures and typically present graphs of data, sample stimuli, equipment setup, diagrams of experiment flow, flowcharts of cognitive processes or diagrams of theoretical models. • Tables have titles that go at the top. Figures have captions that go at the bottom. • Include at least 1 of each in your final report.

22. Discussion • First, state what you discovered during your experiment. • Do not repeat results but interpret them and state whether your hypotheses were confirmed. • Tell whether your findings are consistent with what others have found. • Describe any threats to validity and problems with your experiment (confounds, bias, limitations of generalizability, problems). • Conclusion – what are the consequences?