PSYC512: Research Methods Lecture 18

1. PSYC512: Research MethodsLecture 18 Brian P. Dyre University of Idaho PSYC512: Research Methods

2. Lecture 18 Outline • Research Proposal Information • Writing Research Proposal • The Art of Peer Review • Addressing Reviewer’s Comments • Inferential Statistics • Testing for differences • Determining if a sample represents a population • Determining if two or more samples differ • Testing for relationships • Form of relationship • Strength of relationship PSYC512: Research Methods

3. Writing a Research Proposal • Purpose of Research Proposal • Present a literature review defining concepts and intervening variables pertinent to research question • Present a specific research question with explicit hypotheses to be tested • Develop a plan for addressing that research question empirically, including descriptions of • Target population and subject sampling • The research design, including explicit definition of independent and dependent variables and how the stimuli and procedures implement these variables • Aspects of stimuli and procedures that control for extraneous and confounding variables • Types of analyses to be used, predicted results, and how these predictions relate to the hypotheses PSYC512: Research Methods

4. Research Proposals: Global Concerns • Scientific Writing Style • Precision more important than entertainment, but… • Research is part science and part advertising – not only do you need to develop good ideas but you must be able to sell your ideas • Proposal is often the basis of first impression for the quality of the research project • Proposal must be clear on ALL levels of analysis • Words • Sentences • Paragraphs OUTLINE! • Sections PSYC512: Research Methods

5. Research Proposals: Global Concerns • APA guidelines • Orderly expression of ideas - organization • Smoothness of expression - transitions • Economy of expression - concise language • Precision and clarity – use scientific vocabulary “jargon” correctly and insure that all terms are defined the first time they are used PSYC512: Research Methods

6. Organization of a Research Proposal: APA STYLE • Title Page – title should specifically describe what the paper is about so that it is useful information for other researchers’ literature searches • Abstract • essentially a mini-paper for lit. searches • be extremely CONCISE! (< 150 words) • introduce specific topic • discuss variables, etc. • present major results (no statistics!) • discuss important conclusions PSYC512: Research Methods

7. Organization of a Research Proposal : APA STYLE • Introduction • Purpose • Demonstrate knowledge of relevant research • Define intervening variables and their relation to manipulations and measurements used in previous research • Present and justify research question and hypotheses • Present and justify the general method to be used • Organization • Start broad then narrow to your general purpose • Discuss only relevant research in a logical flow • Near the end provide an explicit statement of hypotheses and an overview of the general research design PSYC512: Research Methods

8. Organization of a Research Proposal: APA STYLE • Method • explicitly state how variables are manipulated • define in separate sub-sections • Participants (subjects) • Design • Stimuli/Apparatus/Materials • Procedures • Results • Describe the Scales of IVs and DVs • Describe Transformations of DVs • Summarized planned statistical analyses • Summarize predictions PSYC512: Research Methods

9. Important Elements of Style (Brian’s pet peeves) • Define scientific terms (“jargon”) and abbreviations at first use, then use terms consistently • Do not use informal or colloquial language • e.g., you don’t “run” subjects, you test subjects • Ambiguous pronouns: if you use the word “it” make sure the surrounding context makes the meaning of the word “it” obvious, otherwise avoid using it (as in “it”) • Sexist pronouns (he vs. she) – word sentences to avoid having to use these if at all possible, or use “he or she” PSYC512: Research Methods

10. Important Elements of Style (Brian’s pet peeves) • Verb tense • Introduction Section: • Discussion of specific previous research  past tense, • Discussion of ideas or concepts that are general for all time and not linked to specific moment in the past present tense • Method and results • Proposal: future tense • research report: past tense PSYC512: Research Methods

11. Important Elements of Style (Brian’s pet peeves) • Plural/singular mismatches • Passive voice – word sentences in active voice • Superfluous, imprecise language (e.g., avoid vague adverbs—search for all words ending in “ly” and consider eliminating them) • Style issues like “that” vs. “which” • For more information on writing style I recommend: • Strunk, W., jr., and White, E.B. (2000) The Elements of Style, 4th Ed. Boston: Allyn and Bacon. PSYC512: Research Methods

12. The Art of Peer Review • Example review available online in lecture schedule • Goal: Assist author in improving the clarity and impact of the paper (or proposal) by offering specific constructive criticism (no name calling!) and complements (where appropriate) • Structure • Summary • Major (or general) criticisms (typically <=3) • Minor (specific) criticisms (any number, specifically listed by page and line number) PSYC512: Research Methods

13. Addressing Reviewer’s Comments • Never blame a reviewer for a negative review • Consider all criticism as constructive—the reviewer is trying to help you! • Assume misunderstandings are your fault, not the reviewers. All misunderstandings occur because you did not write clearly enough • The cover letter (required with submission of revised proposal) • Specifically responds to the reviewers comments by • describing specific changes made in the paper to address criticisms • Presenting a rationale for why a reviewer’s criticism was not addressed • Must be worded very diplomatically! PSYC512: Research Methods

14. Presenting Research • Time-allotted: 20 minutes  strictly enforced! • Use same general format as the written report • introduction (roughly 10 minutes) • method (roughly 8 minutes) • Summary (2 minutes or so) • Given time allotted you cannot go into the same level of detail as your written report • Materials • Talk from an outline of points you wish to make • Visual Aids  power point • Elements of Style • PRACTICE YOUR PRESENTATION! • Anticipate questions and how you will answer them PSYC512: Research Methods

15. Using Inferential Statistics • Which Statistic? • The statistical decision tree Howell Figure 1.1 • Testing for relationships vs. differences (a false distinction) • Relationships: assessing the strength of relationship between measured (dependent) variables • Differences: comparing different groups or treatments on some measurement • But what causes those differences? The relationship between the independent variable defining the groups or treatment and the dependent variable • Hence, testing for differences is really testing the relationship between the IV and DV PSYC512: Research Methods

16. Analyzing Frequencies (Howell, Chapter 5) • Bernoulli Trials: series of independent trials that result in one of two mutually exclusive outcomes • E.g. coin flips, gender of babies born, increase of decrease in a measure after application of a treatment • The Binomial Distribution PSYC512: Research Methods

17. Analyzing Frequencies (Howell, Chapter 5) • Using the binomial distribution • Mean number of successes = Np • Variance in number of successes = Npq • Testing Hypotheses using the binomial distribution: The Sign Test • Ho is typically p= q = .50 (50-50 chance of success of failure), but that doesn’t have to be the case • H1 is typically p ≠q • Plug in values for N, X, p, and q and p(X) directly provides the probability that the pattern of data could result given the null hypothesis is true • Sum the probabilities p(X) for all number >= X to get the total probability of finding p(>=X) • Important: The sign test takes into account direction of differences but not magnitude PSYC512: Research Methods

18. Analyzing Frequencies (Howell, Chapter 5) • What about multiple (more than 2) possible outcomes? • Multinomial distribution PSYC512: Research Methods

19. Analyzing Frequencies (Howell, Chapter 5) • Using the multinomial distribution • Mean Xk = NpXk • Variance in Xk = NpXk (1-pXk) • Testing Hypotheses using the multinomial distribution: • Ho is typically pX1= pX2 … = pXk = 1/k (each outcome has the same chance), but that doesn’t have to be the case • H1 is typically pX1≠ pX2 …≠ pXk • Plug in values for N, X, and pX, and p(X1, X2…Xk) directly provides the probability that this particular pattern of data could result given the null hypothesis is true • Must sum the probabilities for all patterns that deviate equal to or more to get the total probability – time consuming! PSYC512: Research Methods

20. Analyzing Frequencies (Howell, Chapter 6) • Easier Alternative to Multinomial distribution: Chi-square (c2) test • Compare computed value of c2 to value of c2 distribution with df=k-1 • Expected frequencies for the null hypothesis typically = N/k, where N is the total number of observations k is the number of categories in the variable O is the observed frequency for each category E is the expected frequency for each category i is the category index PSYC512: Research Methods

21. Analyzing Frequencies (Howell, Chapter 6) • Using c2 with multiple dimensions: contingency tables—frequencies of one dimension are contingent on the other dimension • Eij = RiCj/N • N is the total number of observations • Compare computed value of c2 to value of c2 distribution with df=(R-1)(C-1) R is the number of categories in the dimension defined by the rows of the table C is the number of categories in the dimension defined by the columns of the table O is the observed frequency for each category E is the expected frequency for each category i and j are category indices PSYC512: Research Methods

22. Analyzing Frequencies (Howell, Chapter 6) • Assumptions of the c2test • Each observation is independent • Inclusion of non-occurrences PSYC512: Research Methods