Topics in Special Education Research

1. Topics in Special Education Research Session 4: Causal-comparative & Correlational Research & Survey Methods

2. Difference between Inter-observer agreement & Treatment integrity/fidelity • Inter-observer agreement (IOA)- involves 2 observers measuring the same behaviors at the same time • It is most often used to determine the reliability of observations of the DEPENDENT VARIABLE • Treatment Integrity/Fidelity (of treatment/intervention/ INDEPENDENT VARIABLE) • This is how the researcher measured how well the treatment/intervention was implemented • Commonly done using checklists and other observers recording the completion of these checklists

3. Experimental and Quasi-Experimental vs other designs • The main difference between experimental/quasi-experimental research designs AND other designs is… • They MANIPULATE the independent variable • Basically…..they introduce an intervention, while other methods (except for single-subject) do not systematically introduce an intervention • ….seeks to make CAUSAL CONCLUSIONS

4. Proposal Assignment & Group Work • A detailed explanation of the assignment is posted on the wiki • What should you be doing in your groups? • At this point you should have a topic and start coming up with your framework for your research project (based on literature). • Start to draft your conceptual framework, research questions & identify your dependent and independent variables • You should walk away from your group time with a list of tasks to complete.

5. Socially Important Issue: • 2. Conceptual Model/Hypothesis: • 3. Research Question(s): • 4. Dependent Variable: • 5. Dependent Variable Measure: • 6. Independent Variable: • 7. Independent Variable Measure: • 8. Research Design:

6. PSU Human Subjects Research Review Committee (HSRC) • http://www.rsp.pdx.edu/compliance_human.php • Portland State University (PSU) is responsible for the rights and welfare of human subjects involved in research sponsored or conducted by the university.   In order to meet this responsibility, the University established the Human Subjects Research Review Committee.  • Members are charged with reviewing all research conducted under the auspices of PSU that involves human subjects to ensure adequate protections are in place.

7. Review for Quiz

8. 1. Identify socially important issue 2. Review current literature 3. Define conceptual model 4. Define specific hypothesis(es) and research question(s) 5. Define dependent variable(s)/measure 6. Identify independent variable(s)/measures 7. Select appropriate research design 8. Obtain consents 9. Collect data 10. Analyze data 11. Communicate results Written presentation Oral presentation Steps in the Research/Scientific Process

9. Experimental Design • Seeks to make causal conclusions. • Direct manipulation of an independent variable (intervention) • Difference between experimental design and quasi-experimental design is the use of random selection of participants and conditions.

10. Validity • Refers to whether a study is able to scientifically answer the questions it is intended to answer. • Extent to which your test (or study) measures what it intends to measure.

11. Internal Validity • Changes observed in the dependent variable (outcome) are due to the effect of the independent variable (intervention)….. • & not to some other unintended variables (extraneous, alternative explanations) • 12 threats to internal validity (noted by Mertens, 2010) • E.g.., History, maturation, testing, instrumentation, mortality, etc.

12. External Validity (think generalizability) • External Validity= extent to which findings in one study can be applied to another situation. • AKA: ecological validity, generalizability • 10 threats posed as questions (noted by Mertens, 2010) • E.g., detail/description of procedures, experimenter effects, sensitization, etc.

13. Quiz

14. Correct Quiz

15. 1. Identify socially important issue 2. Review current literature 3. Define conceptual model 4. Define specific hypothesis(es) and research question(s) 5. Define dependent variable(s)/measure 6. Identify independent variable(s)/measures 7. Select appropriate research design 8. Obtain consents 9. Collect data 10. Analyze data 11. Communicate results Written presentation Oral presentation Steps in the Research/Scientific Process

16. Experimental Design • Seeks to make causal conclusions. • Direct manipulation of an independent variable (intervention) • Difference between experimental design and quasi-experimental design is the use of random selection of participants and conditions.

17. Validity • Refers to whether a study is able to scientifically answer the questions it is intended to answer. • Extent to which your test (or study) measures what it intends to measure.

18. Internal Validity • Changes observed in the dependent variable (outcome) are due to the effect of the independent variable (intervention)….. • & not to some other unintended variables (extraneous, alternative explanations) • 12 threats to internal validity (noted by Mertens, 2010) • E.g.., History, maturation, testing, instrumentation, mortality, etc.

19. External Validity (think generalizability) • External Validity= extent to which findings in one study can be applied to another situation. • AKA: ecological validity, generalizability • 10 threats posed as questions (noted by Mertens, 2010) • E.g., detail/description of procedures, experimenter effects, sensitization, etc.

20. Discussion

21. Lecture

22. Statistics, statistics Descriptive Statistics Inferential Statistics Who is in your data? What your sample says about the population sample population sample population Mean, Median, Mode, standard deviation, variance Tests of significance (t-, F-Tests)

23. Descriptive Statistics • Central Tendency • Mean- average • Median- midpoint in distribution of scores • Mode- most frequent score • Variability • Range- total extension of the data (e.g., 1-10) • Standard Deviation- sum of deviations from the mean squared. How well the mean summarizes the data. • Variance- standard deviation squared. Used in sophisticated analyses

24. Statistics, statistics Inferential Statistics What your sample says about the population sample population

25. Inferential Statistics • T tests- used when have two groups to compare. • Independent samples t- if groups are independent • Different people in each group • Dependent samples t-: if two sets of scores are available for the same people • Matched groups • ANOVA (analysis of variance)- when you have more than 2 groups to compare OR more than one independent variable (reports an F-statistic, which is basically a t-value squared) • ANCOVA (analysis of covariance)- ANOVA that allows for control of the influence of an IV (e.g., characteristics of people) that may vary between your groups before treatment is introduced. • Post-hoc method for matching groups on variables such as age, prior education, SES, or a measure of performance

26. Tests of Significance • Statistical analyses to determine whether a difference is statistically significant (probability for result to occur by chance). • Yes or No answer • Alpha level (p=) • An established probability level which serves as the criterion to determine whether to accept or reject the null hypothesis • Common levels in education • .01 • .05 • .10 Objectives 4.1 & 6.1

27. Data Addressing “WHAT” questions? Depth of Information Qualitative Data Representative, Generalizability Details, Depth, and Variability Quantitative Data • Survey • Large Scale Assessments

28. Numbers with Different Meanings Variable Type Example Nominal Male (0) • Gender Female (1) No (0) • Yes/No Yes (1)

29. Coding Data Check All that Apply Q. Which of the following applications have you used with your students? (Please check ALL that apply) MS Word iMovie MS Excel iDVD MS PowerPoint iTunes SPSS iWeb

30. Numbers with Different Meanings Variable Type Example Strongly Disagree Strongly Agree Disagree Agree (1) (2) (3) (4) Nominal Male (0) • Gender Female (1) No (0) • Yes/No Yes (1) Ordinal • Likert-scale

31. Coding Data Likert-scale Strongly Disagree Somewhat Disagree Somewhat Agree Strongly Agree Disagree Agree Q1. Overall, I have a good Parent-teacher Relationship.

32. Coding Data Likert-scale Strongly Disagree Somewhat Disagree Somewhat Agree Strongly Agree Disagree Agree 1 2 3 4 5 6 Q1. Overall, I have a good Parent-teacher Relationship. NOTE: Distinguishable

33. Coding Data Multiple Categories African American Ethnicity Asian Caucasian Hispanic Other ____________ High School Education Completed Some College BA/BS Master’s Doctoral Decline to state

34. Coding Data NOTE: Mutually Exclusive, Exhaustive, and Distinguishable Multiple Categories African American 1 Ethnicity Asian 2 Caucasian 3 Hispanic 4 Other ____________ 5 High School 1 Education Completed Some College 2 BA/BS 3 Master’s 4 Doctoral 5 Decline to state 888

35. Numbers with Different Meanings Variable Type Example Strongly Disagree Strongly Agree Disagree Agree (1) (2) (3) (4) Nominal Male (0) • Gender Female (1) No (0) • Yes/No Yes (1) Ordinal • Likert-scale Scale • Age, Annual Income, Test-score (Interval/Ratio)

36. Numbers with Different Meaning Variable Type Example Strongly Disagree Strongly Agree Disagree Agree (1) (2) (3) (4) Nominal Male (0) • Gender Female (1) No (0) • Yes/No Yes (1) Categorical Ordinal • Likert-scale Numerical/ Continuous Scale • Age, Annual Income, Test-score (Interval/Ratio)

37. Numbers with Different Meaning Variable Type Example Strongly Disagree Strongly Agree Disagree Agree (1) (2) (3) (4) Nominal Male (0) • Gender Female (1) No (0) • Yes/No Yes (1) Categorical Who CARES?? Ordinal • Likert-scale Numerical/ Continuous Scale • Age, Annual Income, Test-score (Interval/Ratio)

38. Numbers with Different Meaning Variable Type Example Strongly Disagree Strongly Agree Disagree Agree (1) (2) (3) (4) Nominal Male (0) • Gender Female (1) No (0) • Yes/No Yes (1) Categorical Why CARE?? Ordinal • Likert-scale Numerical/ Continuous Scale • Age, Annual Income, Test-score (Interval/Ratio)

39. Variable Types and Analysis Is there an association? Dependent Variable Independent Variable (a.k.a., Outcome) (a.k.a., Predictor)

40. Variable Types and Analysis Is there an association? Dependent Variable Independent Variable (a.k.a., Outcome) (a.k.a., Predictor) Where differences culminate

41. Variable Types and Analysis Is there an association? Dependent Variable Independent Variable (a.k.a., Outcome) (a.k.a., Predictor, Intervention) Where differences culminate Contributing Factors

42. Variable Types and Analysis Is there an association? Dependent Variable Independent Variable Categorical Categorical Numerical/ Continuous Numerical/ Continuous

43. Variable Types and Analysis Dependent Variable Independent Variable Chi-square test Or χ²-test Contingency Tables (a.k.a. Cross-tabs) Categorical Categorical Numerical/ Continuous Numerical/ Continuous

44. Variable Types and Analysis Dependent Variable Independent Variable Contingency Tables (a.k.a. Cross-tabs) Categorical Categorical Annual Salary Gender

45. Variable Types and Analysis Dependent Variable Independent Variable Contingency Tables (a.k.a. Cross-tabs) Categorical Categorical Gender Male Female 25K or below 12% 4% 26K - 35K 18% 6% 36K - 45K 24% 11% Annual Salary 46K - 55K 36% 39% 56K - 65K 8% 28% 66K and up 2% 12%

46. Variable Types and Analysis Dependent Variable Independent Variable Categorical Categorical Analysis of Variance (a.k.a. ANOVA) Numerical/ Continuous Numerical/ Continuous

47. Variable Types and Analysis Dependent Variable Independent Variable t-test or F-test Analysis of Variance (a.k.a. ANOVA) Numerical/ Continuous Categorical Males Females SAT9 Math Score

48. Variable Types and Analysis Dependent Variable Independent Variable Categorical Categorical Regression Numerical/ Continuous Numerical/ Continuous

49. Variable Types and Analysis Dependent Variable Independent Variable Regression Numerical/ Continuous Numerical/ Continuous SAT9 Math Score Household Income

50. Variable Types and Analysis Dependent Variable Independent Variable Categorical Categorical Logistic Regression Numerical/ Continuous Numerical/ Continuous