Understanding Research Validity: Internal, External, Construct

1. Internal Validity Scenarios External Validity Construct Validity Misc 1 pt 1 pt 1 pt 1 pt 1 pt 2 pt 2 pt 2 pt 2 pt 2 pt 3 pt 3 pt 3 pt 3 pt 3 pt 4 pt 4 pt 4 pt 4 pt 4 pt 5 pt 5 pt 5 pt 5 pt 5 pt

2. Internal Validity 1 point Define Internal Validity

3. Internal Validity 1 point What is the observed change in the dependent variable due to a manipulation in the independent variable?

4. Internal Validity 2 points Naturally occurring changes over time could be confused with a treatment effect

5. Internal Validity 2 points What is Maturation?

6. Internal Validity 3 points In a randomized control trial, 70% of my participants in my treatment condition dropped out, and only 10% of my participants in my control condition dropped out

7. Internal Validity 3 points What is attrition?

8. Internal Validity 4 points In quasi-experimental designs when there is no direct manipulation of a variable, such as in correlational analyses, and it is not clear which variable may influence another you have this threat.

9. Internal Validity 4 points What is ambiguous temporal precedence?

10. Internal Validity 5 points Define the threat “instrumentation.”

11. Internal Validity 5 points What is the nature of a measure may change over time or conditions in a way that could be confused with a treatment effect?

12. Scenarios 1 point The participants in my study consisted of 18 – 22 year old white college males.

13. Scenarios 1 point What is threat to external validity?

14. Scenarios 2 points When I initially ran my study, I only had 20 participants in the experimental group and 20 participants in the control group. Because I had so few participants I wasn’t sure if the lack of a significant finding was due to the treatment effect or too few participants, which created this type of threat

15. Scenarios 2 points What is threat to statistical conclusion validity or low statistical power?

16. Scenarios 3 points In my study, I had to use the same research assistants to both administer the manipulation and take measurements so I had to tell them which participants were in the experimental group and which were in the control group

17. Scenarios 3 points What is threat to construct validity or experimenter expectancies?

18. Scenarios 4 points I wanted to get a pre and post measure of intelligence in my experiment to see if students’ IQs increased due to my intervention so I administered the same test before and after the intervention

19. Scenarios 4 points What is threat to internal validity or testing?

20. Scenarios 5 points In an experiment looking at a child with attention issues, I measured a child’s attention through the report of teacher observations only. Then I chose students to participate in my study only if they had extremely severe attention issues or had no reported attention issues at all. This method results in two major validity threats

21. Scenarios 5 points What is Mono-operation bias and regression?

22. External Validity 1 point Define External Validity

23. External Validity 1 point What is the extent to which the findings from a study (or effect) can be generalized to other populations, settings, treatment variables, and measurements?

24. External Validity 2 points A study collected its sample from one undergraduate institution and excluded any participants who were under 18 and over 24 years old. The sample was 70% female, 30% male, 80% White, 15% Asian, and 5% Black. All participants were juniors or seniors. This study’s sample could be improved in several ways

25. External Validity 2 points What is recruit from multiple universities; try to sample community members, use a different sampling method that would yield a more representative sample of the population you would like to measure; expand the age range; make the sample more diverse in terms of education level and race?

26. External Validity 3 points A study looking at the effect of treatment on depression had a large effect size in a controlled, highly regulated setting, however there are doubts as to how the effect will translate into a community clinic or less rigid environment

27. External Validity 3 points What is interaction of the causal relationship with setting or limitation of external validity of the setting?

28. External Validity 4 points In a study looking at the impact of an intervention on depression, there was a treatment effect on depression as measured through the Beck Depression Inventory, but it is unclear how the intervention impacted overall quality of life, activity level, interpersonal relationships, etc.

29. External Validity 4 points What is interaction of the causal relationship with outcomes or limitation of external validity of the outcome observation?

30. External Validity 5 points An explanatory mediator of a causal relationship in one context may not mediate in another context

31. External Validity 5 points What is context-dependent mediation?

32. Construct Validity 1 point Define construct validity

33. Construct Validity 1 point What is the degree to which a measurement/test/instrument measures what it claims to measure?

34. Construct Validity 2 points Constructs that are supposed to be related are related and constructs that are supposed to be unrelated are unrelated

35. Construct Validity 2 points What is convergent and divergent (or discriminant) validity?

36. Construct Validity 3 points In my study, I only used self-report measures to operationalize all of my constructs. This causes a threat to construct validity

37. Construct Validity 3 points What is mono-method bias (or explanation of why this is a threat)?

38. Construct Validity 4 points Two ways to ensure strong construct validity

39. Construct Validity 4 points What are using multiple operationalizations of the same construct and ensure your measure is correlated with a relevant behavioral outcome or with measures of similar constructs (+ other acceptable answers)?

40. Construct Validity 5 points Two ways to assess construct validity

41. Construct Validity 5 points What are face validity (does your measure look like what you want to measure) and content validity (does your measure contain all parts that theory indicates it should)?

42. Misc 1 point The appropriateness of a conclusion or decision

43. Misc 1 point What is validity?

44. Misc 2 points When trying to make a causal claim, we prioritize these two types of validity

45. Misc 2 points What are internal and construct validity?

46. Misc 3 points One way to protect against unreliable treatment (intervention) implementation

47. Misc 3 points What is train your research assistants in a thorough and consistent manner; monitor treatment implementation and provide consistent feedback to avoid continued errors?

48. Misc 4 points Random assignment predominantly influences __________ validity and random sampling predominantly influences ______________ validity

49. Misc 4 points What are internal and external?

50. Misc 5 points A study that has strong internal, external, construct, and statistical conclusion validity