440 likes | 638 Views
Research Design. 10/3/2013. Graduate & Professional School Fair: Oct. 7. Explore Graduate, Law, Medical and Professional Schools THIS coming Monday! Monday , October 7, 2013 10:30 a.m. – 1:30 p.m.. Mabee Ballrooms, 3 rd floor of the Ragsdale Center
E N D
Research Design 10/3/2013
Graduate & Professional School Fair: Oct. 7 • Explore Graduate, Law, Medical and Professional Schools THIS coming Monday! • Monday, October 7, 2013 • 10:30 a.m. – 1:30 p.m.. • Mabee Ballrooms, 3rd floor of the Ragsdale Center • 48 schools & 2 test prep services will be participating.
Readings • Chapter 4 Research Design and the Logic of Control (Pollock)
Office Hours For the Week • When • Friday 10-12 • Monday 10-12 • And by appointment
Homework (Due today) • Chapter 2 • Question 1: A, B, C, D, E • Question 2: B, D, E (this requires a printout) • Question 3: A, B, D • Question 5: A, B, C, D • Question 7: A, B, C, D • Question 8: A, B, C
Course Learning Objectives • First, students will learn the research methods commonly used in behavioral sciences • Students will learn the basics of research design and be able to critically analyze the advantages and disadvantages of different types of design.
What a Research Design Includes • Unit of AnalysisVariables • Data • Hypotheses • Justification for your statistics
The Goal of A Research Design is to create a study that can demonstrate causality
Working for Causality Internal validity of Research Design
Internal Validity • Setting up Research Designs Properly • Having control over the experiment. Especially the independent variable.
Threat 1:History • You cannot account for all previous knowledge and events • You cannot control for all potential independent variables
Threat 2: Maturation • We get older • We get wiser • We get tired (short term) • These are natural changes
Threat 3: Experimental Mortality • Participants leave the research study • Those who remain, may not be like the target group
Threat 4: Selection Bias • Choosing the wrong sample • Picking Respondents to favor your results • Excluding cases or respondents that do not fit your goals • Using volunteers!
Threat 5: Instrumentation A Bad Measure Changing a Measure to Fit your Needs
Threat 6: Design Contamination • People intentionally or unintentionally act differently • “Instrument Reactivity” • We Guess the test, we share information
Which of these are Most Common? • History • Maturation • Contamination is the worst!
Making it Generalizable External validity
What is External Validity • Can we generalize our research • Does it apply to other populations, settings and times. • Does the test apply to the real world
The Lowest Form • “Remember when is the lowest form of conversation” • Personal experience is the weakest form of evidence • Stories • Hypothetical Examples
Unique Program Features/Location • Can it only work in one place • Does the artificial setting harm validity • Creaming (popular with education policy)
Effects of Selection • This is also a threat to internal validity • Subjects in study are unrepresentative • Using Volunteers • Poorly Drawn Samples
Reactive Effects of Experiments • Taking the experiment out to the real world • Participants act differently because they know they are being studied • Administrators work differently as well.
Replication • Can You Reproduce your findings? • Can others reproduce your findings?
Biggest problems of External Validity • Failure to replicate the exam. • The Lack of Real World-Applicability • To increase external validity, increase your sample!
The Goal Is both kinds of validity. The Social Sciences do better on external than internal
Research Design Experimental vs. Non Experimental
Types of Designs • Experimental (mirror the natural sciences • Non-experimental- sacrifice internal for external
The Classic Experimental Design • This design is the best way to demonstrate causality • You have total control • This has 4 key parts
Step 1- Random Selection and Assignment • Subjects are selected at random and assigned to an experimental or control Group • You cannot pick people who you want to be in the experiment • You have two equal groups of participants • This ensures an even baseline between your group
Step 2- A pre-test is given to both groups • A Pretest measuring the Dependent variable is given to each group
Step 3- The introduction of the independent variable • The experimental group receives the independent variable (test stimulus) and the control group does not
Step 4- A Post Test is Given • The researcher measures the dependent variable for both groups after the experimental stimulus is given.
What You want to show R= Random assignment O 1 & O2 Observation for the two groups at time 1 X =Introduction of the treatment for the experimental group O3 & O4 Observation of the two groups at time 2 Note change for the two groups (overtime and between)
This type of design is common • In the natural sciences • In Psychology • In Food Tests
It is rare in Social Sciences • Legal and Ethical Problems • Historical Problems • Reactivity Problems.