Research Methods

Research Methods Techniques Strengths Problems

Experiments • Experiments involve manipulation and control. They use: • IVs and DVs. • Experimental and control groups • Pretesting and posttesting • They seek to understand causation. The IV is thought to lead to, affect, the DV

Logic of experiments (IV) DV Experimental group DV DV Control group DV Ideally, random assignment of subjects to experimental and control groups

Logic of experiments (continued) • If random assignment (randomization) is used, the groups are likely to be equivalent. The P is that the groups will be the same, differences canceled out. • Logic or rationale: If the control and experimental group are the same on the DV at the start of the study, and the only difference is that the experimental

Logic (continued) • group receives the IV and the control group does not, then • If there are differences between the groups on the DV at the end of the study, then they must be due to the IV.

Equivalent groups • Importance of equivalent groups • Randomization best, but not always feasible • Might match on important variables (sex race, SES, intelligence, etc..) • Problem with matching: might not match on some important variable, and the groups will not be equivalent

Some designs • Classic experiment • One group pre-post test • Static-group comparisons

Threats to validity (extraneous) • Internal validity: variables which might affect the DV, aside from the IV • History: events which occur during the course of the study • Maturation: changes that take place because of the aging process • Testing: changes that occur because of pretesting

Threats (continued) • Selection: nonequivalent groups • Regression: problematic when extreme groups are used. People with very low or very high scores tend to score closer to the mean on retesting. (holds true for other phenomenon, such as inheritance to traits) • Mortality: dropping out of a study

External validity (generalization) • One study shows that the IV affects the DV. Will this be true in other settings with other subjects? • Interaction of testing and IV • Experimenter effects, reactive arrangements • Hawthorne study

Advantages of experiments • We can more directly observe the effects of one variable on the other, holding other variables constant • Sometimes can be done with little time and money • Experiments can be more easily replicated than other methods

Disadvantages of experiments • Artificiality • Example: TV and aggression • Example: Zimbardo study

Surveys • Ask people what they think, feel, did • Initial steps in surveys • Selection of population and sample • Development of the questions into a questionnaire • Pretesting and review

Survey methods • Group administered • Mailed • Face to face interviews • Telephone interviews • Each has its advantages and disadvantages

Group administered • Assemble subjects together, have them fill out the questions. A proctor is available to answer questions • Advantages: economical, quick, problems can be handled immediately • Problem: it may not be possible to assemble subjects. Works best with groups who will be together anyway

Mailed surveys • Mail out: (1) questionnaire; (2) cover letter explaining the purpose of the study, who is doing it, and what the results will be used for; and (3) a self-addressed “stamped” envelope. • bulk rate • on the return envelope: stamps, metering, business reply permits

Mailed (continued) • business reply permits : more expensive to purchase, but you only pay for those surveys returned. • Follow-ups--reminder postcards vs. new questionnaire, cover letter and envelope

Advantages of mailed surveys • Relatively inexpensive • Fairly fast results • Geographic accessibility--can reach almost anyone

Disadvantages • Low response rates, with the potential for biased results • No control over the subjects • No control over misinterpretations of the questions • Will subjects be truthful? • More impersonal than other methods

Face to face interviews • Training of interviewers • Identification • How to dress (depends on subjects) • Approaching people (appointment vs. showing up) • How to administer questions and record responses

Face to face (continued) • Practice sessions • Anticipating as many situations as possible and handling problems • Need to monitor interviewers

Advantages of face to face • High response rates • Control over subjects • Can “probe” and ask more complicated questions • Deals better with less educated or more illiterate populations • Better for establishing rapport and getting more information

Disadvantages of face to face • Time-consuming • Expensive • Interviewer bias • Some people may lie to face to face interviewers

Telephone interviews • Same procedures as face to face, but over the telephone • Use of computer systems • Random digit dialing • Coding data as questions are answered

Advantages of telephone • Fast • Less expensive than face to face • Can still ask more complicated questions and probe

Disadvantages • More impersonal • Can terminate the interview more easily • response rate not as high as face to face, but higher than mailed • Cannot be very long

Observational Research • Problem with surveys: what people say and what they actually do are not always the same. • Additionally, there may be important information that no one thought to ask • Therefore it is sometimes useful to observe behavior as it occurs, not ask about it or do experiments

Distinctions in observations • Laboratory vs. field • One subject vs. a group • Unstructured vs. structured • Unstructured: observe a variety of aspects of environment and behavior • Structured: interested in a particular behavior

Distinctions (continued) • Unstructured: Margaret Meade’s studies, prison subculture studies • Structured: police brutality, police discretion at the time of arrest • Participant vs. non-participant observer • Participant: joins in the activities in the environment. May be a genuine member (Becker as a jazz musician)

Distinctions (continued) • Might be pretend to be a member of the group. Problem: is it ethical to deceive? (Rosenhan study, “end of the world” study) • With participant, the observer might become too involved to be objective, and might influence the group’s behavior

Distinctions (continued) • Nonparticipant researcher • Might conceal him/herself. Problem: people’s right to privacy (Not a problem if observing in a public place, where people do not have an expectation of privacy) • Might identify self as researcher.

Distinctions • Problem: Might be rejected. Also, people might act differently if they know they are being observed. • The pros and cons must be considered in such observational studies

What do observers study? • Practices (behaviors) • Episodes and encounters • Roles • Relationships • Groups and organizations

Steps in observational research • Literature • Informants • “Getting in” and rapport, explanation of research • Sampling: often nonprobability--convenience, quota, or snowball. • Observations

Steps in observation (continued) • Unstructured: field notes • Structured: record events in categories as they occur • Unstructured interviews--probe and try to determine what is going on • Categorizing observations • Examples: history of group, biographies

Steps in observation (continued) • religion, rule-breaking, roles. • Finding patterns, deviations in the patterns • Writing up results • More difficult than other methods

Advantages of observation • Depth of understanding gained • Flexibility of the method • Rich source of hypotheses for other types of studies

Disadvantages of observation • Might misinterpret • Difficulties of interpretation • Small number of subjects • Sampling problems--will the observations be representative • Time-consuming • Expensive

Unobtrusive research • Looking at traces left behind • Example: looking at well-worn floors in museums to determine which exhibits are most popular • Our only way of being able to study prehistoric man • Problem: dangers of misinterpretation

Record Research • With the advent of the written word, we can study behavior. Our only method for studying the past. • For example, has America always had a crime problem, particularly violent crime, or has this always been part of our culture? • Have child rearing practices changed?

Content analysis • Use of documents to study a behavior such as crime: books, magazines, songs, newspapers, letters, diaries • Example: fear of crime in Switzerland studied by examining newspapers and Parliamentary debates for concern about crime (In contrast to the U.S., the topic was not raised often)

Content analysis (continued) • Themes put into categories • Need to distinguish manifest (actual) and latent (hidden meanings) content • Use of computers to search for words in documents (advantages and disadvantages)

Record (archival) research • Use of existing statistics and records • government documents • computer tapes of agencies • agency files • Examples: Do judges sentence females differently than males? What percentage of cases are plea bargained? etc..

Advantages of study of records • Inexpensive • Possible to study processes occurring over long periods of time • No effect on those being studied; records are being kept anyway • Can be used for historical and comparative uses • Large numbers of subjects

Disadvantages of record research • Records may have gaps, be incomplete • Ecological fallacy: patterns at a group level may be different than those at the individual. Example: Protestant countries have higher suicide rates than Catholic countries. Are Protestants committing suicides--could be Catholic minority

Disadvantages--record research • Problems with statistics • Official statistics--unreported crime • Crimes depend on laws, which vary • Numbers depend on discretion of officials and their norms, at arrest, plea bargaining and convictions, sentences, probation/parole revocation • Necessary to look behind the numbers

Evaluation Research • Determining the effects of programs, agencies, policies and laws • Four components • Needs assessment • Monitoring • Outcomes • Cost/benefit and cost/effectiveness

Needs assessment • Is the service/change/intervention needed? • Who is the target population? i.e.., who do we want to reach the most? What are their characteristics? • What problems can be anticipated as the change is made? i.e.., halfway houses expect neighborhood resistance

Needs (continued) • Can problems be addressed successfully? • Ways of assessing: records, surveys, key informants and gatekeepers • Without needs assessment, inappropriate interventions might be made

Monitoring • Looks at the processes taking place as the intervention is made • Are the services being provided? In the case of a law, is it being utilized? • Is the intervention reaching the target population? i.e., if your intervention is school based and your targets are truants, it will not work

Research Methods