Research Methods

1. Research Methods Victim Academy 2008 Dr. Helen Eigenberg

2. Objectives Identify the foundations of science and the scientific method as a method of gathering knowledge Understand how and where information about research findings can be obtained. Be familiar with fundamental research and evaluation methods. Identify how practitioners and researchers can partner for better success

3. Problems with Statistics and Victim Advocacy Use too many figures Don’t know the source or methodology of the data Public is skeptical of statistics Need to clarify that data you present represents expert conclusions – not just one isolated study Statistics are valuable but context is important -- sometimes over-reliance on data ignores the moral basis of the issue

4. Ways to establish knowledge the encyclopedia says the bible says my parents, teachers, etc say most recent literature says the research demonstrates my experience my intuition SCIENCE

5. What is reality anyway? Nothing more than a collective hunch. Trudy the bag lady -- Lilly Tomlin’s one woman show

6. Foundations of the Scientific Method

7. Science is an approach -- a method – to gain knowledge Research is the process whereby we test relationships Science cannot make decisions on values (i.e. death penalty) Social science is science – same method – applied to social phenomenon Definitions

8. Science helps guard against: Inaccurate observation Overgeneralization Selective observation Illogical reasoning Personal involvement in research Mystification

9. Where Information About Research Findings Can Be Obtained

10. Exercise 9 groups: Newspapers, Television, Magazines, Textbooks, Internet, Government and Advocacy Organizations, Academic Journals, Experts in the Field

11. Newspapers, Television, Magazines Easily accessible Information often limited in detail Lacks context in terms of larger body of literature Sometimes ideological bias Use of expert opinion as opposed to review of scientific literature

12. Textbooks Good overview of basics in a given area/field Generally accessible Good overall reference for more detailed sources Not generally cutting edge (dated minute they are printed) Example: Karmen, A. (2006) Crime Victims. Belmont, CA: Wadsworth.

13. Internet Accessibility Host of detailed information Listserves Need to evaluate source/host of site Sometimes hard to isolate specific information w/out experience in field Watch for information without citations Watch for ideological biases

14. Governmental and Advocacy Organizations Accessibility Host of detailed information Most have good websites Free materials (posters, handouts, information sheets, referrals, etc.) Be sure it is a legitimate governmental/advocacy organization May have experts you can consult May have newsletters

15. Academic Journals Up to date material More depth on specific issues Constitute “cutting edge” information in the field Can be too full of technical jargon May be difficult to understand technical statistics, etc. (Good article should have clear intro and conclusion) Expensive, difficult to access Examples: Violence and Victims, Journal of Interpersonal Violence, Violence Against Women, Feminist Criminology

16. Academic Partners/Experts Can identify experts that will save you time locating specific information/studies Can “use” their credentials (legitimacy) Generally will provide at least some consulting for free Access to graduate students/interns/theses/research projects, etc.

17. Things to be Mindful of when Evaluating Research Studies

18. Be wary of changes over short periods of time and clever manipulations of data

19. i-Clicker Question There was one rape last year and two this year – rape has increased ______% A: 1% B: 10% C: 50% D: 100%

20. i-Clicker Question There were 100 rapes last year and 101 this year – rape has increased ___% A: 1% B: 10% C: 50% D: 100%

21. i-Clicker Question Rape increased 10% last year. What does that tell us? A – That there is a trend and that rape is going up. B – That there is a trend and rape is going down. C – None of the above

22. Which crime increased the most in this graph?

23. Correlation and Causation

24. i-Clicker Question Drug use and battering are correlated. This statement means that using drugs causes someone to batter. A: True B: False

25. Does drug use cause battering?

26. Be careful of studies that only report correlations for two variables –Over simplistic Explanations DRUGS

28. Issues with Samples

29. i-Clicker Question If you want to establish the risk of rape in the U.S. you could do a study asking UTC students about how often they have been raped. This sample would be acceptable. A: True B: False Importance of probability (random) vs. non-probability (convenience) samples

30. i-Clicker Question If you want to establish the risk of rape in the U.S. you need to survey at least 1 of every 3 women in the U.S. A: True B: False Importance of Sample Size (beware of very small samples (e.g. < 100)

31. i-Clicker Question If you want to determine if white children and minority children are at equal risk of child abuse in the U.S., you could use a sample of child protective records and look at race in these cases. A: True B: False Are the subjects representative of your study group? (e.g. race, age, income, etc.)

32. i-Clicker Question You have surveyed a random sample of Tennessee women and found that 1 in 4 have been a victim of an attempted or completed rape. You can assume that this figure represents the rape rate for women in the U.S. A: True B: False Beware of studies that over generalize – make claims about larger group based on a dis-similar group or group that is not representative (e.g. cannot generalize about a study from TN to all states)

33. Replication

34. i-Clicker Question A national study has just found that mandatory arrest alone deters further acts of domestic violence – e.g. that batterers who are arrested are less apt to commit additional acts of violence regardless of the response of the rest of the criminal justice system. Your police department should use this study to impose a mandatory arrest policy. A: True B: False Beware of studies that lack replication.

35. Replication is critical. One finding – regardless of outcome, should be treated with skepticism until it has been replicated Medical studies example Minneapolis Police Study example

36. Statistical Significance

37. i-Clicker Question Violent crime rates in Chattanooga rise from 1100/100,000 people in 2001 to 1150 in 2002; however, the increase is not statistically significant. This means that statistically speaking there has been no change in the crime rate. A: True B: False Change is non-significant; therefore – statistically – it means there has been no change in the crime rate – what you see is merely random fluctuation and/or error

38. Statistical Significance Most important statistic – uses probability theory to ask whether the finding reported would have normally occurred based on chance alone Generally is .05 or less Means that 95% of the time you can be confident that your finding would not occur based on chance alone

39. Example: When comparing violent crime rates from last year to this year we find a significant difference – means that with 95% confidence, we can conclude that something other than chance or random variation caused this change (i.e.. Change in enforcement, economy, massive prison overcrowding dumps tons of criminals in your community for early release)

40. Confidence Intervals

41. i-Clicker Question Obama is ahead of McCain in the polls by 5% and the poll has a margin of error of + or – 3%. Therefore we can conclude that Obama is ahead statistically speaking. A: True B: False Importance of confidence intervals.

42. Confidence Interval It is the range of accuracy associated with a number -- the number predicted is valid plus or minus a certain amount (usually 5%) I.e. polls – Bush 50%, Gore 48% + 5% points meant statistically there was no difference in their popularity – cannot predict with confidence who is ahead – statistics don’t allow that level of accuracy

43. Example: NCVS

44. Reliability and Validity

45. i-Clicker Question A broken thermometer can measure the temperature reliably. A: True B: False Importance of reliability versus validity.

46. Reliability Does the measurement device measure consistently (reliably)? Most often used in the context of scales

47. Validity Does the measurement device adequately reflect/capture the concept it is supposed to? Are you measuring what you think you are measuring? Can be reliable but not valid – or valid but not reliable

48. Examples of Validity Problems NCS/NCVS National Family Violence Survey CTS (75/85) DV Danger to Police Study

49. Old NCS Screening Questions (1972-92)

50. New NCVS Screening Questions

51. New NCVS Screening Questions – cont.

52. Change in Questions Improved validity resulted in increase in 157% in rape reported by respondents to the survey and increase in 57% in assault.

53. CTS: Screening Question Respondents asked to “think of situations in the past year when they had a disagreement or were angry with a specified family member and to indicate how often they engaged in each of the acts included in the CTS”

54. Violent Acts Included Minor throwing something pushing shoving grabbing slapping Severe kicking biting hitting with fist or something else beating someone up using or threatening to use weapons

55. Findings: NFVS 28% of the couples experienced violence sometime in their past (prevalence rate) Men and women commit about equal amounts of violence – most common study used to refute that domestic violence is a gendered issue – e.g. that men beat women intimates more than vice versa Survey fails to distinguish between self-defense and acts of aggression

56. Studies using CTS only ones to find evidence of “mutual” violence Studies that show women majority of victims: NCS, NCVS, VAWS Victimization surveys from other countries Studies using shelter records Studies using police, prosecution, and sentencing records Studies using emergency room records Studies using homicide records

57. Belief that DV Calls are Most Dangerous Calls for Police Officers Trained police officers that these cases were dangerous -- that victim would turn on officer Used FBI data to examine officer fatalities Category used to make this argument was “disturbance calls” which included bar fights, gang calls, general public disturbances short of a riot, and “man with a gun calls”.

58. Re-analysis of data using only domestic violence cases finds:

59. Collaborative Research Practitioners and Academicians

60. Barriers to Research by Victim Providers Lack of resources (e.g., time and money). Participation is too time consuming. Difficulty identifying victims who are willing to participate in research projects. Lack of trust between victims/service providers and the researchers Need for practitioners to be actively involved in the conceptualization of research. Varying "measurements" utilized by researchers that result in statistics that are "suspect." Concerns over victim safety

61. Barriers for Researchers/Academicians to Overcome Sometimes lay persons do not fully understand the significance of the research. Tendency to see researchers as people who are removed for the “real world” and who do not care about “practical” application of their research – especially when testing theory Lack of mutual respect between practitioners and researchers Lack of understanding of varied environments of researchers (e.g., community research organizations, state and federal research organizations, academicians). Institutional Review Boards

62. IRB: (http://www.hhs.gov/ohrp/) Since 1970s, federal regulations require that any entity applying for a federal grant or contract which involves biomedical or behavioral research involving human subjects establish a board (to be known as an "Institutional Review Board") to review biomedical and behavioral research involving human subjects to protect the rights of the human subjects.

63. Mutually Beneficial Partnerships between Practitioners and Academicians Sharing resources/expertise Mutually advantageous research opportunities Partnering grants Internships Evaluation studies Advocacy

64. Trust your instincts on statistics BUT verify with an expert if you are unsure about methodological problems!