Work Session 2 Methodological Approaches

1. Work Session 2Methodological Approaches Office of Research, Evaluation and Policy Studies Ximena D. Burgin, Ed.D. December 2nd, 2010

3. Intent of a Study Purpose statement: Overall direction i.e., The purpose of this grant is to enhance and expand existing partnerships. The Partnership and Community Readiness section of the application should demonstrate that a school-community partnership has been in existence for 6 or more months prior to submission of the application (henceforth referred to as preexisting school-community partnership). The preexisting school community partnership should involve at least two of the four required SS/HS partners that have worked together on issues of school safety, drug and violence prevention, and/or healthy childhood development issues for 6 or more months prior to submission of the application.

4. Work Plan Requirements… The SS/HS Initiative draws on the best practices of education, justice, social services, and mental health systems to provide integrated and comprehensive resources for prevention programs and prosocial services for youth. This comprehensive plan include six elements to address problems of school violence and alcohol and other drug abuse: ♦ Element 1: Safe school environment. ♦ Element 2: Alcohol and other drugs and violence prevention and early intervention programs. ♦ Element 3: School and community mental health preventive and treatment intervention services. ♦ Element 4: Early childhood psychosocial and emotional development programs. ♦ Element 5: Supporting and connecting schools and communities. ♦ Element 6: Safe school policies.

5. Grant Measurements… GPRA1 – SS/HS grant sites will experience a decrease in the number of violent incidents at schools during the 3-year grant period. GPRA2 – SS/HS grant sites will experience a decrease in substance use during the 3-year grant period. GPRA4 – SS/HS grant sites will increase mental health services to students and families during the 3-year grant period.

6. Grant Outcomes… GPRA1 – SS/HS grant sites will experience a decrease in the number of violent incidents at schools during the 3-year grant period. Outcomes: Objective 1. In each of three years cumulatively, decrease by 60% the percentage of violent incidence occurrence for each of the categories: threatening behavior, unprovoked attack on student, fighting, fighting with or threatening teacher, possession or use of dangerous weapon, arson, and endangering safety of others. GPRA2 – SS/HS grant sites will experience a decrease in substance use during the 3-year grant period. Outcomes: Objective 2. In each of three years cumulatively, decrease by 60% the percentage of substance abuse occurrence for each of the categories: alcohol, tobacco, and controlled substances (possession or use). GPRA4 – SS/HS grant sites will increase mental health services to students and families during the 3-year grant period. Outcome: Objective 4. In each of three years cumulatively, increase by 50% the percentage of students and families accessing and receiving mental health services, as measured by frequency of service requested or inquired about, follow-through in accessing such service, and persistence in pursuing such service to closure.

7. Is the study feasible? • Tradeoffs between rigor and practicality. You may have to: • Control the implementation of your program more carefully. • Ask program participants lots of questions. • Evaluate research rigor, desire and practicality.

8. Is the study feasible?…cont. • Practical considerations for a feasible research project: • How long the research will take to accomplish. • Ethical constraints • Needed cooperation • Costs.

9. GPRAs &Outcomes (Research Questions)

10. Quantitative Research Questions • Quantitative research is mainly concerned with the degree in which phenomena possess certain properties, states and characters, and the similarities, differences and causal relations that exist within and between.

11. Qualitative Research Questions • Qualitative research is concerned with the properties, the state and the character (i.e., the nature, of phenomena). Emphasis on processes and meanings.

12. Grant Outcomes… GPRA1 – SS/HS grant sites will experience a decrease in the number of violent incidents at schools during the 3-year grant period. Outcomes: Objective 1. In each of three years cumulatively, decrease by 60% the percentage of violent incidence occurrence for each of the categories: threatening behavior, unprovoked attack on student, fighting, fighting with or threatening teacher, possession or use of dangerous weapon, arson, and endangering safety of others.

13. Grant Outcomes… GPRA2 – SS/HS grant sites will experience a decrease in substance use during the 3-year grant period. Outcomes: Objective 2. In each of three years cumulatively, decrease by 60% the percentage of substance abuse occurrence for each of the categories: alcohol, tobacco, and controlled substances (possession or use).

14. Grant Outcomes… GPRA4 – SS/HS grant sites will increase mental health services to students and families during the 3-year grant period. Outcome: Objective 4. In each of three years cumulatively, increase by 50% the percentage of students and families accessing and receiving mental health services, as measured by frequency of service requested or inquired about, follow-through in accessing such service, and persistence in pursuing such service to closure.

15. Qualitative Analysis • Qualitative analysis seeks to capture the richness of people's experience in their own terms. Understanding and meaning emerge from in-depth analysis of detailed descriptions and verbatim quotations.

16. QUANTITATIVE (Deductive) Ho, Ha RQ Variables Predict Instruments before study starts Difference over time. QUALITATIVE (Inductive) Ho, Ha RQ Single concept (central phenomena) Build themes Open-ended questions. Stance-emerge during the study Deep understanding. Differentiating Approaches

17. Research Methodology • Experimental design • Correlational design • Survey design • Grounded theory design • Ethnographic design • Narrative design • Mixed design • Action research design

18. Experimental design • To establish possible cause-effect between IV and DV. • Between-group designs • True experiments • Quasi-experiments • Factorial analysis • Within-group or Individual designs • Time series experiments (interrupted, equivalent) • Repeated measures experiments • Single-subject experiments

19. Correlational design • To predict scores and explain the relationship among variables. Raw Data: Self-esteem GPA score 1 42 3.8 2 10 1.4 3 15 2.5 4 22 3.1 Resource: http://www.uky.edu/Classes/PSY/313-001/research%20method%20ch2s.ppt

20. Survey Design • To describe attitudes, opinions, behaviors, or characteristics. • Cross-sectional • survey designs Resource:http://images.freshmeat.net/editorials/r_intro/images/line-graph-1.jpg

21. Survey Design… • Longitudinal survey Designs Resource: http://www.nature.com/eye/journal/v21/n6/images/6702322f1.jpg

22. Survey Design… • Trend studies Resource: http://coastalchange.ucsd.edu/images/wet_dry_2.gif

23. Survey Design… • Cohort studies Resource: http://www.ncsall.net/fileadmin/resources/research/br11-graph2.gif

24. Survey Design… • Panel studies Resource: http://www.nsf.gov/news/special_reports/survey/images/dynamic_logo_sm.jpg

25. Grounded Theory Design • Systematic design • Open coding • Axial coding • Selective coding

26. Grounded Theory Design… • Emerging design • Exists at the abstract level. • Grounded in data • Meet four central criteria: • fit • work • relevance • modifiability

27. Grounded Theory Design… • Constructivist design • Position between a positivist view and postmodern view. • Interest on views, values, beliefs, feelings, assumptions, and ideologies of individuals. • Explain the experience of the phenomenon or process rather than facts and acts.

28. Ethnographic Design • Realist ethnography • Confessional ethnography • Life history • Autoethnography • Microethnography • Ethnography case study

29. Ethnographic Design… • Critical ethnography • Feminist ethnography • Postmodern ethnography • Ethnography novels

30. Narrative Research Design • Autobiographies • Biographies • Life writing/histories/stories • Personal accounts/narratives/documents • Narrative interviews • So on

31. Mixed Methods Design • Triangulation Mixed Methods Designs • Embedded Mixed Methods Designs Resource: Hanson et al, 2005

32. Mixed Methods Design… • Explanatory Mixed Methods Designs • Exploratory Mixed Methods Designs Resource: Hanson et al, 2005

33. Mixed Methods Design… Resource: Burke & Onwuegbuzie, 2004

34. Mixed Methods Design… Resource: Hanson et al, 2005

35. Action Research Design • Practical • Participatory

36. Sampling • Random sampling methods: • Simple random sampling

37. Sampling… • Stratified random sampling • Cluster random sampling Resource: http://www.socialresearchmethods.net/kb/Assets/images/sampstrt.gif

38. Sampling… • Two-stage random sampling: combine Cluster random sampling & Simple random sampling Resource: http://www.clas.ufl.edu/jur/200411/images/emerson_2.jpg

39. Sampling • Nonrandom sampling methods: • Systematic sampling Resource: http://www.disector.com/stereology-analysis-graphics/sampling1.gif

40. Sampling… • Convenient sampling Resource: http://www.nedarc.org/nedarc/collectingData/participantSelection/probabilitySampling/images/simpleSample.gif

41. Sampling… • Purposive sampling Resource:http://www.david.curtis.care4free.net/esfig3.gif

42. Sample Size • It depends on the type of study. However, the larger the sample the better.

43. Quasi-Experimental Design REQUEST FOR APPLICATIONS Evaluation of State and Local Education Programs and Policies CFDA Number: 84.305E C. Implementing Rigorous Evaluations “The methodological requirements for evaluations under this program…other experimental and quasi-experimental designs that substantially minimize selection bias or allow it to be modeled may be employed” (pg.7). a. Random Assignment “Studies using random assignment to intervention and comparison conditions have the strongest internal validity for causal conclusions and thus are preferred whenever they are feasible” (pg.7).

44. Related to participants: History Maturation Regression Selection Mortality Interaction with selection Related to treatments: Diffusion of treatments Compensatory equalization Compensatory rivalry Resentful demoralization Threats to Validity: Internal • Related to procedures: • Testing • Instrumentation

45. External: It may affect generalizability: Interaction of selection and treatment Interaction of setting and treatment Interaction of history and treatment Threats to Validity: External

46. Threats to Internal Validity in Types of Experimental Designs

47. Threats to Internal Validity in Types of Experimental Designs

48. Types of Between-Group Designs Quasi-Experimental Designs Time Time