1 / 27

First Two Years Project Cathy Crosby-Currie Christine Zimmerman Bringing Theory to Practice March 2007

First Two Years Project Cathy Crosby-Currie Christine Zimmerman Bringing Theory to Practice March 2007. Modeling the Multiple Influences on Civic Development and Well-Being. Astin’s Theory of Involvement: I-E-O Model .

lindsay
Download Presentation

First Two Years Project Cathy Crosby-Currie Christine Zimmerman Bringing Theory to Practice March 2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. First Two Years Project Cathy Crosby-Currie Christine Zimmerman Bringing Theory to Practice March 2007

  2. Modeling the Multiple Influences on Civic Development and Well-Being

  3. Astin’s Theory of Involvement: I-E-O Model

  4. Terenzini’s General Conceptual Model of College Influence on Student Learning Terenzini, P., Springer, L., Pascarella, E., Nora, A.(1995). Influences Affecting the Development of Students’ Critical Thinking Skills. Research in Higher Education, 36, 23-39.

  5. Methodological Considerations

  6. Experimental v. Quasi-Experimental Design • Key difference between experimental and quasi-experimental • Researcher’s control over the “input” variable • Experimental: YES! -> cause/effect conclusions • Quasi-Experimental: NO! -> examine relationships • Control v. comparison groups

  7. Experimental v. Quasi-Experimental Design • Quasi-experimental power comes from: • Ability to detect change through design • e.g., interrupted time series design • Equivalence of comparison group to experimental group

  8. Longitudinal v. Cross-Sectional Designs • Cross-sectional: • Comparing groups of different ages at one point in time • Convenient but lacks statistical and conceptual power • Longitudinal: • Comparing individuals to themselves across time • Multiple cohorts is ideal

  9. St. Lawrence’s Quasi-Experimental, Longitudinal Design • Participants: • Two Cohorts – Selected Students from Classes ‘09 and ’10 • Experimental Group – students in Brown College • Second year added a second experimental group • Comparison Group – non-equivalent group matched on key variables of interest • Comparison Group Sample II.xls

  10. St. Lawrence’s Quasi-Experimental, Longitudinal Design • Data Collection • Pretest (9/05 & 9/06) • Posttest (2/06 & 2/07) • Follow-up (4/07 & 4/08)

  11. Challenges of Quasi-Experimental and/or Longitudinal Designs • Creating comparison group(s) • Participant attrition • Communication incl. letter from president • Personalized letters & email • Contacting students multiple times/multiple ways • Accommodate students’ schedules • Institutional Review Board Approval • Reframe as a positive contribution to your research not a hurdle to overcome

  12. Challenges of Measurement • Valid and Reliable Measures • Direct - Indirect Measures • Quantitative - Qualitative Data • Process - Outcomes Measures

  13. Reliability … is the consistency or repeatability of responses • Random error (noise) • Systematic error (bias)

  14. Reliability (cont.) Ways to increase data reliability: • Clear directions • Clear questions • Consistent order of questions • Clear survey layout • Trained proctors/interviewers • Consistent data entry and scoring

  15. Reliability (cont.) How to assess the reliability of your instrument: • Pilot-test your study • Test-retest your survey • Focus-group survey or interview questions • Include similar questions in same questionnaire

  16. Validity … the extent to which the instrument truthfully measures what we want to measure • How well does the instrument content match what we want to measure? • Do respondents interpret the questions correctly? • Do respondents’ answers reflect what they think? • Are the inferences we make from this study accurate? Can they be generalized?

  17. Validity (cont.) • Use multiple measures and multiple methods • Derive measures from literature review & existing research / participate in national survey instruments and tests • Expert review • Pilot-test your own survey How to establish validity:

  18. Direct – Indirect Measures Direct: tangible, actual evidence Indirect: proxy for what we try to measure

  19. Qualitative – Quantitative Measures Qualitative: unit of data = words Quantitative: unit of data = numbers

  20. Process – Outcomes Measures Process Measures • What did we do?(=data to demonstrate the implementation of an activity/program) Outcomes Measures • What are the results?(= data used to measure the achievement of an objective/goal) • Initial • Intermediate • Long term

  21. Administrative ChallengesAnd Best Practices • Buy-in and Support • Form campus partnerships early on • Build on existing data collections • Institutional survey cycles and survey timing • Copyrights of survey instruments • Liability for use of certain measures • Survey recruitment & retention

  22. Select Survey Instruments and Literature

  23. Sampling of Survey Instruments • Entering Student Survey • CIRP Freshman Survey (HERI, UCLA) • College Students Expectations Questionnaire CSXQ (Indiana) • Enrolled Undergraduate Students/Alumni • As a continuation of CIRP: Your First College Year/College Senior Survey • As a continuation of CSXQ: College Student Experience Questionnaire (CSEQ) • National Survey of Student Engagement (NSSE) • Consortia Senior and Alumni Surveys (e.g. HEDS. COFHE)

  24. Sampling of Survey Instruments • Depression/Mental Health Measures • Beck’s Depression Inventory (BDI II) • Brief Symptoms Inventory (BSI) • Optimism/Pessimism/Happiness Scales • Mehrabian Optimism/Pessimism Scale • http://www.authentichappiness.sas.upenn.edu/ • Alcohol/Drugs/General Wellness • CORE Alcohol And Other Drugs Survey • ACHA-NCHA

  25. Sampling of Survey Instruments • Civic Development (from Lynn Swaner) • CASA TELEPHONE SURVEY INSTRUMENT • Other National Surveys • HERI Faculty Survey • Other In-House Institutional Surveys • Course evaluations • Program evaluations • Satisfaction studies

  26. Select Literature • The National Center on Addiction and Substance Abuse at Columbia University (2003): Depression, Substance Abuse, and College Student Engagement: A Review of the Literature. Report to The Charles Engelhard Foundation and The Bringing Theory to Practice Planning Group.http://www.aacu.org/bringing_theory/research.cfm • The National Center on Addiction and Substance Abuse at Columbia University (2005): Substance Abuse, Mental Health and Engaged Learning: Summary of Findings from CASA’s Focus Groups and National Survey. Report to Sally Engelhard Pingree and The Charles Engelhard Foundation for the Bringing Theory to Practice Project, in partnership with the Association of American Colleges and Universities.http://www.aacu.org/bringing_theory/research.cfm • Swaner, L.E. (2005). Linking Engaged Learning, Student Mental Health and Well-being, and Civic Development: A Review of the Literature. Prepared for BTtoPhttp://www.aacu.org/bringing_theory/research.cfm

  27. Select Literature • Pascarella, E., Terenzini, P.(1991). How College Affects Students: Findings and Insights from Twenty Years of Research. San Francisco: Jossey-Bass. • Bringle, R. G., Phillips, M.A., Hudson, M. (2004). The measure of service learning: Research scales to assess student experiences. Washington, D.C. American Psychological Association • Suskie, L. (1996). Questionnaire Survey Research: What works. Tallahassee: Association for Institutional Research

More Related