1 / 26

Institutional Research and the Campus Commitment to a Culture of Assessment AIRPO Conference June 20, 2006

Institutional Research and the Campus Commitment to a Culture of Assessment AIRPO Conference June 20, 2006. James A. Anderson, Ph.D. Vice President for Student Success Vice Provost for Institutional Assessment and Diversity University at Albany JAANDERSON@UAMAIL.ALBANY.EDU.

kevyn
Download Presentation

Institutional Research and the Campus Commitment to a Culture of Assessment AIRPO Conference June 20, 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Institutional Research and the Campus Commitment to a Culture of AssessmentAIRPO ConferenceJune 20, 2006 James A. Anderson, Ph.D. Vice President for Student Success Vice Provost for Institutional Assessment and Diversity University at Albany JAANDERSON@UAMAIL.ALBANY.EDU

  2. Integrative Assessment Reporting • Institutional Goals or Objectives • College/Division/Accreditation Goals and Objectives • Department/Program Goals/Objectives/Outcomes • Course-Bases Objectives/Outcomes

  3. Integrated Assessment Process Means Integrating • Institutional Effectiveness • Program Review • Outcomes Assessment • Institutional Research • Enrollment Management • Survey Research

  4. Student-Centered Learning Environment • Refers to a student success effort which is characterized by: • Clearly defined and measurable outcomes in the areas of teaching, learning, retention, and student social-psychological development • Front-loaded activities which engage the student, as soon as possible, to the academic and social expectations of the institution • An ongoing assessment effort associated with student and program improvement • Faculty and staff who are well-trained to deliver the appropriate services

  5. Profiling the Student Support Clientele • What institutionally based information on student needs and characteristics is utilized by student support personnel? • What specific information about student subpopulations (women, students of color, non­traditional students, etc.) is utilized by student support personnel? • What information about student culture, Faculty concerns and campus climate is utilized by student support personnel? • Describe a profile of the students who utilize student support services the most/least? • Who benefits the most/least?

  6. Generate Longitudinal Database to Monitor Key Indicators of Student Success Examples • Overall course completion ratios • Successful completion of gatekeeper courses • Successful completion of developmental educational courses • Semester-to-semester retention • Year-to-Year retention • Degree and certificate attainment rates From: “Achieving the Dream: Community Colleges Count” Lumina Foundation & Pell Institute

  7. Retention Scorecard • Indicators of Institutional Effectiveness • Indicators of Program Effectiveness • Indicators of Student Academic/Social Integration • Indicators of Student Success/Achievement • Indicators of Student Learning • Indicators of Impact of/on Diversity

  8. Questions That May Help You Recast Your Services or Delivery Systems • What are the thinking tasks, intellectual experiences, and/or cocurricula experiences that need to be designed relative to the preparation level and diversity of the students at your institution? • What structures need to evolve to assure that students have the opportunity to enhance their academic self-concept and understand their role in the culture of learning at your institution? • What data sets do I need to access to generate the appropriate questions and answers about student engagement and student learning?

  9. Integrated Model of Undergraduate Education

  10. ACADEMIC OUTCOMES • Grades (General) • Grades (By Course Type) • Retention (Semester to Semester) 1st to 2nd Year • Graduation Rates • Time to Degree (Calendar vs. Seat Time) • Changes in Academic Interests • Transfers to Four-Year Institutions

  11. STUDENT OUTCOMES • Academic Adjustments/Integrations • Social Adjustments/Integration • Attitudes Towards Learning • Expectations for Success • Academic Self-Esteem • Utilization of Academic Supports • Non-Cognitive Factors

  12. Questions to Ask Yourself • Which outcome and assessment methods will help me understand what it is that I am doing that is leading to the outcome? • Which outcome and assessment methods help me understand why I am doing what I am doing? • Will this kind of evidence help me make the decisions I need to make?

  13. Components of An Assessment Plan • Mission • Objectives • Outcomes • Evaluation Methods • By Outcomes • Implementation of Assessment • Who is Responsible for What? • Timeline • Results • Decisions and Recommendations

  14. Clark College (Allied Health Program) • Category A • SAT • HS Transcript • Personal Interviews • Recommendations • Category B • California Study Test • Coop Eng. Test • Watson-Glaser Critical Thinking • Appraisal • Category C • Basic Information Sheet (Personal Information & Personal/Social Adjustment)

  15. Program Outcomes • Students in your program will develop a positive attitudes towards learning and increase their commitment and responsibility for their own learning • Pre-post gains or comparison with matched pair cohort on dimensions like academic skills, critical thinking, values toward learning, etc. • Develop a plan of study • Develop a learning portfolio • Learning environment preferences Survey (Perry Scheme)

  16. Reasons for Learning Style Assessments that Incorporate Diversity • Self-assessment feedback • Cohort comparisons • Cluster analysis of behaviors • Development of effective cooperative clusters • Matching of learning styles/teaching styles • Correlation with other dimensions • Identification of critical dimensions

  17. Meaningful Use of Datafrom Peggy Maki, Ph.D. • Collect data from different sources to make a meaningful point (for example, program samples and other samples of student work). • Collect data you believe will be useful to answering the important questions you have raised. • Collect data that will help you make decisions for continuous improvement. • Organize reports around issues, not solely data. • Interpret your data so that it informs program improvement, budgeting, planning, decision-making, or policies.

  18. Some Methods That Provide Indirect Evidencefrom Peggy Maki, Ph.D. • Alumni, Employer, Student Surveys • Focus groups • Exit Interviews with Graduates • Graduate Follow-up Studies • Percentage of students who go on to graduate school • Retention and Transfer Studies • Job Placement Statistics

  19. Indirect EvidenceCont. • Courses selected or elected by students • Faculty/Student ratios • Percentage of students who study abroad • Enrollment trends • Percentage of students who graduate within five-six years • Diversity of student body • CAS Standards

  20. Some Methods That Provide Direct Evidence • Student work samples • Collections of student work (e.g. Portfolios) • Capstone projects • Course-embedded assessment • Observations of student behavior • Internal juried review of student projects • Evaluations of performance

  21. Direct EvidenceCont.from Peggy Maki, Ph.D. • External juried review of student projects • Externally reviewed internship • Performance on a case study/problem • Performance on problem and analysis (Student explains how he or she solved a problem) • Performance on national licensure examinations • Locally developed tests • Standardized tests • Pre-and post-tests • Essay tests blind scored across units

  22. Building an Assessment Website • Easy to navigate • Definitions • Principles of Student Learning • Resources and Tool Kits • Projects and Portfolios • Presentations and Papers • Contact Information

  23. Community College Assessment Challenges • Academic programs • Administrative services • Student development programs • Integrate/connect proposed and ongoing assessment efforts across units and levels • Developing a culture of evidence • Emphasize link between MACRO-LENS and MICRO-LENS of assessment • Helping faculty see link between institutional effectiveness, aggregate data gathering, and outcomes–based assessment

  24. Community College Assessment Challenges • Questions pertaining to the creation and improvement of an “Effective Academic Learning Environment” cannot be answered by • Admissions/enrollment data • Persistence/attrition data • Survey data • Demographic comparisons

  25. Community College Assessment Challenges • Global teaching evaluations • Department comparisons • Teaching loads and credit hours • Entering first year student statistics • Transfer rates

  26. How Can We Support Faculty Work and Impact Academic and Student Development? • Lead the discussion that illuminates critical questions/concerns/issues • Provide initial sources of data and documentation directly to departments and programs • Keep the focus on measurable objectives and outcomes • Neutralize political concerns • Do all necessary programming • Facilitate the development and implementation of an assessment web site • “Champion” the development of assessment plans • Partner in assessment with your other providers of the service

More Related