360 likes | 384 Views
Assessment of Student Affairs Initiatives for First-Year Students. National Conference on First-Year Assessment October 12-14, 2008 San Antonio, Texas. Jennifer R. Keup Director, National Resource Center for The First-Year Experience & Students in Transition keupj@mailbox.sc.edu.
E N D
Assessment of Student Affairs Initiatives for First-Year Students National Conference on First-Year Assessment October 12-14, 2008 San Antonio, Texas Jennifer R. Keup Director, National Resource Center for The First-Year Experience & Students in Transition keupj@mailbox.sc.edu
Goals for Today • Clarify the relationship between FYE, Student Affairs & Assessment for you • Identify and discuss challenges to FYE assessment in Student Affairs • Review a Comprehensive Assessment Model • Discuss Assessment Outcomes • Assessment Audit • Assessment Approaches • Feedback Loops
What is Assessment? • “Assessment is any effort to gather, analyze, and interpret evidence which describes institutional, divisional, or agency effectiveness.” • Effectiveness includes: • Student learning outcomes • Clientele satisfaction • Compliance with professional standards • Comparisons with other institutions • Assessment guides good practice
Introductions and a Question • Introduce yourself: • Name • Institution • Venn Diagram Exercise • Identifying the relationships between FYE, student affairs, & assessment at your institution. • Where do you fit in this model on your campus?
Venn Diagram Exercise:Identify the Relationships Assessment Student Affairs The First-Year Experience You
Venn Diagram Example:Identify the Relationships Assessment Me The First-Year Experience Student Affairs
What is the assessment process? • Identify expected/desired outcomes. • Plan programs and services that accomplish outcomes. • Implement programs and services. • Assess the learning responsive to the outcomes through sound data collection & interpretation • Use results to improve learning opportunities.
Why is this so hard? Challenges of First-Year Assessment in Student Affairs
Barriers to Assessment • Lack of Commitment & Support from Leadership • Lack of Assessment Expertise • Lack of Resources • Fiscal: $$$ • Human: Professional Staff Support • Fear of Results
Let’s “Talk Turkey” What are some of the barriers that are unique to ASSESSMENT in Student Affairs? What are some of the barriers that are unique to ASSESSMENT of the First-Year Experience? What is really going on for you?
Purposes of FYE Assessment Gain Information About: • Backgrounds & characteristics of entering students • Entering students’ needs • Entering students’ satisfaction with orientation & FYE programs • Outcomes of orientation & FYE programs • Institutional climate for first-year students • Comparisons to national standards • Cost-effectiveness
How do I approach this? Model of Assessment for First-Year Initiatives in Student Affairs
Steps for Assessment • Define the problem • Determine the purpose of the study • Determine where to get the information needed • Determine the best assessment methods • Determine whom to study • Determine how the data will be collected
Steps for Assessment (cont.) • Determine what instruments will be used • Determine who should collect the data • Determine how the data will be analyzed • Determine the implications of the study • Report the results effectively
Define the problem2. Determine the purpose of the study Identifying meaningful and measurable outcomes of Student Affairs Initiatives for First-Year Students
Identifying Outcomes What are the questions we need data to answer? What do we want to know? How do these answers relate to FYE initiatives in Student Affairs?
Types of Outcomes • Program Outcome: • What your program is accomplishing? • What is the degree to which it is being accomplished? • Student Learning Outcomes: • What the students are learning as a result of programs & services? • What is the degree of learning attained? • Student Development: • What the students are learning that advances their abilities to perform effectively in a life or work situation?
Examples of Outcomes • Persistence • Analytical & critical thinking skills • Knowledge of university requirements • Interpersonal/Communication skills • Creation of a sense of community • Identity exploration and development • Develop educational career goals/Declare a major • Institutional climate
Examples of Outcomes (cont.) • Cohort/generation characteristics • Multicultural competence • Civic engagement/Develop as democratic citizens • Ability to identify, seek, & utilize organizational resources and student programs • Physical health & emotional wellness • Leadership skills • Moral and Ethical Development
Writing Outcomes • Be clear about what you are assessing • Program Outcome • Student Learning/Development Outcome • Make sure that your outcome is meaningful • Clearly state what you want students to know and be able to do • Incremental steps, stages, points of progress • Indicate the point that represents fulfillment/success
Outcomes Exercise • Pick an outcome of a program or service for First-Year Students on your campus that is managed by a Student Affairs unit or department. • Define the outcome in specific terms: • What are you assessing? • How is it meaningful? • What do you want students to be able to know and do? • Key question: Is this outcome measurable?
3. Determine where to get the information needed Conducting an audit of current assessment activities of Student Affairs Initiatives for First-Year Students
What to look for? • Are student data collected? • What is being collected? • By whom? • Where are they housed? • Are these data currently being utilized? • If so, how are the results communicated? • If not, why not? • What future data collection efforts are planned?
Potential Sources of Data • Admissions/Registrar’s Data • Institutional participation in national surveys of first-year students • Program/course evaluations • Utilization statistics of SA services/programs • Satisfaction surveys for SA services/programs
Potential Sources of Data (cont.) • Home-grown IR surveys • SA departmental surveys, focus groups, or student interviews • Accreditation reports • Grants • Strategic planning processes • Self-study processes/CAS • Other sources?
“Who do I know?” Exercise Write the name of a “responsible party” from your campus next to as many of the “Potential Sources of Data” items as possible.
Who do I talk to? • Exercise: • How many were you able to fill in? • Who were the key players? • Resources: • Student Affairs information, assessment & research specialist • FYE Director (or other equivalent) • IR Director • VC/VP for Student Affairs • Outside agencies
4. Determine the best assessment method6. Determine how the data will be collected Assessment methodologies
Methods to Collect Data • Quantitative • Describes and analyzes “what is” • Examples • Surveys • Analysis of existing data • Qualitative • Collect data about the meaning of events & activities to the people involved • Examples • Interviews • Focus Groups • Observations
Methods to Assess Outcomes • Indirect • Self-reported comments that reveal a perceived increase in understanding or appreciation • The perception is usually not verified through any demonstration of knowledge acquisition or observation • Direct • Demonstration of abilities, information, knowledge, etc. as the result of participation in a program or utilization of a service
Indirect Assessment Methods • Examples • Satisfaction measures • Program evaluations • Self-ratings of skills • Self-assessment of change • Agreement with statements • Inventories • Informal peer-to-peer conversations • Pro/Con of indirect assessment methods
Direct Assessment Methods • Examples • Portfolios • Capstones • Performances • Common assignments • Exams • Observations of behavior/performance • Standardized tests • Work samples • Pro/Cons of direct assessment methods • Making indirect methods direct
10. Determine the implications of the study11. Report the results effectively Closing the loop
Implications • Grounded in Steps 1 & 2: Purpose & Problem • Have you answered the original assessment question? • What have we learned about our outcome of interest?
Effective Reporting: Audience • Who is your client? • Use it as an opportunity to connect with your list of “responsible parties” for data. • Who should know about this information? • If the information can inform practice, whose practice will it inform? • Know the climate and context of your university • Venn Diagram • List of Challenges
Effective Reporting: Structure & Dissemination • Reporting formats: • Paper • Web • Presentation • Executive summary • Visual identity • Leverage information channels that already exist • Responsible parties for data • Student Affairs Program Review