Assessing the Academic Climate

1. Assessing the Academic Climate AIRPO Annual Conference Stony Brook, New York June 19-21, 2006 Bruce Szelest & Jeff Gerken bszelest@uamail.albany.edu jgerken@uamail.albany.edu

2. The Albany Outcomes Assessment Model Personal Traits Alumni Outcomes Pre-College Characteristics Educational Outcomes Graduate Degrees Earned Academic Age, Ethnicity, & Gender College Experiences Arts & Letters Disciplinary Study Scientific Method Academic Achievement Persistence/Graduation Acad Integration Occupational Status Parents’ Education Acad Conscientiousness Classroom Experiences Faculty Contact Field of Study Aptitude Test Scores Income Level High School Achievement Satisfaction with Job and Career Personal Social Integration College Preparation Peer Relations Employment Extra-Curr. Activities Residential Experience Interpersonal Skills Openness & Tolerance Functioning Independently Leadership & Service Parental Influence Awards & Recognition Involvement Receptivity to Services Educational Aspirations Inst’l Integration Alumni Giving Inst’l Commitment Advisement Financial Aid Affinity of Values Acad Motivation: Intrinsic, Extrinsic Self-regulation

3. The Albany Longitudinal Survey Program Year 1 Year 2 Year 3 Year 4+ Entering Student Senior Experience Survey Survey Freshmen Student Experience Survey

4. Entering Student Survey • Focus on student pre-college characteristics to identify students at risk prior to first semester of study • Concentrate institutional resources on students in-need of attention - individual student profiles • Communicate general findings back to new freshmen to raise awareness of successful student profiles • One leg of the Advisement Services Center’s three-legged assessment program • http://web.albany.edu/ir/samplesurveys/ess.html

5. Pre-college Scales Parental Influence Composite of mother's and father's education My parents encourage me to do the best in whatever I do I can go to my parents when I have a problem Educational Aspirations (scale: importance) To gain a broad, well-rounded education (in the sciences, humanities, and arts) To learn to think creatively and analytically To learn more about myself, my values, and my life's goals To learn more about other cultures and groups of people. Involvement (scale: degree of chance) Participate in intercollegiate or intramural sports Belong to a fraternity or sorority Join a student club (academic, political, or cultural) Participate in student government, student newspaper, and/or campus radio station Receptivity to Services (scale: degree of chance) Visit professors during their office hours Attend a study group from one of your classes Contact my academic advisor more than once per semester Visit the Career Development Center College Preparation (scale: yes or no) Pre-calculus or calculs Foreign language Science Advanced placement (AP) courses

6. Pre-college Scales – cont’d Academic Motivation - Intrinsic (scale: not true to true) I enjoy the classes that challenge my mind. I will feel successful if I have the highest test scores. Thinking about my future motivates me to work hard in school. Solving difficult problems in my classes gives me a sense of satisfaction. I feel successful when I learn something interesting. Learning new things gives me a sense of accomplishment. Schoolwork is interesting because you see yourself making progress. I read books not assigned in classes. Academic Motivation - Extrinsic (scale: not true to true) I study hard because I want to prove myself as capable as anyone else in class. I worry about not doing as well academically as others in college. I don't feel I can gain much in college other than obtaining a degree. I feel that as long as my teachers are satisfied with my academic performance, that is enough. I don't really care how well I will do in college, as long as I can pass. Getting a high paying job is the main reason I study hard in college. I feel ashamed if I don't do as well as other students do in college. I worry about failing exams and getting into academic trouble. Self-Regulation (scale: not true to true) When I get something wrong in school, I always stop and try to find out what went wrong rather than simply asking my instructor the correct answer. I take notes while reading textbooks. When I run into difficulties in schoolwork, I feel if I put in extra effort, I will eventually figure them out. When I am reading my textbooks, I stop and check whether I really understand what I have just read. …cont’d

7. Student Experience Survey (frosh & seniors) • Examine changes in students’ psychological make-up after the first college year • Gauge institutional engagement and relationships with educational outcomes • First pass at gauging centralized student advisement • Communicate general findings back to students and faculty to support the development of a campus culture of assessment • http://web.albany.edu/ir/samplesurveys/froshses.html

8. College Experience Scales & Items Academic Integration: Academic Conscientiousness (scale: various 5 pt scales) How frequently have you worked harder than you thought you could to meet an instructor's expectations? Developed your study skills. Utilized your study skills. I do very little studying on weekends. I put a good deal of effort into being well prepared for examinations. My primary goal at UAlbany is to take advantage of academic opportunities. Doing homework and studying outside of class. Classroom Experiences (scale: never to almost always) Been satisfied with your academic experiences? Had out-of-class assignments that were good learning experiences? Worked with other students on class assignments? Received feedback (written or oral) from instructors on the quality of your work? Had to critically appraise the value of information, arguments, or methods in your classes or class assignments? Faculty Contact Items How frequently have you had discussions, meetings, or conversations with instructors outside of class? Visit professors during their office hours? Seek out a faculty mentor?

9. College Experience Scales & Items– cont’d Social Integration: Peer Relations (scale: disagreement to agreement) I have developed strong friendships with other students. My interpersonal relationships with other students have had a positive influence on my personal growth. The student friendships I have developed have been personally satisfying. It has been difficult for me to make friends with other students. I know several UAlbany students who would be willing to help me if I had a personal problem. My interpersonal relationships with other students have had a positive influence on my intellectual growth. I am confident in my ability to make friendships. Extra-curricular activities (scale: yes or no) Participate in intercollegiate or intramural sports?. Belong to a fraternity or sorority? Employment Working on-campus (volunteer or paid). Working off-campus (volunteer or paid).

10. College Experience Scales & Items– cont’d Institutional Integration: Institutional Commitment (scale: yes or no) Visit the Writing Center? Visit the Career Development Center? Contact your academic advisor more than once per semester? Advisement Items (scale: disagreement to agreement) Advisor was knowledgeable about course requirements for the major(s). Advisor was knowledgeable about course requirements for the minor(s). Advisor was helpful in assisting with scheduling/ registration procedures. Advisor showed genuine interest in my academic progress. Advisor went over my degree audit with me to inform me of course requirements I have not yet met. Advisor was knowledgeable about academic policies (e.g., graduation requirements, GPA requirements, General Education guidelines, residence requirements, etc.). Advisor was available when I needed him/her. Advisor showed concern for my personal growth and development. Advisor helped me to identify career areas (or graduate school opportunities) which fit my skills, abilities, and interests Advisor encouraged me to talk about myself and my college experience. Advisor was easy to talk with. Advisor promptly returned my phone calls, e-mails, and/or messages.

11. Educational Outcomes Academic (scales: none to very large college contribution) Arts & Letters Writing effectively. Developing intellectual curiosity. Developing the ability to formulate creative ideas and/or solutions. Appreciating artistic and creative expression. Placing current problems in historical perspective. Scientific Method Evaluating ideas, materials, and methods critically. Developing problem solving skills. Thinking analytically and logically. Understanding scientific findings. Understanding mathematical concepts. Disciplinary Study Gaining factual knowledge. Learning how to learn. Synthesizing a body of information. Understanding a particular discipline's research methods. Understanding a particular discipline's various schools of thought. Understanding the inter-relatedness of different fields of study. Using information and/or computer technology in your academic discipline

12. Educational Outcomes – cont’d Social (scales: none to very large college contribution) Interpersonal Skills Speaking effectively. Developing interpersonal and social skills. Functioning effectively as a member of a team. Preparing for active participation in a democratic society. Coping with conflict. Adapting to different social situations. Openness & Tolerance Understanding cultural differences. Gaining exposure to a variety of new intellectual areas. Developing an openness to new ideas. Coping with moral and ethical issues. Developing a better understanding of myself (e.g., interests, talents, values, limitations). Understanding cultural differences. Relating well to people of different races Functioning Independently Functioning independently (self-reliance). Exercising personal responsibility. Developing a better understanding of myself.

13. Entering Student Survey ID at-risk students General profiles of successful students Personal Traits Alumni Outcomes Pre-College Characteristics Educational Outcomes Graduate Degrees Earned Academic Age, Ethnicity, & Gender College Experiences Arts & Letters Disciplinary Study Scientific Method Academic Achievement Persistence/Graduation Acad Integration Occupational Status Parents’ Education Acad Conscientiousness Classroom Experiences Faculty Contact Field of Study Aptitude Test Scores Income Level High School Achievement Satisfaction with Job and Career Personal Social Integration College Preparation Peer Relations Employment Extra-Curr. Activities Residential Experience Interpersonal Skills Openness & Tolerance Functioning Independently Leadership & Service Parental Influence Awards & Recognition Involvement Receptivity to Services Educational Aspirations Inst’l Integration Alumni Giving Inst’l Commitment Advisement Financial Aid Affinity of Values Acad Motivation: Intrinsic, Extrinsic Self-regulation NSSE Freshmen & Senior Surveys Expansion of the Albany Model Gauge institutional engagement Evaluate advisement (centralized & in the major) Build culture of assessment (generally & in acad units)

14. The IR Survey Program in Action Communicating Results and Next Steps

22. Strategies for Affecting the Course Experiences Dimension • Earth & Atmospheric Sciences - “treat them right” • Interaction space - Weather center; Map room • Develop sense of community with other students • Interact with graduate students • Faculty in and out all day • Take undergraduates to conferences • Undergraduate leadership conference • Pre-graduation dinner and roast for graduates, families, and professors • Faculty available regardless of office hours, everyday • Make students feel part of something more intimate than the greater University

23. Strategies for Affecting the Course Experiences Dimension • Public Policy – Attention to the curriculum • Cluster of faculty review the intentionality of the curriculum • Senior seminar (capstone course) • FT faculty involvement • A lot of one-on-one contact with faculty • Research paper • Writing intensive, and oral discourse components • Moved away from disproportionate instruction by doctoral students • Careful mentoring of doctoral student instructors • 1st teach with a full-time faculty member • Review performance • Make the final experiences for students a positive one

24. Strategies for Affecting the Course Experiences Dimension • English – Personalizing a large major • Smaller classes at the Upper Division • Juniors and seniors get the most experienced instructors in their areas of interest • Undergraduate research conference each Spring • Interaction with FT faculty • Guest speaker • Active honors program • Very active chapter in National Honor Society

25. Strategies for Affecting the Course Experiences Dimension • Business Administration – Focus on the pragmatic • Meaningful project experiences - a lot of group work • Focus on presenting student work • Opportunities for collaboration • Consulting reports • Research projects • Active use of SIRF (by dean and chairs) • Work with faculty who can benefit from further development • Master teachers program • Capstone course – wrap it all up for students

29. Successful Approaches to Advisement in the Major • Earth & Atmospheric Sciences – Advise beyond the norm • Make effort to sit down with every student (even intended majors) • High involvement of FT faculty • Divide up advising among professors • Training sessions (informal) for professors • FT faculty, regardless of office hours, available everyday

30. Successful Approaches to Advisement in the Major • Biological Sciences – Lot’sa Majors, personal attention • Faculty advisor for every student (25-30 students/FT faculty) • Junior faculty have fewer advisees than senior faculty • Coordinator for undergraduate advisement • Gate keeper, assign students to advisors, approve advisor changes • Handle course substitutions, waivers, dropping issues, etc. • Transfer advisement team (faculty and staff) • Trained to interpret degree audit, PS functionality • Registration and scheduling help • Bring issues of import to students to faculty attention • Availability of written information and policies • FAQs in central locations • Advisement info on department web site • PS Data Warehouse to get student e-mail addresses

31. Successful Approaches to Advisement in the Major • History – Hybrid approach • Historically: • Very decentralized approach, individual faculty kept advisee files • Now: • Centralized file locations • Keep notes on student intentions, advisor comments, etc. • Print degree audits on-site • Students make appointments with FT faculty in centralized location • FT faculty take turns handling walk-in advisee needs (advise & forms available) • Recommend each major select an individual FT faculty advisor (promote mentoring relationships) • Advisement on the www, FAQ • Survey of majors, get info on student concerns, likes/dislikes, etc.