Assessment Day May 2017

1. Assessment Day May 2017 General Education Science Assessment Leadership Team Victor Bondzie, Timothy Barnett, and Rita Luther

3. Graduation Rates (%) 6.2% 10.7% 9.2% 3.4%

4. Five Year Graduation Rate By Success in First Five Courses Graduation Rates (%) 8.9% 7.7% 4.2% 0.8%

5. Success Rates by Gender/Ethnicity All Fall 2016 Courses – College-Wide Gender Ethnicity

6. From Achieving the Dream to the Academic Initiative Review at Valencia College Four At-Risk Student Populations Identified: • Students not successful in any of their first five courses or do not enroll twice in their first four terms are not as likely to graduate. • New students attending college for the first time (FTIC) • “Student success is largely determined by a student’s experience during his or her first-year in college” (Upcraft, et al. 2005) • Students taking math and identified as in need of support (developmental) • 2015 Strategic Report found that 46% of student success in developmental math from Fall 2011 to Summer 2013 semesters. • Students who are among those that show declining rates of persistence, such as African American males.

7. Fall 2016 Survey of At-Risk Students Indicated: • Some students are using and trying to follow advice given by support services. • Students want more feedback and in some cases more direction from faculty. • They want to learn strategies for engaging faculty, learning support and advising at Valencia College. Developmental Education Research 2016-17 English courses: Survey of students (n=1,387 13% response rate), 1 focus group (n=7), and 7 student interviews Math courses: Survey of students (n=1,226 12% response rate), 3 focus groups (n=23), and 5 student interviews Nichole Jackson, Assistant Director of Learning Assessment, Valencia Institutional Assessment

8. Students who are at risk tell us they need: • Support transitioning from high school to college • Strategies making the most of services available • Preparation to achieve in courses higher than their current ability level • Encouragement, incentives, or requirements in order to seek help • A connection to support services: • that will deepen their knowledge • be available for them to use repeatedly and regularly in person and online

9. Agenda Items • Review Learning Outcomes • Review 2017 Spring Data • Discuss 2016-17 Outcomes Resulting From Data • Discuss Future Directions of the Science Gen. Ed. Work

10. REVIEW OF LEARNING OUTCOMES • Outcome #1: Critical Thinking-Effectively analyze, evaluate, synthesize, and apply information and ideas from diverse sources and disciplines. • Faculty developed (2014-2015) and employed Biology, Chemistry, and Physics scenarios to address Observation, Hypothesis, Data and Conclusions. • Outcome #2: Quantitative and Scientific Reasoning- Students should be able to use processes, procedures, data, or evidence to solve problems and make effective decisions - Not yet attempted.

11. REVIEW OF 2017 SPRING DATA

12. Current Expectations • The students should be able to apply the scientific method. • Expectations assessed through student analysis of the scientific method in the context of a specific scientific problem. • We expect that at least 3000 student participants. • We also expect that 70% or more of our students will earn a grade of 75% or higher.

13. Summary of Spring 2017 Data Overall Responses Minimum One Question Completed Taking ≥ 4 Minutes

14. Science Gen Ed Assessment “Passing” Rates % Students Scoring ≥ 75% Biology Assessment Chemistry Assessment Physics Assessment

15. Distribution of Scores

16. Percent Correct Per Question % Students

17. Participation By Campus

18. Faculty Participation 33% 30% 20% 16%

21. QUESTION BREAKDOWN

22. Question #1: Which of the following observations support conditions for the existence of water? ObservationsQuestion #2: A scientist graphed the event of a single temporary drop in the brightness of a star that was observed during an observation period of 1,500 days this way: The scientist created 4 additional graphs to show the timing of brightness changes in four other stars that were also observed for 1,500 days each. If each planet is described by a distinct drop, which of the following graphs would be most consistent with the presence of two planets circling a star, rather than just 1 planet? Hypothesis/Prediction/Data AnalysisQuestion #3: Which of the following statements is true from the experimental data obtained by the Kepler team? Data AnalysisQuestion #4: Which of the following statements can reasonably be concluded?Conclusions

23. Question #1: Which of the following observations support conditions for the existence of water? Fall 2016 Spring 2017

24. Question #2: A scientist graphed the event of a single temporary drop in the brightness of a star that was observed during an observation period of 1,500 days this way:The scientist created 4 additional graphs to show the timing of brightness changes in four other stars that were also observed for 1,500 days each. If each planet is described by a distinct drop, which of the following graphs would be most consistent with the presence of two planets circling a star, rather than just 1 planet? Fall 2016 Spring 2017

25. Question #3: Which of the following statements is true from the experimental data obtained by the Kepler team? Fall 2016 Spring 2017

26. Question #4- Which of the following statements can reasonably be concluded? Fall 2016 Spring 2017

27. Spring 2017: Question #1 Disaggregated Data By Ethnicity and Gender # Responses (%) Females # Responses (%) Males

28. Spring 2017: Question #2 Disaggregated Data By Ethnicity and Gender # Responses (%) Females # Responses (%) Males

29. Spring 2017: Question #3 Disaggregated Data By Ethnicity and Gender # Responses (%) Females # Responses (%) Males

30. Spring 2017: Question #4 Disaggregated Data By Ethnicity and Gender # Responses (%) Females # Responses (%) Males

31. Fall 2016: Question #1-4 Disaggregated Data By Grade (% Percent Correct) Spring 2017 Fall 2016

32. STUDENT SCORING BY SUNDRY VARIABLES

33. Student Scoring by Sundry Variables

34. Student Scoring by Sundry Variables Scores by Number of Science Courses Completed Prior to Fall 2016

35. Student Scoring by Sundry Variables Scores by Discipline

36. SUMMARY/DISCUSSION OF SPRING 2017 DATA • Student responses and distribution of answers overall were similar between Fall 2016 and Spring 2017 Sci Gen Ed Assessments • Data indicated a consistent discord between ethnic groups within males and female student populations for correct answer selections in the Sci Gen Ed Assessment with african american students showing the lowest percentage of students selecting correct answers. This was followed by hispanic students and finally caucasian students showing the highest percentage of students selecting correct answers. • Data also indicated gaps between male and female students in question performance for african american and hispanic female students. Caucasian female students had a higher percentage of students selecting correct answers among females and in some cases overlapping with the performance of some male populations.

37. SUMMARY/DISCUSSION OF SPRING 2017 DATA • Trends shown in proportion of student with correct answers based on course grades • Sundry variables: • Comparable frequencies of students completed the tutorial between Fall 2016 and Spring 2017 • Mean scores of those completing the tutorial are slightly higher, but not a striking difference (too early) • Slightly higher mean scores for students that completed more science courses, but not a striking difference. • Physics, chemistry, biology, earth science, horticulture (Score performance differences from higher to lower)

38. FUTURE DIRECTIONS OF SCI GEN ED WORK • How best do we assist at-risk students? (Mentoring, tutoring, NSE, other ideas) • Where do we go from here… • The fall and spring data is almost identical. Can we run one assessment each fall and know that it will be representative of students overall? (This would give you much more time to move from results in fall to action plan development in spring towards assessment day). • Should we run biology assessment (edited) in Fall 2017? • How do we get full adjunct faculty buy-in to assessment to increase student participation? • We need new volunteers to serve on the Assessment team with Rita for two years • Further discussions on assessment day.

39. Thank you!