Assessment of the Core – Science

1. Assessment of the Core – Science Charlyne L. Walker Director of Educational Research and Evaluation, Arts and Sciences

2. Assessing the Major • SACS requires us to assess all degree programs bachelors through doctorate as part of Institutional Effectiveness • FIU has been assessing our degree programs since 1999-2000 • Recent changes in SACS has moved us toward more direct measures • Departments assess programs in a variety of ways • Sample IE document

3. Science Core Category Description • Our technologically dependent world requires an understanding of the processes that led us here. • Learning the basic concepts and ideas of scientific fields provides contact with not just those fields but with how science is done.

4. Science Core Category Description • In these courses students study the scientific method through examination of the foundational theories of modern scientific thought. • Students apply scientific principles and theories to problem solving, evaluate scientific statements, and incorporate new information within the context of what is already known. • Emphasizing the essential connection between theory and experiment, the hands-on laboratory experience provides the context for testing scientific theories. • Students will be able to describe the scientific method through examination of the foundational theories of modern scientific thought.

5. Foundations of Science • Students will be able to • Apply scientific principles and theories to problem solving, • Evaluate scientific statements, • Interpret new information (e.g. scientific data, journal articles, media reports of scientific findings, etc.) within the context of what is already known, and • Apply the scientific method by conducting experiments to test scientific hypotheses.

6. Objectives and Measures • There should be objectives for the courses in course syllabi • As objectives are developed, we may be asking for you to include them into course syllabi • As measures are developed, you might be asked to include them into course syllabi to let students know what measures are being used • This is not a “short term” solution for the 2010 SACS visit • The measures must be on going as part of SACS requirements beyond 2010

7. Types of Direct Measures • Embedded questions • Set of locally developed questions intended to measure specific student learning outcomes • Placed within tests of all sections of the same course • Used to track cognitive or skill development through a program, sets of embedded questions, with each expecting a higher level of proficiency than the previous, may be used across sequential courses • Papers or short answer questions using rubrics – a rubric is used to standardize the grading of work

8. Types of Direct Measures • Locally developed exams - exams produced by faculty within a discipline • Pre/Post Test - a locally developed test that measure students’ incoming and post program levels of knowledge, skills, behaviors and attitudes to measure students’ gains • You can find more types of measures at http://oeas.ucf.edu/alc/dir_measures.htm

9. Examples of Objectives and Measures from UCF • PSC 1121 Physical Science - Demonstrate an understanding of the scientific method. In particular, to demonstrate an ability to assess which claims presented as "scientific" are indeed scientific and not pseudoscientific. • Measures: • 7.1 PSC 1121 Physical Science - A pre-test and a post-test will be given. A gain of 30 % will be considered satisfactory, a gain of 50 % will be considered well above satisfactory, and a gain of 65 % or more will be considered outstanding. The gain is defined as the net change in the average score divided by the average of the pre-test. If full tests cannot be given, few questions will be sufficient. • 7.2 PSC 1121 Physical Science - In each exam, questions will be embedded to enable an assessment of the extent to which this objective is met. The questions may be multiple choice, long answer, or quantitative problems. An overall score of 70% on these questions will be considered acceptable.

10. Examples of Objectives and Measures from UCF • GLY 1030 Geology and its Applications - To demonstrate an understanding of multifaceted scientific terminology and an ability to apply the definitions involved.  • Measures: • 19.1 GLY 1030 Geology and its Applications - A set of 8 tightly-written definitions, several involving at least 3 characteristics, will be presented and illustrated in class, in conjunction with graded homework assignments. A 7-question quiz (or quiz section) will follow, testing knowledge of precisely what is included in each definition. Each quiz question will require more than one answer; and no partial credit will be given, in order to stress the importance of learning the entire definition for application later in the course. Because any error or omission can render an answer incorrect, a score of at least 4 out of 7 on these questions by 100% of students who earn a "C" or higher in the course will be considered acceptable. A similar, but smaller, set of questions will be included on a pretest in Spring 2007 to establish a baseline for use in subsequent years.

11. Examples of Objectives and Measures from UCF • GLY 1030 Geology and its Applications - To demonstrate an understanding of multifaceted scientific terminology and an ability to apply the definitions involved.  • Measures:  • 19.2 GLY 1030 Geology and its Applications - A subsequent test will include an embedded set of 7 questions that require simultaneous application of all the characteristics of each definition from measure 19a that is tested. For example, determining the number of minerals in a collection described in the question-set requires consideration of whether to count: types or pieces, natural and/or synthetic materials, and crystalline and/or noncrystalline materials. In addition, knowledge of the basic mineral compositions of the igneous rocks is needed. Again, no partial credit will be given, in order to promote the learning of knowledge as a cohesive whole, rather than as a set of disconnected fragments. Because any error or omission can render an answer incorrect, a score of at least 4 out of 7 on these questions by 100% of students who earn a "C" or higher in the course will be considered acceptable. A similar, but smaller, set of questions will be included on a pretest in Spring 2007 to establish a baseline for use in subsequent years.

12. Examples • UCF General Education Science Foundation http://iaaweb.ucf.edu/assessment/visitor/view_2006_07.asp?program_ID=1240 • UCF Assessment Plans for 2006 http://iaaweb.ucf.edu/assessment/visitor/y67.asp?group=visitor