"Outcomes Assessment” Developing Meaningful Tools to Assess Learning Outcomes

1. "Outcomes Assessment” Developing MeaningfulTools to Assess Learning Outcomes"Outcomes Assessment” Developing MeaningfulTools to Assess Learning Outcomes An “overview” of the process to develop and implement a meaningfulassessment of your program’s major learning outcomes. Presented by: JulieAnne Adamich Ph. D., CPA St. Petersburg College E-mail address: adamichj@spcollege.edu

2. “Improvements in learning will depend on how well assessment, curriculum, and instruction are aligned and reinforce a common set of learning goals” Pellegrino, et al (2001,p. 25) It also is a way of ensuring that students have been exposedto and have “acquired” the major learning outcomes …. essential for successful job placement. Assessment accomplishes two important goals

3. Assessment is an integral part of the Planning and EvaluationProcess Formative Goals: Continuous Improvement Both Formative and Summative Goals: Focus: Allocating Resources Summative Goals: Accountability, Performance Evaluation

4. Define the purpose of the Program Assessment Process • It is a Formative evaluation process • based on theContinuous collection and analysis of data • with the primary purpose of emphasizing Improvement of: - student learning - student development - the program within the institution

5. Another reason-“Accreditation”Requirements for SACS Reaffirmation • “The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that incorporate a systematic review of programs and services that (a) results in continuing improvement and (b) demonstrates that the institution is effectively accomplishing its mission.” (Core Requirement 5) • “The institution identifies college-level competencies within the general education core and provides evidence that graduates have attained those competencies. (3.5.1 Gen Ed Competencies)

6. Overview of the Program Assessment Process at St. Petersburg College Establish Program/Course Goals Translate Goals into Major Learning Outcomes Use Results for Decision-making, i.e., program/course improvement Design & Implement Educational Environment, i.e., program of study Evaluate Assessment Findings Design & Conduct Assessments, i.e., measure major learning outcomes

7. Step 1: Define program’s major learning outcomes. • Select those outcomes that are “absolutely essential” for academic success. • While the outcomes should be comprehensive, it is not feasible to assess every learning outcome within a program/course. • Each outcome should be defined with the intention of connecting the outcome with a specific method for assessment.

8. Step 2: Develop/select an assessment plan or methodology to assess the major learning outcomes. • Identify a plan that will provide you meaningful results, including how, when, and where to conduct the assessment; collect the data; analyze & use the results. • Select one or more suitable assessment tools , e.g., national exams, faculty-developed multiple choice exams, projects, portfolios etc.

9. Assessment Specifications Plan • Assessment Purpose: Describe briefly the purpose of this assessment. For example, “To make summative interpretations for program improvement. This instrument will be used to assess the achievement by majoring learning outcome of students enrolled in the (identify)program.” • Population:Describe who will be assessed (i.e., the course, which semester(s), etc.). Clarify whether or not all student data collected will be used for the assessment. For example, “only data for students enrolled in (identify the A.S. program) who have completed 46 credits will be used for the assessment of this program.” • Domain Specification: Students will: List each major learning outcome to be assessed. (Note: If deemed useful, also identify further the specific performance indicators and corresponding cognitive levels).

10. Assessment Specifications Plan(Continued) • Assessment Method: • Identify method: (1) type of instrument (i.e., multiple-choice examination, project, portfolio, etc.); (2) where it was developed/validated (i.e., national exam or locally developed instrument). • Weights Desired:Describe how many points each question is worth. • Scoring Method:Describe how the instrument will be graded (i.e., the type of rubric – analytic or holistic, or with an answer key). • Who will score the assessment:Identify the evaluators and their credentials, or the forms/equipment that will be used to score the assessment. • Assessment Conditions:Describe briefly the methodology for conducting the assessment. For example, “Students enrolled in (identify) course, will complete a closed book multiple-choice examination during their twelfth week of instruction. Calculators are permitted. Estimated length of the assessment is approximately 90 minutes. • Attach: 1) Copy of the instrument 2) Copy of the evaluation rubric 3) For alternative assessments, the template format to be used to tabulate the evaluation results (i.e., a copy of the Excel spreadsheet template).

12. Step 3: Based on your assessment plan, coordinate/implement the assessment. • How = Select a tool, e.g., multiple choice exam, project, portfolio, etc. • When/Where = in a capstone course, as a graduation requirement, etc. • Collect assessment data = quantify the evaluation results at a level of detail that provides meaningfulfeedback to support each learning outcome.

13. Step 4: Summarize the assessment results for each major learning outcome. • Multiple Choice Exams: • Scantron forms using Parscore software to document results (descriptive statistics and item analysis) • WebCT or Angel (descriptive statistics and item analysis) • Projects/Portfolios - Rubric with the option of web-based input • EXCEL file detail by student (rows) and quantified evaluation of each learning outcome (columns) • Summarize each column providing descriptive statistics (i.e., mean, standard deviation, frequency distribution, histogram, etc.)

14. Step 5: Analyze assessment results and prepare an assessment report Analyze each major learning outcome, based on the statistical results. Include the following: • Criteria for Success, i.e., expected statistical results • Summary of Assessment Findings, i.e., actual statistical results • Use of Results, i.e., analyze results to determine if planned program improvements are deemed necessary • Action Plan and Timetable for Implementation • Budgetary and Planning Implications

15. Step 6: Implement the action plan and prepare a follow-up report. • Program/Course assessment provides a systematic plan for generating program improvement. Therefore, the analysis/evaluation of assessment results should generate possibilities for curricular improvement, e.g., revision of course objectives, etc. • A “Follow-up” Report should be submitted during the following fiscal year reflecting the status of each action plan item.