Assessment at California Baptist University

1. Assessment at California Baptist University • The goal of assessment is the continual improvement of the learning environment.  Student Outcomes Assessment Cycle • Components of the Cycle • Defining • Collecting • Analyzing • Reporting • Improving Assessment at CBU

2. Course / Program & University Student Out Matrix I = Introduced; P = Practiced; D = Demonstrated Results are analyzed & will help identify strengths and weaknesses in curriculum, pedagogy, methods, etc. ANALYZING -REPORTING Schools and Departments develop PSOs for each major or program with “critical assignments” and evaluation rubrics DEFINING Schools and Departments complete the CPUSO matrices to link USOs & PSOs with particular courses in which specific critical assignments will be evaluated DEFINING Professors provide guidance in syllabi/rubrics and direct critical assignments based on School/Dept planning DEFINING Students produce “critical assignments” in noted courses using pictured rubrics as guide as well as suggested formats, etc. Assignments are shared with professors for evaluation COLLECTING Profs evaluate student’s critical assignments using rubrics, etc. In LiveText, the rubric results are gathered for ALL that are evaluated using that rubric ANALYZING Schools and Departments develop plans for curricular improvement. IMPROVING Plans for improvement are implemented and assessment continues. External constituencies will be canvassed to validate outcomes & PSO listings. IMPROVING Annual Student Outcomes Assessment Process, page 2 Final Reports explain analyses from the standpoint of intended improvements Initial Report ID’s “AYs PSO Focus” Collected data is arranged in various charts for analysis Annual Recap Reports and FEEE display process progress and intended adjustments LiveText Rubrics serve to provide for formative assessment for students and summative assessment for critical assignmnts/PSOs Goal: Continuous Improvement of the Learning Environment Assessment at CBU asmtz/asoprocess.ppt

3. Schools and Departments develop PSOs for each major or program with “critical assignments” and evaluation rubrics DEFINING Course / Program & University Student Out Matrix I = Introduced; P = Practiced; D = Demonstrated Schools and Departments complete the CPUSO matrices to link USOs & PSOs with particular courses in which specific critical assignments will be evaluated DEFINING Students produce “critical assignments” in noted courses using pictured rubrics as guide as well as suggested formats, etc. Assignments are shared with professors for evaluation Profs evaluate student’s critical assignments using rubrics, etc. In LiveText, the rubric results are gathered for ALL that are evaluated using that rubric ANALYZING Assessment Cycle Data is analyzed and compiled in LiveText Schools and Departments analyze results, identify strengths and weaknesses in curriculum, pedagogy, methods, etc. for focus in INITIAL ANNUAL REPORT Professors provide guidance in syllabi/rubrics and direct critical assignments based on School/Dept planning DEFINING Students submit assignments via LiveText LiveText Rubrics serve to provide for formative assessment for students and summative assessment for critical assignments/PSOs Assessment at CBU

4. ANALYZING -REPORTING Schools and Departments develop plans for curricular improvement. IMPROVING Continuous Program Improvement Initial Report ID’s “AYs PSO Focus” Submitted in LiveText Annual LiveText Recap Reports and FEEE display process progress and intended adjustments Plans for improvement are implemented and assessment continues. External constituencies will be canvassed to validate outcomes & PSO listings. Final LiveText Reports explain analyses from the standpoint of intended improvements 4 Assessment at CBU

5. DEFINING Fall 2008 Fall 2007 CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING RESULTS ACTION TAKEN Student Outcomes Assessment Cycle USO 3 Critical Thinking and Literacy Skills Faculty Workshop Theme Writing Across the Curriculum Faculty across campus report decline in student written and oral skills when expressing ideas Faculty implement plans and report results and plans for follow-up at Fall Assessment Colloquium. Identify points at which writing and expressing ideas are introduced, practiced, evaluated Academic units develop a plan for enhancing writing instruction and assignments requiring critical thinking. Assessment at CBU

6. USOs, PSOs and Discipline Rubric Relationships University Student Outcomes (6) Discipline X PSOs w/PSO Rubric Discipline Y PSOs w/PSO Rubric Discipline Z PSOs w/PSO Rubric Course X1 w/ Disp Rubric incl PSO Course Y1 w/ Disp Rubric incl PSO Course Z1 w/ Disp Rubric incl PSO Course X2 w/ Disp Rubric incl PSO Course Y2 w/ Disp Rubric incl PSO Course Z2 w/ Disp Rubric incl PSO Course X3 w/ Disp Rubric incl PSO Course Y3 w/ Disp Rubric incl PSO Course Z3 w/ Disp Rubric incl PSO • “Disciplines” represent the Schools, Departments and their individual Majors or Progams. Individual Majors or Programs have distinct PSO sets that Identify a graduate in that discipline. The PSO Rubric is developed to evaluate the critical elements of any student’s “outcomes” with the ultimate goal of determining efficacy of the program. • The subordinate courses use the same rubric, but can have distinctive elements of the discipline evaluated as a subordinate component of the rubric by using “milestones.” Milestones can be time-related such as entry level, mid-range and capstone. Other milestone options could be differing components of a single discipline, etc. • Using these LiveText rubrics and the milestone characteristic allows the various disciplines to develop a variety of reporting levels that support the course, PSO, discipline and university evaluation of student outcomes. Assessment at CBU

7. Step 1, Defining • In this stage the development of the Program Student Outcomes (PSO) occur. • In one major or program, the list of PSOs DEFINE what knowledge, skills, abilities and behaviors a graduate in that discipline should possess. • This stage also is where the Course and Program/University Student Outcomes matrix is developed (and reviewed and revised) Assessment at CBU

8. Step 2, Collecting • Stage one merges into this Collecting stage as the courses and critical assignments are identified for data collection planning • In each annual cycle approximately one-fifth of the PSOs are identified for evaluation • The specific means of assessment are decided upon and plans are put in place for collecting the data Assessment at CBU

9. Step 3, Analyzing • Stage two merges into this Analysis stage as the noted means of assessment occur and data is collected. • Faculty and assessment coordinators evaluate the collected data, develop means for analysis and conduct the analysis • Results of the analyses are prepared for reporting. Assessment at CBU

10. Step 4, Reporting • Stage two merges into this Reporting stage as the data and analyses are placed in the Annual Final Report and Recap Report • A VERY key step in this stage is arriving at the “so what” of the analysis in terms of what changes will be made to improve • In addition to the two reports noted above, each unit will also complete a Self-Evaluation on their assessment Process (the FEEE) Assessment at CBU

11. Step 5, Improving • This final step is the culmination of all of the assessment effort  Improving the learning environment as a result of acting on what is discovered in the annual cycle. • Units will take action such as modifying pedagogy or curriculum, etc. • This step merges back into the first step of Defining as the cycle repeats with a review of the PSOs, etc. Assessment at CBU

12. Examples of “Improving”, p.1 • Reviewing the evaluation process for standardization • Review the use and integration of writing instruction • Enhance the development of the “voice” concept in writing instruction • Better incorporate the involvement of faculty in assessment development • Increase coverage of effective modes and media for diverse population communication • Develop a rubric for better evaluation of PSO • Develop a new required course to provide for missing skills and knowledge Assessment at CBU

13. Examples of “Improving”, p.2 • Review standards • Review exit review process • Enhance curricular and assessment processes Note: These examples are extracted from the CBU “Assessment Activities Review ‘Sampler’ for the 2006-2007 Academic Year. Assessment at CBU

14. DEFINING CONTINUED IMPROVMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 1 CA State Comp Exam writing pass rates = 14% at 80 – 90%; 82% at 90% or above; 4% incomplete. Dr. Collins and Dr. Stokes will continue to collect exam pass rate data for the next academic year School of Behavioral Sciences Masters in Family Therapy Most instruction is adequate or better and not indicative of any significant shortcomings. This early evaluation needs to be reviewed for standardization. Assessment at CBU

15. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 3 = Written Communications 90% of students scored at 78% or better on writing rubric. Dr. Minton and Professor Moore will continue the development of process and procedures. School of Behavioral Science BA in Sociology Standard of success was met (80% at 75th percentile). Continue to use and review use of writing/integration instruction. Assessment at CBU

16. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 5 = Integration of Faith 30% exemplary, 20% proficient, 9% adequate, 9% inadequate, 32% not evaluated. Dr. Herrity and Dr. Chapman will collect and analyze rubric data and peer evaluation of faculty using new methods. School of Business BA/S in Business Training of faculty on the use of Power Posters and other assessment methods. Rubrics will be developed. Generally acceptable instruction, but process of evaluation needs improvement. Assessment at CBU

17. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 4 = Research skills MS/SS Rubric- 43% exemplary; 55% proficient. SOE Portfl. Rubric- 69% exemplary; 30% N/A Dr. Crist, Dr. Timmons, and all program coordinators will continue the development of process and procedures School of Education Graduate Programs Curriculum is clear and students have good opportunities; some weakness remains in establishing their “voice”. Continue/Improve focus on writing; Enhance developing “voice” concept; Continue review and development of PSO. Assessment at CBU

18. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 6 = Cross Cultural Communication Class average 80.7% on critical assignments evaluated. Professor Cabral will coordinate integration of new curricular elements in targeted classes in each grad program. School of Education Graduate Programs Instruction on working with diverse populations is adequate. Increase coverage of effective modes and media for diverse population communication. Assessment at CBU

19. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Faculty Self-Assessment Cycle Increased Scholarly Activity Personal Development Plan Faculty Faculty Self Study identifies number of conferences, papers, and presentations for previous year Faculty Self Study reports results of efforts and personal level of satisfaction with growth. Faculty member implements improvement plan (answers additional calls for papers, collaborates with colleagues, takes advantage of internal opportunities.) In conference with supervisor, faculty member formulates an improvement plan. Assessment at CBU

20. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 3 = History and Political Science Result for neither major were acceptable: History only 24% exemplary or proficient; Poly Sci only 50% were exemplary or proficient. Dr. McHorney and Political Science and History faculty will continue the development of process and procedures. History, Political Science, Criminal Justice BAS Poly Sci faculty will develop a new required course to provide for the missing skills and knowledge. History faculty will consider this as well. Instruction, curriculum and/or evaluation and MOAs need to be improved/revised. Assessment at CBU

21. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 3 = History and Political Science Result for neither major were acceptable: History only 24% exemplary or proficient; Poly Sci only 50% were exemplary or proficient. Dr. McHorney and Political Science and History faculty will continue the development of process and procedures. History, Political Science, Criminal Justice BAS Poly Sci faculty will develop a new required course to provide for the missing skills and knowledge. History faculty will consider this as well. Instruction, curriculum and/or evaluation and MOAs need to be improved/revised. Assessment at CBU

22. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 1 Integration of Faith PSO 4 Original Scholarship Success was achieved with all students meeting the standards at some level Dr. Lu and Graduate English faculty will pilot a new Thesis Defense rubric in 2008-09. Modern Languages and Literature MA in English Thesis proposal process is developing properly to include these elements. Evaluation rubric for thesis proposals will be designed; further refinement will continue. Assessment at CBU

23. DEFINING CONTINUED IMPROVEMENT COLLECTING DATA ANALYZING DATA ACTION TAKEN Student Outcomes Assessment Cycle PSO 2 Respect Diversity PSO 6 Literature Knowledge Success was generally achieved in these evaluations with small exceptions in each PSO. Professor Flores and Spanish faculty will monitor newly implemented curriculum in SPA 125. Modern Languages and Literature BA in Spanish As a new program this evaluation is good news . The noted shortcomings are the lack of time in SPA 115 for literature. Literature will be introduced in SPA 125 allowing SPA 115 focus on vocabulary, language, and culture. Assessment at CBU