Assessment: Developing Learning Outcomes

1. Assessment:DevelopingLearning Outcomes

2. Teaching Learning Assessment Cycle • Establish Learning Goals • Provide Learning Opportunities • Assess Student Learning • Use the Results

3. Establish Learning GoalsMeaningful Learning goals should lead to the improvement of teaching and learning Institution Program Course

4. Institution: • UCCC’s Mission Statement: • SUNY Ulster is a vibrant community of learners distinguished by academic excellence, collaboration, innovation, service and responsible use of resources. As a public, comprehensive two year institution dedicated to providing affordable accessible education, we work in an ever changing environment to: • Prepare students to transfer to four institutions • Prepare students for success in college and in the workforce • Provide enrichment and life long learning opportunities • Augment learning through the integration and application of emerging technologies • Prepare students to live and work in a global society • Play an active role in economic development, and • Enhance the quality of life for residents in Ulster County

5. Benefits of Learning Outcomes • Helps students understand the nature of the program or course and what skills or knowledge will be expected of them. • Provides a guide to students so they are assured they have not missed any major content. • Gives structure to the instructor in lecture development – what to teach

6. Gives structure to the instructor in exam or assignment development – link to goals. Insures the student learning outcomes are linked to the college mission. Enables the public and four year schools to understand what the program or course intends to accomplish. Most importantly – leads to improvement of teaching and curriculum development.

7. Developing Learning Goals: • Meaningful learning goals will lead to improvement of teaching and learning. • There should be an interconnectedness of goals in programs and courses.

8. Developing Learning Outcomes • Learning outcomes should be developed collaboratively from research and reflection. • Collaborate: • This ensures the integration of course outcomes to program outcomes. As many faculty as possible should be involved.

9. Research • Internal Sources: • Mission statement • Current Syllabi • Previous program reviews and surveys   • External Sources: • Associations • Advisory Councils • Other colleges Program descriptions (websites)

10. External Sources: Advisory Councils Other colleges Program descriptions (websites) Associations

11. Reflection: • Key word: WHY. • Why do you want your students to know what you put down as a learning outcome.

12. Middle States Recommended Discussion Topics: • What is this course or discipline all about? • What does the department aim to achieve with this program? • What do we value about our discipline? • What are the most important things students learn in this discipline? • Why are those things important? • What does our disciplinary association think students should learn?

13. Why does the college offer this program? Why is it important student study it? How will this experience prepare them for post graduation? What do we expect of students in this program? • What do we want students to get out of this program? • How does this program relate to the college mission? • How does it relate to other programs? • What do our students do after graduation?

14. How will students be different after completing the program? What will the student be able to perform after the completing the program? What skills will the students acquire after the completing the program? What indicators will be used to measure the student’s performance?

15. Knowledge Thinking Attitudes

16. Knowledge • Duplication of procedure, learning definitions summarizing characteristics, understanding components. • Memorization • Not as stressed today as in previous decades

17. Examples of Knowledge Goals • Explain the three goals of a macro economy and how they are measured. • Summarize distinctive characteristics of a various novelists • Describe the …scientific model

18. Thinking • Application of knowledge in a new context • Analysis or the ability to understand the relationships amongst the components of a complex concept • Evaluation or making informed judgments on new material through research and problem solving • Synthesis or the ability to theorize, generalize, reflect generate new hypothesis or ne ideas. • Critical Thinking or seeking truth and ability to question

19. Examples of Thinking Goals • Explain a reaction • Explain the impact • Analyze errors • Compare and contrast • Explain why • Judge the effectiveness • Evaluate the validity

20. Research and identify Choose the appropriate procedure Write a poem using.. Theorize design Recognize Design and conduct…. Identify a problem…..

21. Attitudes • Dispositions and values • Appreciation of the subject • Enjoyment of learning • Metacognition • Formation of opinions and values • Development of a sense of ethics.

22. Example of Attitude Outcomes • Be intellectually curious • Choose an ethical course of action • Appreciate the merits of … • Identify the strengths and weakness of others’ work • Work independently • Work in groups • Develop leadership skills

23. Writing Learning Outcomes • Avoid broad nebulous terms: • Students will: • Learn • Know • Understand • Become aware • Appreciate • Think critically

24. Action words help students understand what we want them to learn and it makes assessment easier • Describe the solar the key characteristics of each planet in the solar system • Make appropriate references and deductions from biological information • Develop graphic spreadsheet to support a decision • Design a chemical experiment to test a hypothesis

25. Systematically analyze and solve problems… Defend one’s views Present original interpretations… Critically evaluate the effectiveness… Apply basic problem solving skills… Conduct research

26. Assessment of the Learning Outcomes

27. Two Categories of Measures • Direct measuresprovide evidence of whether or not the student has command of the subject matter • Indirect Measuresprovide data that is related to the act of learning. It implies the student has learned, i.e. how the student perceives whether or not he/she has learned. Questionnaires and surveys are indirect.

28. Direct vs Indirect • Direct evidence should answer the questions what did the student learn? What did he/she not learn? But it does not answer the question why did the student learn it? Why didn’t the student learn it? • Indirect evidence focuses on the learning process. Comments about homework or lectures or group work.

29. Choosing Evaluation Instruments: QUESTIONS TO ASK: • 1. Is the evidence provided by the method linked to learning outcomes? • 2. Is a standardized instrument appropriate for the learning goals of the institution • 3. Is the evaluation method comprehensive enough? • 4. Are important learning outcomes measured by multiple means?

30. 5.Do the test questions test what is desired to be learned? 6. Are the questions clear? 7. Do the questions elicit information useful for making improvements? 8. Does everyone interpret the response the same way? 9. Do the results make sense? Are the results corroborated by evidence?

31. Tools and techniques:

32. Tests • Standardized vs Home grown • Quote from Middle states: • Although a test created locally may not have the same statistical validity and reliability as a standardized instrument, it’s relevance to the specific learning goals in question may make it a more appropriate and more effective instrument.

33. Test blueprint is an outline that matches test items to the intended learning outcomes.

34. Rubrics • A set of criteria used to assess papers, projects, performances, field experiences. • Four Types: • Checklist • Rating Scale • Descriptive • Holistic

35. Checklist Rubrics • A list of yes or no or Did or Didn’t do • Used primarily in lab or clinical settings

36. Rating Scale Rubric • This is a checklist with a rating scale added to show the degree to which the things you’re looking of are present • Uses terms like Outstanding, Very Good Adequate Or • Strongly Agree, Somewhat Agree etc..

37. Descriptive Rubrics • These give a description of what constitutes deficient adequate or exemplary • Or unacceptable, Fair Proficient

38. Holistic Rubrics • For the more massive assessments, rating scale and descriptive rubrics may not be feasible i.e reading 1000 essays for placement. • Holistic rubrics have short narrative descriptions of the characteristics for outstanding work, acceptable work, and unacceptable work

39. Other Assessment Tools • Self reflection Engage in metacognition. Ask students to reflect on what they have learned This gives insight into the learning process and it helps students integrate what they have learned

40. Ratings and comments from internship supervisors • Placement Rates • Ad Hoc Surveys • Focus Groups • Portfolios

41. What is Middle States Looking For? • They will ask the fundamental questions: • Is the Institution fulfilling it’s mission? • Are institutional and program level goals clear to the public, students, faculty and staff? • Are Institutional programs and resources organized and coordinated to achieve these goals? • Is the Institution using assessment results to improve student learning and advance the institution.

42. Characteristics of Assessment Processes that meet Middle States Expectations • Useful Lead to improvement • Cost Effective No need to start from scratch use what is there • Reasonably Accurate Is evidence clear and visable -both direct and indirect • Planned Assessment must be interrelated to strategic plan and academic plan • Organized, systemized and sustained Implement an ongoing system and review it.

43. What Should Be Documented? • Learning outcomes in programs and courses . • An assessment Plan • Assessment results • Uses of results to show what is being changed towards improvement

44. Assessment: Using the Results To Improve Teaching and Learning

45. Three steps: • Choose a standard benchmark • Set a standard or benchmark for exemplary, adequate and inadequate • Setting targets for the collective class or program

46. Choosing a Benchmark

47. What is a Benchmark • The process of comparing the results of your test, essay, paper or other assignment against something.

48. Local Standards: Department members agree to what is a good grade on a specific exam. Having faculty involved in courses and exams leads to more attention paid to assessment. External Standards: This is the use of disciplinary association standards. Usually there are tests that are required from the association. Certifications have required exams with established benchmarks i.e. NCLEX

49. Internal peer benchmark The bell curve – or looking at how Psychology majors do on the economics test vs business majors. Or looking at all the students in the program. Which ones succeed and why? External peer benchmark We compare Jane Doe’s 65 to other instructors’ or colleges’ 65.

50. Setting Standards: • Choose the bench mark most appropriate for your course or program and set a specific appropriate performance standard. • Ask what is an example of exemplary work? • How does this differ from adequate and inadequate?