2008 Greater Expectations Institute June 18-22, 2008 Snowbird, Utah

2. Assessment develop your targeted outcomes verify that your efforts are aligned with the outcomes develop an assessment plan collect assessment evidence assess the evidence close the loop

3. Let�s focus on each step. First, the targeted outcomes.

4. AAC&U Suggestions: The Essential Learning Outcomes Knowledge of human cultures and the physical and natural world Critical and creative thinking Quantitative literacy Civic knowledge and engagement Ethical reasoning and action

5. Campus Examples

6. Knowledge of Human Cultures: Students can describe and analyze the capacity of race, ethnicity, class, gender, sexuality, disability, age, generation, and/or nationality to inspire, inform, and influence writers, artists and audiences (CSU Monterey Bay)

7. Quantitative Literacy Students can use tables, graphs, charts, and diagrams to explain concepts or ideas (Cabrillo CC)

8. Ethical Reasoning and Action Students can engage questions of ethics and recognize responsibilities to self, community, and society at large (U. of Delaware)

9. Alignment Is your intervention or program systematically designed to achieve your targeted outcomes? How is each outcome systematically developed?

10. A Cohesive Curriculum Coherence Ongoing Practice of Learned Skills Systematically Created Opportunities to Develop Increasing Sophistication and Apply What Is Learned Synthesizing Experiences to Integrate Learning

11. Curriculum Map

12. Assessment Planning

13. Some Characteristics of Quality Assessment Valid Reliable Efficient and cost-effective Engage respondents Interest us

14. Common Direct Assessment Strategies for Assessing Student Learning Published Tests Locally-Developed Tests Embedded Assignments and Course Activities: Signature Assignments Portfolios Capstone Projects

15. Common Indirect Assessment Strategies Surveys Interviews Focus Groups

16. Assessing the Evidence Direct assessment often involves the application of rubrics. Raters should be normed/calibrated to achieve reliable results. Indirect assessment requires taking an honest look at what people are telling us.

17. Closing the loop requires collegiality and flexibility often requires focusing on our curriculum (vs. my course) or our campus-wide initiative (vs. my office) may require the support and collaboration of faculty, staff, administrators, and faculty and staff development professionals

18. Some Friendly Suggestions Focus on what is important. Don�t forget your adjunct faculty. Close the loop. Learn from your colleagues�both on-campus and external.

19. Without assistance, each of us will not only reinvent the wheel, we�ll also reinvent the flat tire.

20. Why do I � a librarian � care � or even know about assessment? What can I add to a conversation where others may have spent a career focusing on it � some of you in this audience. The answer lies in the LEAP report itself � translated into this visual� Why do I � a librarian � care � or even know about assessment? What can I add to a conversation where others may have spent a career focusing on it � some of you in this audience. The answer lies in the LEAP report itself � translated into this visual�

21. Our common goal is student learning - articulated best for me by the essential learning outcomes from Leap � the articulation of the knowledge, skills, and personal and social attributes we expect students to exhibit when they leave us. But it is also about how the foundation for student learning is engagement -- engaging students in their educational experience in three important ways. How they are inspired and engaged by faculty as instructors in and out of the classrooms, and through learning strategies and approaches developed by faculty to develop each student�s own passion and curiosity � teaching methods that are student centered � research based � rooted in real life. Teaching methods that require students to navigate within the world of information as critical explorers..regardless of the topic or the discipline... Ways to excite and motivate students through the content of the curriculum- engaging in the big questions � �both contemporary and enduring�. And then the importance of engaging students to take steps for their own development through a myriad of experiences available external to the curriculum � but intentionally linked to it as we design those learning experiences � together. This image is intentionally simplistic � but it reflects the inter-relationships of of student learning and for me is a reminder of the overall more complex framework for learning that extends far beyond the content of the course.. Libraries success is measured in large part on their contribution to student learning � but how do we know what that is? Our success is inextricably linked to the content of the courses � and even more-so � how the learning is structured � - especially around assignment design Our common goal is student learning - articulated best for me by the essential learning outcomes from Leap � the articulation of the knowledge, skills, and personal and social attributes we expect students to exhibit when they leave us. But it is also about how the foundation for student learning is engagement -- engaging students in their educational experience in three important ways. How they are inspired and engaged by faculty as instructors in and out of the classrooms, and through learning strategies and approaches developed by faculty to develop each student�s own passion and curiosity � teaching methods that are student centered � research based � rooted in real life. Teaching methods that require students to navigate within the world of information as critical explorers..regardless of the topic or the discipline... Ways to excite and motivate students through the content of the curriculum- engaging in the big questions � �both contemporary and enduring�. And then the importance of engaging students to take steps for their own development through a myriad of experiences available external to the curriculum � but intentionally linked to it as we design those learning experiences � together. This image is intentionally simplistic � but it reflects the inter-relationships of of student learning and for me is a reminder of the overall more complex framework for learning that extends far beyond the content of the course.. Libraries success is measured in large part on their contribution to student learning � but how do we know what that is? Our success is inextricably linked to the content of the courses � and even more-so � how the learning is structured � - especially around assignment design

22. Beginning in school, and continuing at successively higher levels across their college studies, students should prepare for twenty-first-century challenges by gaining: Knowledge Of Human Cultures and The Physical and Natural World Intellectual And Practical Skills Personal And Social Responsibility Integrative Learning through the application of knowledge, skills, and responsibilities to new settings and complex problems College Learning for the new global century identifies Essential Learning Outcomes � Knowledge.. Skills personal and social responsibility and it underscores the importance of creating an integrated learning experience.. I am going to focus on the intellectual and practical skills � expecting that the implications transfer to other essential learning outcomes such as those that are part of social and personal responsibility College Learning for the new global century identifies Essential Learning Outcomes � Knowledge.. Skills personal and social responsibility and it underscores the importance of creating an integrated learning experience.. I am going to focus on the intellectual and practical skills � expecting that the implications transfer to other essential learning outcomes such as those that are part of social and personal responsibility

23. These are the I and p skills in LEAP are� and it�s important not to lose track of the recommendation that� My own area of expertise for almost 20 years is that set of learning outcomes known as information literacy -- what they are � how to create learning environments where they can be developed � more dependent on how we teach than what we teach � and how we ultimately assess student learning of those outcomes. There is a great deal of overlap between info lit and these other I and p skills � oftentimes if you were to ask me to apply one of these labels on a specific outcome � I would be hard pressed to do so.. These are the I and p skills in LEAP are� and it�s important not to lose track of the recommendation that� My own area of expertise for almost 20 years is that set of learning outcomes known as information literacy -- what they are � how to create learning environments where they can be developed � more dependent on how we teach than what we teach � and how we ultimately assess student learning of those outcomes. There is a great deal of overlap between info lit and these other I and p skills � oftentimes if you were to ask me to apply one of these labels on a specific outcome � I would be hard pressed to do so..

24. Consider for a moment outcomes in these three areas� and since I don�t have clickers � I am going low tech and going to ask for show of hands � I am going to flash learning outcomes from sources in one of these three areas� and I want to know if you think it is A � info lit B � critical thinking Or c � communication Consider for a moment outcomes in these three areas� and since I don�t have clickers � I am going low tech and going to ask for show of hands � I am going to flash learning outcomes from sources in one of these three areas� and I want to know if you think it is A � info lit B � critical thinking Or c � communication

25. Is this a or b or c Hands????Is this a or b or c Hands????

29. These are a few of the 122 learning outcomes that are part of info lit competency standards for higher education�These are a few of the 122 learning outcomes that are part of info lit competency standards for higher education�

30. It doesn�t matter what we call the learning outcomes � or how we cluster them.. Whether they are the I and p skills from LEAP It doesn�t matter what we call the learning outcomes � or how we cluster them.. Whether they are the I and p skills from LEAP

31. Or what are referred to as integrated skills at some institutions � like Washington StateOr what are referred to as integrated skills at some institutions � like Washington State

32. Or a phrase I just heard recently from some Australian colleagues � they have an office for the development of these skills � in and out of curriculum � called academic skills� Or a phrase I just heard recently from some Australian colleagues � they have an office for the development of these skills � in and out of curriculum � called academic skills�

33. Or what is sometimes called this � particularly by the literacy communities� NFIL - with over 200 member organizations - just held an international colloquium in Alexandria - cosponsored by UNESCO and IFLA -- Alexandria Proclomation.. Information Literacy lies at the core of lifelong learning. It empowers people in all walks of life to seek, evaluate, use and create information effectively to achieve their personal, social, occupational and educational goals. It is a basic human right in a digital world and promotes social inclusion of all nations. OECD The knowledge-based economy is characterised by the need for continuous learning of both codified information and the competencies to use this information. As access to information becomes easier and less expensive, the skills and competencies relating to the selection and efficient use of information become more crucial... Capabilities for selecting relevant and discarding irrelevant information, recognising patterns in information, interpreting and decoding information as well as learning new and forgetting old skills are in increasing demand. Or what is sometimes called this � particularly by the literacy communities� NFIL - with over 200 member organizations - just held an international colloquium in Alexandria - cosponsored by UNESCO and IFLA -- Alexandria Proclomation.. Information Literacy lies at the core of lifelong learning. It empowers people in all walks of life to seek, evaluate, use and create information effectively to achieve their personal, social, occupational and educational goals. It is a basic human right in a digital world and promotes social inclusion of all nations. OECD The knowledge-based economy is characterised by the need for continuous learning of both codified information and the competencies to use this information. As access to information becomes easier and less expensive, the skills and competencies relating to the selection and efficient use of information become more crucial... Capabilities for selecting relevant and discarding irrelevant information, recognising patterns in information, interpreting and decoding information as well as learning new and forgetting old skills are in increasing demand.

34. These skills and abilities have some common elementsThese skills and abilities have some common elements

35. It�s irrelevent what you call theme � and I would argue that its equally irrelevent how you label them�The first and most important step is to define specific learning outcomes for your students � and the devil is in the details � We all want students who are critical thinkers � but what precisely do we mean by that � what do the outcomes specifically look like � in measurable ways� and how do we explicitly communicate our expectations to our students so that they know what is expected Here are some examples Oftentimes the process of articulating the learning outcomes for the institution is the most important part of the assessment process � because the conversations should include he various stakeholders � faculty � student life � librarians � students � and the articulation about the specific learning e.g. information literacy standards provide a framework � yes there are five standards and 122 learning outcomes � but it is very important for each institution to have a process of articulating those learning outcomes for your own students� It�s irrelevent what you call theme � and I would argue that its equally irrelevent how you label them�The first and most important step is to define specific learning outcomes for your students � and the devil is in the details � We all want students who are critical thinkers � but what precisely do we mean by that � what do the outcomes specifically look like � in measurable ways� and how do we explicitly communicate our expectations to our students so that they know what is expected Here are some examples Oftentimes the process of articulating the learning outcomes for the institution is the most important part of the assessment process � because the conversations should include he various stakeholders � faculty � student life � librarians � students � and the articulation about the specific learning e.g. information literacy standards provide a framework � yes there are five standards and 122 learning outcomes � but it is very important for each institution to have a process of articulating those learning outcomes for your own students�

36. Most of you know that a learning outcomes approach means basing program and curriculum design, content, delivery, and assessment on an explicit identification of the integrated knowledge, skills, and values needed by both students and society. It focuses on what sudents should be able to do and not what knowledge they possess. Most of you know that a learning outcomes approach means basing program and curriculum design, content, delivery, and assessment on an explicit identification of the integrated knowledge, skills, and values needed by both students and society. It focuses on what sudents should be able to do and not what knowledge they possess.

37. But here is where we have to pay greater attention� in a learning outcomes approach - But here is where we have to pay greater attention� in a learning outcomes approach -

38. As a matter of fact � the assessments are so firmly embedded in the curriculum that they are often indistinguishable from instruction -they are designed to provide the student with a genuine rather than a contrived learning experience Recommendation 13 of College Learning Students should know from the time they enter college that they will be expected to complete milestone and culminating projects� �authentic performances��to demonstrate both their progress in relation to the essential outcomes and their ability to use the learning outcomes in the context of their chosen fields. Here are some authentic assessments at a course level Checklist - steps followed� Categorizing Grid - print tools to identify articles in disciplines electronic databases to find articles� Journals - reflection on process portfolios - cumulative process - samples of work over time -can include work - self assessment - progress report conferences - ask specific questions to focus on process and provide feedback Rubrics - scaled set of criteria that clearly defines what acceptable and unacceptable performance looks like As a matter of fact � the assessments are so firmly embedded in the curriculum that they are often indistinguishable from instruction -they are designed to provide the student with a genuine rather than a contrived learning experience Recommendation 13 of College Learning Students should know from the time they enter college that they will be expected to complete milestone and culminating projects� �authentic performances��to demonstrate both their progress in relation to the essential outcomes and their ability to use the learning outcomes in the context of their chosen fields. Here are some authentic assessments at a course level Checklist - steps followed� Categorizing Grid - print tools to identify articles in disciplines electronic databases to find articles� Journals - reflection on process portfolios - cumulative process - samples of work over time -can include work - self assessment - progress report conferences - ask specific questions to focus on process and provide feedback Rubrics - scaled set of criteria that clearly defines what acceptable and unacceptable performance looks like

39. Let�s go back to a couple learning outcomes for information literacy and think about responsibility for the learning and for the authentic assessment The outcomes included in info lit competency standards for higher ed range from simple to complex on Bloom�s taxonomy of cognitive processes..Let�s go back to a couple learning outcomes for information literacy and think about responsibility for the learning and for the authentic assessment The outcomes included in info lit competency standards for higher ed range from simple to complex on Bloom�s taxonomy of cognitive processes..

42. And since I reference Iskills as a standardized assessment tool � I thought I would share just a bit more information about it. According to my analysis � there is overlap in some important areas with the CLA� but some even more important areas of uniqueness. According to the common scoring rubric for CLA as compared to the detailed outcomes for ISkills , CLA does not cover outcomes defined in Iskills as defining and accessing information � specifically articulate a need for information that defines a hypothesis or problem in operational terms, develop and apply a systematic strategy for ethically and legally finding, retrieving, and sorting information from a variety of relevant resources, representing a wide range of perspectives, acknowledging sources appropriately And Iskills � although it includes several areas of communication � does not address the craft of writing � punctuation, syntax, sentence structure etc.. Further analysis would be important for any institution that decides to use these tools. And since I reference Iskills as a standardized assessment tool � I thought I would share just a bit more information about it. According to my analysis � there is overlap in some important areas with the CLA� but some even more important areas of uniqueness. According to the common scoring rubric for CLA as compared to the detailed outcomes for ISkills , CLA does not cover outcomes defined in Iskills as defining and accessing information � specifically articulate a need for information that defines a hypothesis or problem in operational terms, develop and apply a systematic strategy for ethically and legally finding, retrieving, and sorting information from a variety of relevant resources, representing a wide range of perspectives, acknowledging sources appropriately And Iskills � although it includes several areas of communication � does not address the craft of writing � punctuation, syntax, sentence structure etc.. Further analysis would be important for any institution that decides to use these tools.