1 / 104

First, complete the knowledge survey on knowledge surveys

First, complete the knowledge survey on knowledge surveys. Plan Your Training From A to Z: Using Knowledge Surveys . Ed Nuhfer, Director of Faculty Development California State University at Channel Islands ed.nuhfer@csuci.edu. Knowledge Surveys are Assessment instruments.

darrell
Download Presentation

First, complete the knowledge survey on knowledge surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. First, complete the knowledge survey on knowledge surveys

  2. Plan Your Training From A to Z: Using Knowledge Surveys Ed Nuhfer, Director of Faculty Development California State University at Channel Islands ed.nuhfer@csuci.edu

  3. Knowledge Surveys are Assessment instruments 1. = I have insufficient knowledge to answer this question. 2. = I have partial knowledge or know where to quickly (20 minutes or less) obtain a complete answer to this question. 3. = I can fully answer this question with my present knowledge.

  4. ….what knowledge surveys sample

  5. All learning produces complex interconnected affectiveandcognitive synaptic “wiring,” and knowledge surveys sample a mix of both.

  6. A Framework for Planning a KS Responsibilities published institution's goals that apply to the course Goals -usually course instructors' goals (unpublished & general) Outcomes specifics of knowing & doing in course (usually published) How would I confirm student mastery? Items that require knowledge, skills, decisions, generating products Organize in order of course presentation

  7. Responsibility Example --a published descriptive statement "Goal 5: To understand how the physical sciences explain the natural world. Courses in the physical sciences that fulfill this requirement (1) examine the processes by which scientific knowledge is gained, (2) introduce the basic concepts and terminology of one or more of the physical sciences, and (3) explore how scientific knowledge influences human society." (Idaho State University Undergraduate Catalog, 2006)

  8. Example of goals that may not necessarily be a responsibility • I want students to achieve a life-long interest in my subject • I want students to develop self-assessment skills • I want students to understand the nature of critical thinking

  9. Example of outcomesThese are action statements."Students will be able to: • describe the scientific method and provide an example of its application; • pick a single theory from the science represented by this course and explain its historical development; • provide two examples of testable hypotheses; • provide two specific examples that illustrate why it is important to the everyday life of an educated person to be able to understand science; • describe two current examples of the relationship between physical science and public policy…. (4 more)"

  10. Making a knowledge survey (KS)Consider a skill or knowledge challenge such as might be on a test. For example, a challenge from the Geoscience Concept Inventory… Are rocks and minerals alive? (A) Yes, rocks and minerals grow (B) Yes, rocks are made up of minerals, and minerals are analogous to plant cells (C) Yes, rocks and minerals are always changing (D) No, rocks and minerals don't reproduce (E) No, rocks and minerals are not made up of atoms

  11. Make a KS item from that cognitive challenge by adding an affective root • I can distinguish the basis by which scientists tell whether rocks and minerals are alive. • (A) Yes, rocks and minerals grow • (B) Yes, rocks are made up of minerals, and minerals are analogous to plant cells • (C) Yes, rocks and minerals are always changing • (D) No, rocks and minerals don't reproduce • (E) No, rocks and minerals are not made up of atoms

  12. We may choose to address the same content with several items framed in different ways. Here, the former content addressed in multiple choice format is presented again, but now in essay format. • I can articulate the basis by which scientists distinguish living from non-living material.

  13. What kinds of items can you put onto a knowledge survey? Any challenge you can express-- Any test or homework item Convergent challenges & factual knowledge Divergent challenges evaluated with rubrics Ethical decisions Projects outcomes All levels of thinking

  14. Example-content comprehension

  15. Project: Understand development of a major theory • 97. In under 500 words, I can describe how the contributions of all of the following were needed to achieve the current theory of plate tectonics: 1. Thales of Miletos (600 BC), 2. Aristotle (384 - 322 B.C.), 3. Nicolaus Steno (Neils Stensen) (1669), 4. William Smith (1769-1839), 5. Baron Georges Cuvier (1769-1832), 6. James Hutton (work of 1795), 7. Charles Lyell (1797 - 1875), 8. Charles Darwin (work of 1859), 9. William Thomson (Lord Kelvin - work in 1897), 10. John Joly (work of 1908), 11. Thomas Chamberlin (work of 1899 - 1909), 12. Madame Curie (1867 - 1934), 13. Bertram B. Boltwood (work 1905 - 1909), 14. Alfred Wegener (work in 1912 - 1924), 15. Arthur Holmes (work of 1920s - 1930s) 16. Harry Hess (work in mid 1960's), 17. J. Tuzo Wilson (work in mid 1960's), 18. Fred Vine (Work in mid 1960's), 19. John Dewey (work in early 1970's), 20. Niles Eldredge and Stephen Jay Gould (1972 –2000).

  16. The knowledge survey emerges when we arrange the items in the order students will encounter the challenges in the course. • Students respond to each challenge at least twice (starting and ending of the course) through a three-point rating scale. • 1. = I have insufficient knowledge to answer this question. • 2. = I have partial knowledge or know where to quickly (20 minutes or less) obtain a complete answer to this question. • 3. = I can fully answer this question with my present knowledge. • Calculate the class average students give to each response. (Aggregate data of class as a whole.) • Plot the results (next slide)

  17. Results: pre-course knowledge survey • 1. = I have insufficient knowledge to answer this question. • 2. = I have partial knowledge or know where to quickly (20 minutes or less) obtain a complete answer to this question. • 3. = I can fully answer this question with my present knowledge.

  18. At a minimum, run the same survey again at the end of the course and compare results. Best practice: refer often to the knowledge survey during the course.

  19. Why do this? For very good selfish reasons For AWESOME altruistic reasons!

  20. Teaching for Learning & Satisfaction (after K. Feldman, 1997; 1998)

  21. Skills Quizzes Outcomes Projects Facts Assignments Facts Facts Skills Facts Enduring Concepts Readings Facts Papers Discussions Facts Facts Exams Skills Facts What we want to teach- what students see -

  22. Why not give them a map… to those destinations we want them to reach?

  23. Why make a knowledge survey? • Being prepared and organized • Make time early with text and materials • Ability to plan and prioritize • Avoid getting caught in content overload

  24. Why is it worth the time up front?

  25. Knowledge surveys—what will students new to the tool think?

  26. And how do they feel when they go through the survey at the end of class?

  27. Let's look again at the pre-course knowledge survey. • 1. = I have insufficient knowledge to answer this question. • 2. = I have partial knowledge or know where to quickly (20 minutes or less) obtain a complete answer to this question. • 3. = I can fully answer this question with my present knowledge.

  28. We can start to know our students in ways we cannot without the initial measure. Note: We asked the students to rate their present knowledge. We did NOT ask students to rate their confidence to eventually master this material. Pre-course ratings bear no relationship to what each student can or will ultimately achieve.

  29. Pre-course surveys offer useful information about particular class populations Confidence Rating -->

  30. Student Learning Outcomes versus Goals • Student Learning Outcomes are action statements that specify that a student must demonstrate achievement through some clearly observable action or generation of product. • Goal statements are necessarily broad statements that describe the general traits of learning. A goal statement need not be directly assessable.

  31. Examples of Outcome Statements - What we should use as KS Items 24. I can define psychology. 25. I can describe the evolution of psychology as defined from the 1920s through today. 26. I can summarize the nature-nurture debate in psychology, and describe the principle of natural selection.

  32. Examples of Goal Statements that were twisted into KS items. These goal statements began with "Students will be able to" and were converted into knowledge surveys by replacing that phrase with the root "I can…." • I can think clearly and logically. • I can find and critically examine information. • I can communicate at an appropriate level in both oral and written forms.

  33. Goal statements used as knowledge survey items produce no information of value. They usually stand out as aberrations in an otherwise well constructed knowledge survey. Post-course Pre-course

  34. Construct a full battery of items that address the high priority content of the entire course. • Be certain to address all chosen outcomes with knowledge survey items.

  35. Revelation - a Problem with Pacing

  36. With First Use - Expect to See Learning Gaps

  37. Pacing Problem Eliminated

  38. Next… Varied case examples

  39. Case - "Closing the Loop"

  40. Music Theory Course: First Use

  41. Case - Longitudinal Study- "Closing the Loop"

  42. The Value of Longitudinal Data:Using Post-Course Knowledge Survey Data from Two Classes Example Courtesy of Dr. Dexter Perkins, U of ND

  43. Using a Knowledge Survey in American Government Pre and Post Survey of 75 Questions Survey is Conducted Online, Through Blackboard Survey Questions are Organized Chronologically Results of Surveys are Shared with Students

  44. Samples of Items Used Question 8: I can explain ways in which the Constitution has become more democratic since its beginning in 1789. Question 17: I can tell you what freedoms are protected by the First Amendment. Question 37: Thinking of political party membership, I know what an independent voter is. Question 56: I know the difference between a standing committee in the Congress and a conference committee. Question 69: I can identify the three levels of the federal court system. Question 72: I can explain how judges are selected in California.

  45. POLS 150-04 American Political Science Spring 2008

  46. POLS 150-04 American Political Science Spring 2008

  47. POLS 150-04 American Political Science Spring 2008

  48. POLS 150-04 American Political Science Spring 2008

More Related