Simone C. O. Conceição, PhD University of Wisconsin-Milwaukee

1. Effective Learning, Teaching, and Research Strategies Using Concept MapsAssessmentsSeptember 2017 Simone C. O. Conceição, PhD University of Wisconsin-Milwaukee Beijing Normal University Beijing, China

2. Outline

3. Icebreaker

4. Words of Wisdom Choose emoticons to form sentences that express your feelings about the course until now.

5. Icebreaker • What do you know about learning assessment? • What you want to gain from this session

6. LEARNING ASSESSMENT STRATEGIES

7. Outcomes and Assessments • Outcomes: what we hope will result from the learning experiences. • Assessments: determine if the intended outcomes occurred; measure what the learners gained from the instruction. (Cennamo & Kalk, 2005)

8. Critical Questions • What changes in thinking or performance should occur? • How will you know these changes have occurred? • What activities will help facilitate these changes in thinking or performance? (Cennamo & Kalk, 2005)

9. Developing Outcomes and Assessments • Identify learning goals and outcomes • Classify outcomes by type of learning represented • Determine the subskills required to obtain the desired outcomes • Develop assessments for each outcome and subskill (Cennamo & Kalk, 2005)

10. Identifying Learning Goals and Learning Outcomes • What will success look like? • What is the expected outcome? • How will you know it is successful? All instruction is developed with some purpose in mind. Clients often express purpose in terms of the product instead of the learning outcome. (Cennamo & Kalk, 2005)

11. Identifying Learning Goals and Learning Outcomes (Cont.) • Who is the target audience? • What should the target audience do or think after viewing the program? To determine the learning outcome, you have to know something about the target audience, the content, and the context in which the skills should be applied. (Cennamo & Kalk, 2005)

12. Goals reflect the overall learning target The big picture Stated not in one word, but as a phrase Outcomes provide the details to translate those goals into measurable learning requirements Statement of what the instruction will accomplish, what the learners will be able to do after the instruction Outcomes focus on changes in thinking or performance that the instructional activities facilitate Identifying Learning Goals and Learning Outcomes (Cont.)

13. Types of Learning Outcomes • Verbal information – declarative knowledge • Intellectual skills – procedural knowledge • Motor skills – physical task • Attitudes – affective knowledge • Cognitive strategies – metacognitive or conditional knowledge (Cennamo & Kalk, 2005)

14. Writing Objectives Describe: • The behavior the learner will exhibit • The conditions under which the behavior must occur • The criteria that must be met for acceptable performance (Cennamo & Kalk, 2005)

15. Writing Objectives (Cont.) Should include: • Audience • Behavior: observable and measurable • Conditions under which the behavior must be performed • Degree to which proficiency in the behavior will be demonstrated (Cennamo & Kalk, 2005)

16. Benefits of Learning Outcomes • Activates learners’ prior knowledge • Allows learners to adopt or reject objectives as their personal goals • Helps set up a cognitive organizational scheme for the new information • Provides cues about what to attend to within the instruction (Cennamo & Kalk, 2005)

17. Sample Assessments for Types of Learning Outcomes (Cennamo & Kalk, 2005)

18. Types of Scoring Keys Using Concept Maps Assessments should be valid and reliable (Cennamo & Kalk, 2005)

19. Questions to Test Validity • Does this concept map measure what it should? • Is anything missing? • Is the concept map too complex? • Does the concept map delve into the depth of knowledge learners should demonstrate? • Would those who develop more complex concept maps also perform the job well? (Cennamo & Kalk, 2005)

20. Strategies to Concept Map Reliability • Clarity of propositions • Clarity of concept items • Consistency with assignment instruction • Sufficient number of concepts • Precise scoring

21. Assessment Purposes • Monitor Learning Progress: determine the degree to which learners have mastered the course content in a concept map • Determine Instructional Effectiveness: Tells the designer if the instruction about concept mapping was effective (Cennamo & Kalk, 2005)

22. Using Assessments to Monitor Learning Progress • Provides students with a learning opportunity • Helps learners monitor their own understanding or performance by comparing concept maps throughout the course • Helps learners monitor progress over time (Cennamo & Kalk, 2005)

24. Rubrics and Quality concept maps

25. What is a rubric?A rubric is a tool that teachers use to assess many different types of assignments. What are the elements of a rubric? • Expectations for the assignment • Criteria, arranged in levels of quality from excellent to poor, that a student must meet • Points or grades a student can earn based on the levels of quality

26. Criteria for Grading a Concept Map Graphical Structure • Number of concepts • Number of cross-links • Number of hierarchical segments • Unconnected parts • Spatial distance • Graph connectivity • Hierarchiness Allows an objective evaluation • Hierarchical structure of knowledge leads to hierarchical concept maps • Cross-links show understanding Not all structural components provide valid assessment

27. Criteria for Grading a Concept Map Content • Quality of concepts • Completeness of concepts used • Quality of concept labels • Completeness of relationships • Proposition correctness • Proposition quality (correctness, validity) • Propositions depth of explanation • Proposition relevance • Correct placement of concepts and relations • Richness of relationships Conciseness • An excellent concept map is concise • Concise does not mean small • All concepts and propositions should be relevant • Avoid including unnecessary concepts, propositions, or cross-links

28. Beginning Concept Mappers • Once they learn to create concept maps, they tend to make complex concept maps and use linking phrases • They tend to include a large number of concepts, propositions, and linking words • They often construct a good concept map that does not respond to the focus question

29. Issues with Structural and Content Criteria • Large number of relevant propositions • More complete concept maps may not necessarily provide greater insight into a learner’s thinking • Measures are often taken independently – proposition quality is assessed for each proposition without considering the coherence with other propositions • Rubrics evaluate whether the proposition is true or false, independent of relevance to the topic of the concept map

30. Excellent Concept Map • Has high clarity • Has a clear message • Communicates key ideas • Is the result of an iterative effort

31. Good cmap responds to the focus questionExcellent cmaps explains the response in a clear fashion

32. electronic cmap tools software and timely feedback

33. Use of Electronic Cmap Software Advantages Limitations Extra load for teachers to evaluate maps Students don’t receive feedback right away Without immediate feedback, students tend to lose motivation • Ease of making corrections • Flexibility of presenting content • Availability of promoting interactions among teacher and students (Wu, Hwang, Milrad, Ke, & Huang, 2012)

34. Timely Instructor Feedback • Helps students revise and resubmit cmaps • Enhances student motivation, learning, reflection, and understanding • Helps students progress more efficiently and effectively in courses (Wu, Hwang, Milrad, Ke, & Huang, 2012)

36. Examples of concept map activities and assessments

37. Analyzing Readings and Developing Papers Map of Course Readings Develop Papers Submit a concept map of a paper a student plans to write Instructor provides feedback before student starts writing paper This helps conceptualize paper and understand relationships Good for writing literature reviews • Pick three articles • Read articles • Immediately after reading articles map out the articles demonstrating relationships among them Daley, Cañas, & Stark-Schweitzer (2007)

38. Analyzing Experiences • Create a map of personal development as an adult • Show specific links between own unique experiences and adult development theories being studied Daley, Cañas, & Stark-Schweitzer (2007)

39. Case Studies • Instructor presents a case from the field • Instructor asks students to work in groups to map out the case • Students participate in online groups to discuss course readings • At the conclusion of the module, students develop a second concept map based on the new knowledge they acquired in the module Daley, Cañas, & Stark-Schweitzer (2007)

40. Relating Theory to Practice • Nursing and medical students develop concept maps that depict the care they will provide a client based on the understanding of theories they have learned • Students can develop maps that link the disease process of a client to the treatment plan, medications, and family influences Daley, Cañas, & Stark-Schweitzer (2007)

41. Examples of Grading criteria and rubrics

42. Synthesizing Concepts in Readings

44. Summarizing a Chapter

45. Example of a Rubric Using Concept Maps Source: www.ncsec.org/team11/RubricConceptMap.doc

46. Example of a Rubric Using Concept Maps (Cont.) Source: www.ncsec.org/team11/RubricConceptMap.doc

47. Example of a Rubric for Papers Source: http://teach.its.uiowa.edu/files/cft.uiowa.edu/files/Concept%20Map%20Rubrics.pdf

48. Bartels’ Scoring Rubric for Concept Maps Adapted from (Bartels, B. H. (1995). Promoting mathematics connections with concept mapping. Mathematics Teaching in the Middle School, 1(7), 542-549.)

49. Example of a Rubric Using Cmaps Source: http://www.rcampus.com/rubricshowc.cfm?code=KX9WA5&sp=true&

50. Cronin et al.’s Evaluation of a Concept Map Adapted from (Cronin, P. J., Dekker, J., Dunn, J. G. (1982). A procedure for using and evaluating concept maps. Research in Science Education, 12(1), 17-24.)