Alternative Assessment with Electronic Concept Mapping Software

1. Alternative Assessmentwith Electronic Concept Mapping Software Dianne Raubenheimer and Kevin Oliver, NCSU

2. 1. What is Concept Mapping

3. What is concept mapping? • concept mapping is a technique for organizing and representing information • a true concept map must include: • core concepts - usually enclosed in circles or boxes • relationships illustrated by lines and arrows connecting concepts AND by propositions or statements on those lines that explain the nature of the relationship Pages Books are made of Binding

4. From: http://cmap.ihmc.us/

5. How are maps used? • for assessment to gain insight into student knowledge structures--valid understandings and misconceptions • for group thinking to collectively brainstorm and structure issues relevant to a topic • for planning to organize information for a report, presentation, debate, or discussion • for research to analyze how groups respond to or perceive something

6. Concept Map Forms • a concept map can take many forms, provided it includes concepts and illustrates relationshipsthrough BOTH lines ANDproposition statements: • spider, with informationorganized outwardaround a central topic

7. Concept Map Forms • hierarchy, withinformation structuredfrom most to leastimportant; or frommost inclusiveand general toless inclusiveand specific

8. Concept Map Forms • flowchart, with information orderedin a linear format

9. Concept Map Forms • systems, withinformation orderedlike a flowchartwith inputsand outputs

10. Non-Examples A concept map is a very specific thing; it is NOT any of these things: A flow chart of a process that has no proposition statements explaining the relationship between stages

11. Non-Examples an organizational chartthat has nopropositionsor statementsthat explainhow differentpeople supportone other

12. Non-Examples Story Webs: analyze various literary elements (plot, characterization, theme), dissect characters, plots and sequence of events, to learn about composition

13. Non-Examples • a mind map is similar to a concept map, with a central topic • main themes arenoted on "branches" • main themes areexpanded into details • propositions explainingrelationships betweenconcepts are not included

14. 2. What is the Process for Developing a Map?

15. Select Domain • select a domain of knowledge somewhat familiar to the person constructing the map (e.g., air pollution) • the domain should belimited to a manageable size: • a segment of text • a lab activity • part of a course

16. Identify Key Concepts in Domain • list key concepts in domain (e.g. flowering plants, dicotyledon, monocotyledon, etc…) • order concepts from the most inclusive or general to the less inclusive and specific • Types of - dicotyledon, monocotyledon • Roots – fibrous, tap, tuberous, aerial, etc. • Leaves – simple, compound, pinnate, etc. • Stems – herbaceous, woody • Flowers – actinomorphic, zygomorphic

17. Construct Preliminary Map • use software or post-it notes to arrange concepts visually • attempt differentstructures:hierarchical,spider,flowchart,systems

18. Examine Relationships • insert lines/arrows and write preliminary proposition statements that explain relationships between concepts • look for cross-links between different domains of knowledge identified (if topic includes multiple domains) • help students understand every concept "could be" related to every other concept, but they must evaluate and choose the most important concept relationships

20. string maps illustrate poor understanding of a knowledge structure

21. 3. What are Some Strategiesfor Employing Concept Mapsin the Classroom?

22. Strategies: Complete Fill-in • fill-in blanks in pre-structured map from a list of concepts • helps studentsto consolidateunderstandingof conceptsand relationsamong them

23. Strategies: Partial Fill-in

24. Strategies: Fill-in-the-Propositions

25. Strategies: Fill-in-the-Propositions

26. fossil fuels deforestation clouds atmospheric water vapor ocean CO2 Industrial Revolution longwave solar radiation global warming human activity evaporation transpiration greenhouse gas concentrations ice albedo photosynthesis feedback loops shortwave solar infrared radiation Strategies: Pre-Selected Term Mapping Design a concept map to show your understanding of CO2, a greenhouse gas, and its past and future effect on climate. Use the following concepts:

27. Strategies: Seeded Term Mapping • also called "micro-mapping" • provide students with a list of "starter" or "seed" terms (5-6) • students must use these terms in their map AND use another 5-6 terms from their own knowledge of the topic

28. Strategies: Collaborative Mapping E.g. via Powerpoint and E-mail • assign articles to read on a selected topic • group students and provide each group with a Powerpoint file to pass around via e-mail • slide 1 contains the concept map; each student adds 3-4 items per round (e.g., concepts, links, propositions) • slide 2 contains names of group members • slides 3-4 contain comments from students describing changes they made to the group file, or relevant suggestions

29. Strategies: Advanced Organizer • develop an expert map of a course or a course topic • provide to students as an advance organizer prior to teaching and forreview purposes

30. Unguided Maps • Once students are familiar with concept mapping processes, they can also be asked to do unguided maps, e.g. • After reading a section of text, • At the end of a section of course work. • At the end of the course. • This helps with identifying and tying together core concepts, and also acts as a learning tool for test revision.

31. 4. How are Concept MapsUsed for Assessment?

32. ‘Live’ links available at: http://www4.ncsu.edu/~cdrauben/concept%20maps/Assessing%20Concept%20Maps.html

33. Items Commonly Assessed • were the correct concepts or nodes identified? • do the lines and arrows correctly link concepts together into clusters or hierarchies? • do the propositions or statements on the lines between concepts accurately describe how two concepts are related? • how detailed are the resources or elaborative examples attached to the concepts (e.g., Web links, articles, annotations, notes, excerpts cut and pasted from a class discussion board, etc.)

34. 4 Scoring Methods (McClure, Sonak & Suen, 1999) • were the correct concepts or nodes identified? • do the lines and arrows correctly link concepts together into clusters or hierarchies? • do the propositions or statements on the lines between concepts accurately describe how two concepts are related? • how detailed are the resources or elaborative examples attached to the concepts (e.g., Web links, articles, annotations, notes, excerpts cut and pasted from a class discussion board, etc.)

35. Structural Scoring Novak Scoring Protocol hierarchical structure, concept-links, cross-links, and examples

36. Sample Instruction Sheet (Dempsey et al, 2002) • Think about the following questions: • What is climate? • How is climate studied? • What factors determine climate? • How has climate differed in the past? • What can cause climate to change? • Recall the instructions for how to draw a hierarchical concept map. • Construct a concept map of your ideas about climate, starting here: Climate

37. Sample Scoring Protocol • a sample scoring protocol for this "climate" map assessment: • half a point for each of 12 specific concepts • one point for each of up to four coherent, labeled, hierarchical lines of connections related to prompting questions • extra points for good cross connections between hierarchies • note: large differences had to be reconciled between two scorers; the two scores averaged

38. Relational Scoring • Quality or importance of each individual concept-link, without regard to the overall structure of the map, e.g. one point for each correct link and proposition. • Modified relational scoring where, concept-links, cross-links, and examples were assessed using the following scale (West, et al) : • invalid relationship between concepts (0 points); • valid relationship between concepts but propositional label is incorrect (1 point); • valid relationship and propositional label correct but lacks foundational or core relationship to subject matter (2 points); and • valid relationship and propositional label and foundational or core relationship apparent (3 points).

39. Holistic Scoring • Using a rubric • Example http://www.markville.ss.yrdsb.edu.on.ca/ history/west/west_conceptmap.pdf • Combining structural and relational scoring

40. Other Scoring ideas Concept List: Creating Instability Cut these out to make it easier to move them around on the incomplete concept map (Dempsey et al, 2002)

41. Other Scoring ideas Points for each correct concept placed in the partially completed map

42. Some issues • structural scores increase significantly after instruction (West, et al, 2002) • relational method is the most reliable across raters (Rye & Rubba, 2002, West et al, 2002) • scores obtained from the correctness of propositions (relational method) were most likely to correlate with state and national standardized tests (Rye & Rubba, 2002) • but, relational method is less sensitive in measuring changes and differences over time (West, et a.)

43. 5. What Electronic Toolsare Available to SupportConcept Mapping?

44. Mapping Tools There are several commercial mapping tools: Some Examples: Microsoft Visio--business & technical diagrams SmartDraw--mind map templates, timeline templates, flowcharts, org charts, network diagrams, floor plans

45. Mapping Tools • Kidspiration and Inspiration--used heavily in K-12 settings, education templates and clip art, move between map and outline modes, attach notes, audio, hyperlinks, and other external resources, export function • MatchWare OpenMind--create mind maps with templates, export function

46. Mapping Tools cMap– Free online tool built for collaborative concept mapping, can attach video, images, files, links, and other external resources, export function

47. From Concept to Content Maps • increasingly, mapping programs allow people to attach resources to concepts (e.g., notes, links, images, files), reminiscent of e-portfolio systems • selecting and adding content resources to concepts provides another layer of detail and helps to show the types of information a student associates with a concept (e.g., a specific research article) • encourage students to annotate or add notes to explain the relationship of the attached resources to the concept; otherwise you may be forced to "guess" why they chose to include a resource

48. References • Dempsey, D., O'Sullivan, K., & White, L. (2002). Action research for SFSU's NASA-NOVA course. Retrieved September 27, 2005, from the World Wide Web:http://funnel.sfsu.edu/courses/gm310/assessment/ • McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map assessment of classroom learning: Reliability, validity, and logistical practicality. Journal of Research in Science Teaching, 36(4), 475-492. • Rye, J. A., & Rubba, P. A. (2002). Scoring concept maps: An expert-based scheme weighted for relationship. School Science and Mathematics, 102(1), 33-44. • Trochim, W.M.K. (2004). An introduction to concept mapping for planning and evaluation. Retrieved September 27, 2005, from the World Wide Web: http://www.conceptsystems.com/papers/intro_article.cfm • West, D.C., Park, J.K., Pomeroy, J.R., & Sandoval, J. (2002). Concept mapping assessment in medical education: A comparison of two scoring systems. Medical Education, 36, 820-826.