1 / 41

Computer-based and computer-assisted tests to assess procedural and conceptual knowledge

Computer-based and computer-assisted tests to assess procedural and conceptual knowledge. Richard C. Rayne and Glenn Baggott School of Biological and Chemical Sciences Birkbeck College, University of London, WC1E 7HX, UK r.rayne@bbk.ac.uk. Pedagogical Problem….

Antony
Download Presentation

Computer-based and computer-assisted tests to assess procedural and conceptual knowledge

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer-based and computer-assisted tests to assess procedural and conceptual knowledge Richard C. Rayne and Glenn Baggott School of Biological and Chemical Sciences Birkbeck College, University of London, WC1E 7HX, UK r.rayne@bbk.ac.uk

  2. Pedagogical Problem… • Biosciences are essentially practical and experimental subjects • Biosciences undergraduates therefore must develop abilities to: • acquire and apply procedural knowledge • demonstrate understanding of concepts that underpin procedures • use procedural knowledge in appropriate contexts; i.e. to develop strategic knowledge within the domain

  3. Knowledge framework c.f. Shavelson et al. (2002) EARLI/Northumbria Conference keynote address

  4. Declarative knowledge… • “Knowing that…” • a fact-base, or knowledge store

  5. Procedural knowledge… • “Knowing how to do something” • Consists of… • if/then rules • sequence of steps • Calls upon… • declarative knowledge

  6. Schematic knowledge… • Having a scientifically justifiable conception, i.e. “knowing why” • May be used to… • interpret problems • troubleshoot • explain an outcome • predict an outcome • Depends on… • having an understanding of principles • “conceptual knowledge” = combination of declarative and procedural knowledge

  7. Strategic knowledge… • Knowing when, where and how to use certain types of knowledge in a new situation • conditional knowledge, strategies • what strategic knowledge is applicable here? • essential for solving new problems

  8. “Conceptual knowledge”…* • a subset of schematic knowledge • taken to equate to “understanding the basis for a particular procedure” * as I have used in the abstract

  9. Knowledge framework c.f. Shavelson et al. (2002) EARLI/Northumbria Conference keynote address

  10. A key aim… • Exploit the strengths of CBA to promote development of procedural and conceptual knowledge in biosciences students • Context… • first-year module: Molecular Cell Biology • using “case-based” approaches in different forms

  11. Two Approaches • TRIADS CBA focusing on a “classic experiment” • Meselson-Stahl experiment: outcome simple, but to appreciate its power requires understanding of several sophisticated principles • CaseIT! simulation software • to teach and assess procedural and conceptual knowledge of key molecular biology techniques • http://www.uwrf.edu/caseit/caseit.html

  12. CaseIT! • Virtual molecular biology lab • “apparatus” for performing separation of DNA fragments and for identification of specific DNA sequences • apply molecular biology techniques in the context of a case study of human genetic disease • must perform a Southern blot…

  13. Southern Blot Procedure • Component tasks • obtain DNA sample and select enzyme • perform reaction to digest the DNA • load sample onto electrophoresis gel • run gel; stain and destain • transfer DNA from gel to a membrane • select radioactive hybridisation probe and apply to membrane • reveal specific DNA sequences by exposing membrane to X-ray film

  14. Movie • CaseIT in action

  15. Computer-Assisted Test • t0 = Introduction to techniques and to CaseIT! (demonstrated in lecture) • t+1 = Classroom session, hands-on (3 h) • students work on cases involving Southern blotting as the key technique • t+5= Test session • paper-based test with unseen case • must use CaseIT to obtain results • test items (short answer responses) depend on correctly performing Southern blot

  16. Why did we do it? • History • In 2002, only 4 of 40 students correctly answered an item on a TRIADS exam which required understanding of the Southern blot procedure • For 2003, we devised and used the CaseIT test; it was used again in 2004 • CaseIT! Test Results • 2003 & 2004: extremely poor on the whole • 2004 mean = 42% • 30 of 66 got <35%! • but 61 of 66 (in 2004) were able to perform the Southern blot procedure correctly • TRIADS Item, 2004 • 18 of 57 scored 100%

  17. Southern blot TRIADS item

  18. Summary • CaseIT! provides a reasonably realistic analogue of “real” lab techniques • helps build understanding of rationale/ approach in “fail-safe” environment • The CaseIT! test “forced” students to learn blot procedure! • have to learn a logical sequence of procedures that mimics the learning required in the “real” situation • CAA/CBA need not be a “test given on a computer”

  19. Acknowledgements • Funding: • Birkbeck College Development Fund • Higher Education Funding Council of England (HEFCE) through FDTL4 • People at Birkbeck: • Authorware Programmers: Dijana Maric, Ellen McCarthy, Caroline Pellet-Many • People elsewhere: • Don Mackenzie and his team at the Centre for Interactive Assessment Development, University of Derby • Mark Bergland, author of CaseIT!, University of Wisconsin-River Falls

  20. References Bergland M., et al. (2004) DNA Electrophoresis Module for CaseIT!, version 4.0.2. http://www.uwrf.edu/caseit/caseit.html Shavelson, R.J., Li, M., and Ruiz-Primo, M.A. (2002) Evaluating new approaches to assessing learning. Keynote address to the Joint Northumbria/EARLI Assessment Conference, Newcastle, UK. August 2002.http://www.stanford.edu/dept/SUSE/SEAL/Presentation/Presentation.htm

  21. Contact • Dr Richard Rayne • Lecturer in Biology and Director of the OLAAF ProjectSchool of Biological & Chemical SciencesBirkbeck CollegeUniversity of LondonMalet StreetLondon WC1E 7HX, United Kingdom+44 (0)20 7631-6253r.rayne@bbk.ac.uk • OLAAF web site: • http://www.bbk.ac.uk/olaaf/

  22. Meselson-Stahl Experiment • Outcome: • DNA replication is semi-conservative in bacteria • Component knowledge/understanding required to understand the experiment: • what is an isotope? • what is density? • how does a centrifuge work? • how to separate molecules based on their density? (density gradient centrifugation)

  23. TRIADS CBA • 2004: text and supplementary written materials provided • 5 items on TRIADS exam; 1 comprehension + 2 application + 2 problem solving • 2005: TRIADS tutorial to replace (or supplement) the paper materials • uses animations to demonstrate concepts visually • will test basic understanding at “component level” (i.e. concept-by-concept) • Recall Qs precede Comprehension Qs • Achievement: must answer lower level to proceed • same items as in 2004 to be used on exam to probe conceptual understanding

  24. M-S 1.1

  25. M-S 1.2

  26. M-S 1.3

  27. M-S 2.1

  28. M-S 2.2

  29. M-S 2.3

  30. M-S 3.1

  31. M-S 3.2

  32. M-S 3.3

  33. M-S 4.1

  34. M-S 4.2

  35. M-S 4.3

  36. M-S 5.1

  37. M-S 5.2

  38. M-S 5.3

  39. Assessment Development Model

  40. Cognitive type inventory • ReCAP* • Recall • Comprehension • Application • Problem solving (= analysis + synthesis + evaluation) * Imrie (1995) Assessment & Evaluation in Higher Education 20 (2): 175-189.

  41. Southern blot item 2

More Related