270 likes | 411 Views
Assessing critical thinking through scaffolded asynchronous online discussion. Yiong Hwee, TEO National Institute of Education Nanyang Technological University, Singapore yhteo@nie.edu.sg Evaluations & Assessment Conference Adelaide November 24-25, 2003. Outline. Introduction
E N D
Assessing critical thinking through scaffolded asynchronous online discussion Yiong Hwee, TEO National Institute of Education Nanyang Technological University, Singaporeyhteo@nie.edu.sg Evaluations & Assessment Conference Adelaide November 24-25, 2003
Outline • Introduction • Collaborative critique framework • Socratic questioning in Online Discussion • Using Knowledge Community to scaffold & assess thinking • Conclusion Assessing critical thinking through scaffolded asynchronous online discussion 2
Singapore Education Initiatives • Thinking Programme & Project Work (PW) in schools • Masterplan for IT in Education (1997-2002) • In-service teacher training at NIE • Hypermedia design module (Jul 03; 8 weeks) • Design a hypermedia project • Engage in Computer-supported Collaborative Learning (CSCL) using Knowledge Community Assessing critical thinking through scaffolded asynchronous online discussion 3
Collaborative Critique Framework • Value of critique of design projects • Use examples of professional work to establish standards • Use examples of students work to define high and low quality work (Thomas & Mergendoller, 2000) • Why use asynchronous online critique? Assessing critical thinking through scaffolded asynchronous online discussion 4
Research Question • What types of thinking skills and levels of information processing are exhibited by in-service teachers in a scaffolded online group discussion? Assessing critical thinking through scaffolded asynchronous online discussion 5
Collaborative Critique Framework In-service teachers critique past year student hypermedia project in online discussion: 5 – step critique process 1. Identify project process, audience & expertise 2. Justify strengths and suggest improvements 3. Justify weaknesses and suggest improvements 4. Weigh alternatives 5. Transfer to own project Assessing critical thinking through scaffolded asynchronous online discussion 6
Thinking Skills • Dimensions of Thinking (Marzano, 1998) • Map of Thinking Domain Framework (Swartz & Parks, 1994) • Socratic Questioning Method (Socrates; Richard Paul, 1995) All reasoning has a purpose, is based on assumptions, done from some point of view, and based on information & evidence; All reasoning leads somewhere, has implications and consequences Assessing critical thinking through scaffolded asynchronous online discussion 7
Socratic Thinking Prompts Assessing critical thinking through scaffolded asynchronous online discussion 8
Using Knowledge Community to Scaffold Thinking Assessing critical thinking through scaffolded asynchronous online discussion 9
Using Knowledge Community to Scaffold Thinking Assessing critical thinking through scaffolded asynchronous online discussion 10
Using Knowledge Community to Assess Participation &Thinking • Evaluation of online participation • Statistics on reading & writing behaviour Assessing critical thinking through scaffolded asynchronous online discussion 11
Evaluation of types of Socratic thinking • Message units according to Socratic Thinking Types Assessing critical thinking through scaffolded asynchronous online discussion 12
Evaluation of types of Socratic thinking • Sorting by Scaffold Prompts Assessing critical thinking through scaffolded asynchronous online discussion 13
Evaluation of level ofinformation processing Henri (1992) method of coding • Critical thinking – surface level • “I do not agree that the quit button is redundant” • Critical thinking – in depth level • “A different viewpoint is that the choice of content is excellent. Pupils can relate to the content chosen since it depicts pupil’s own daily experience i.e. doing various activities in a day.” Assessing critical thinking through scaffolded asynchronous online discussion 14
Conclusion Preliminary Analysis • Promising potential of ‘scaffolding prompts’ of KC reveals that • More students take a stand • Comments were focused • Comments were longer, detailed, with justifications • Software-scaffolding enable students to use all thinking types BUT they are weak in ‘Probing Assumptions’ (4.3%) & ‘Probing Implications & Consequences’ (5.8%) Assessing critical thinking through scaffolded asynchronous online discussion 15
Conclusion • Educators can help students to improve their thinking in certain areas where they are weak and further study can help us design better thinking scaffolds in online discussion forums Assessing critical thinking through scaffolded asynchronous online discussion 16
Future Research • How does scaffolding prompts help pre-service teachers in online discussion? • What have pre-service teachers learned and benefited from producing a group educational video? Assessing critical thinking through scaffolded asynchronous online discussion 17
THANK YOU! yhteo@nie.edu.sg (Knowledge Community: www.globalkc.net) Assessing critical thinking through scaffolded asynchronous online discussion 18
Critiquing Video Cases Research Question 1: How does critiquing professional productions and past student video projects help pre-service teachers’ learning? Method: • Content Analysis of discussion log of both groups • Online survey questionnaire at end of course: MCQ & essay type questions Assessing critical thinking through scaffolded asynchronous online discussion 19
Critiquing Video Cases Q1: My knowledge of what makes a good educational video was Assessing critical thinking through scaffolded asynchronous online discussion 20
Critiquing Video Cases Q: Seeing & discussing past student projects is a good way to define what low and high quality work look like Assessing critical thinking through scaffolded asynchronous online discussion 21
Critiquing Video Cases : Transfer of Learning Q: My group has incorporated some of the insights and knowledge gained from the online video critique discussion in our own video production. Assessing critical thinking through scaffolded asynchronous online discussion 22
Critiquing Video Cases :Qualitative Results Research Question 1: How does critiquing professional productions and past student video projects help pre-service teachers’ learning? • Free response in survey questionnaire & Content Analysis • “…have a better idea of how the standards differ and allow me to gauge where my project stands” • “…see clearly what are the certain do’s and don’t’s when producing a video” • “…give direction and pre-empt mistakes in my own production” • “learn from more people’s comments rather than just within my group members” • “Engaging in video critique by looking at people’s comments and critiquing is a fun activity” Assessing critical thinking through scaffolded asynchronous online discussion 23
Asynchronous Online Discussion Q: The use of asynchronous online discussion has more advantages over face-to-face discussion. Assessing critical thinking through scaffolded asynchronous online discussion 24
Asynchronous Online Discussion Q: After reading my peers’ comments, some of my beliefs and ideas changed. Assessing critical thinking through scaffolded asynchronous online discussion 25
Critiquing Video Cases :Qualitative Results Research Question 2 : What benefits do pre-service teachers report that asynchronous online discussion has on their learning? • Free response in survey questionnaire & Content Analysis • Reading peers’ comments broadens point of view and obtaining a different perspective • Confirmed advantages of asynchronous discussion: • tend to contribute more as not afraid of ‘reprisals’ face-to-face • Permanent record encourages reflective thinking • Overcome limitations of distance and time Assessing critical thinking through scaffolded asynchronous online discussion 26
Role of Scaffolding Stems Q: The use of scaffolding prompts before posting a note helps me focus my thoughts Assessing critical thinking through scaffolded asynchronous online discussion 27