Using Electronic Interviews to Explore Student Understanding DJ Wagner 1,2 , JJ Rivera 1 , Fran Mateycik 1 , and Sybilly

1. Using Electronic Interviews to Explore Student Understanding DJ Wagner1,2, JJ Rivera1, Fran Mateycik1, and Sybillyn Jennings3 1Rensselaer Polytechnic Institute, 2Grove City College, 3Russell Sage College Interviewer Interviewer S302 Interviewer S302 Interviewer S302 Interviewer S302 Interviewer Interviewer S302 Context: Study: Abstract: The Science of Information Technology (ScIT) introduces students of all majors to the physical principles behind the operation of information systems. NSF is supporting current efforts to make ScIT materials useful to a large audience. As part of that effort, we are designing diagnostic questions addressing the topics covered in ScIT. Diagnostic questions are most useful when they address student preconceptions [1]. Clinical Interviews have long been regarded as an effective means of eliciting students’ conceptual frameworks [1-3]. Piaget [3] developed the method of critical exploration to see what thoughts lay behind a child’s initial answer to a question, and the PER community has used this method extensively [1-2]. We used interviews to study students’ understanding of total internal reflection (TIR) and how TIR relates to the operation of optical fibers. This poster reports on methods used to probe student understandings of optical fibers and total internal reflection (TIR). The study was conducted as part of the expansion and improvement of web-based materials for an innovative introductory physics course. Development of these materials includes the refining of multiple-choice diagnostic questions by examining preconceptions and misconceptions commonly held by students. Initially, we conducted face-to-face Piaget-style interviews with a convenience sample. Our next step was to interview students taking the course at Rensselaer. Physical limitations necessitated that this be done from a distance, so we conducted “e-interviews” using a Chat Room. In this paper we focus on the e-interview experience, discussing similarities to and differences from the traditional face-to-face approach. In the process, we address how each method informs us about students’ activation of prior experiences in making sense of unfamiliar phenomena (e.g., “transfer of learning”). • “What is an optical fiber and what does it do?” • Follow-up questions included “How does the fiber keep the light from escaping?” and “What can you tell me about refraction?” 2 Physics Faculty • Group A: Faculty, REU students, and in-service (non-physics) teachers were interviewed in a traditional (face-to-face) form in the Summer of 2002. • Group B:ScIT students (both pre- and post-instruction) were interviewed via WebCT’s Chat Room in the Spring of 2003. 5 Physics Students (REU) 5 In-Service Teachers 22 ScIT Pre-Instruction 8 ScIT Post-Instruction Face-to-Face Interviews Electronic Interviews Features Demographics of Interviewees • Convenience: Can be conducted from wherever, whenever, and however is convenient (at home, listening to music, brushing teeth, in jammies, etc.). S313: “Yea, I must agree that this is very convenient for me too.” • Can involve participants in different parts of the country, as was the case for our study. Physics Information Also see our other poster  • Typed thoughts often come across more coherently than spoken thoughts – no place holders. Features Students in both formats Activated Memories • Minimal transcription needed!!! (WebCT records conversations.) I: When you hear optical fiber (or fiber optic) what do you envision?S310: Those wands that they would sell in Disney Land I: OK. Tell me about those wands.S310: Well, they were handheld and looked similar to a flashlight with a battery inside and a switch to turn them on or off -- when you turned them on, there were all of these wires coming out of the top of the wand, and the ends would glow different colors • The well-established standard – lots of studies, literature S5: Well I'm kind of thinking like those things you get at like Disney World or something with a characters head and then it's got like those little strings that come out of the top. And there's like a light that you see at the end of the strings, but the strings themselves don't look lit up. … Like in those I think they're plastic because they’re soft and you can kind of brush your hand through them. So I would imagine that it's some kind of non-metallic material. • Interviewer does not need to worry about her gestures giving away her thoughts about the students’ answers. • Can utilize multiple communication channels. (words, gestures, tone of voice, etc) • Apparent anonymity (due to lack of face-to-face contact) makes some students more comfortable in expressing their lack of knowledge. • Can supply props or equipment. • Lack of distractions: 1 interview at a time in controlled environment Students in both formats “Tried On” Different Models as they tried to describe fibers. • Multitasking: Can hold 2 or 3 overlapping interviews in different “Rooms”, typing questions for student 3 while students 1 and 2 are typing their responses. (4 at a time is a bit much.) For complete S314 transcript, see papers below S314 starts with a visualization involving water: “I think of a long hollow cable, made of a light weight material, it should be waterproof … Possibly glass or plastic -- Light-weight in the sense that it should be flexible -- I think it has these characteristics to ensure that the data will reach or has the potential to reach its final destination.” Provides more detail upon request: “[light flows] Inside the hollowed out area -- Like water flowing through a pipe. … The light doesn't escape because the water surrounding the cable on the outside doesnt [sic] allow the light to ecape [sic] from the inside of the cable. … I think the water has a higher density surrounding the cable, allowing the light not able to escape the cable. … The light never touches the water, it only flows in the inside (hollowed part ) of the cable -- The water surrounds the outside of the cable.” Upon further questioning, she starts to doubt her model and qualifies her statements: “Now I think I am confused -- Let me re-phrase my answer … We have a hollow cable, made of either glass or plastic … This cable is felxible [sic] … Data flows through this cable, i think this data is made of bundles of light … I am not sure if this cable should be waterproof or not … I think as long as the data (bundles of light) are able to bounce around inside the cable, this data should be able to travel from one port to another. For complete S7 transcript, see papers below S7 starts with a reflective model: “it’s almost a series of reflections … I don’t want to say mirrors, but it’s got to be mirror-like, a mirror-like substance.” Then thinks about a containment model: “I guess if, if you did just enclose light in, cause I know the, the tubing … you know the light can’t get out of it. … Maybe it, maybe it wouldn’t need to reflect if it, uh, if it, you can’t escape the, the insulator, right? … maybe it can just, shwooo, travel right through. Maybe it doesn’t need to reflect.” Then goes back to reflection: “Well, from there [the edge of the fiber], from there, you would have to, would have to reflect off ‘cause you gotta keep going. So it would be kind of like, uh, [draws zig-zag]. … You know?” And ends in a quandary: “But then it also couldn’t slow down at the same time … I’ve seen, it almost looks like ... it’s a plastic substance, I know, ‘cause they use it … for computers and things, and it almost looks like … it’s a plastic, consistent construction, but … it’s gotta be reflecting somehow. I don’t know.” • Can draw figures and write eq’ns on paper (see samples below). • Ubiquity of Instant Messaging makes e-interviews familiar and “fun”. S301: “i must say this is pretty neat” • Interviewees do not need to be familiar with technology Constraints • E-interviews are a good fit for development of web-based materials. Constraints • Substantial transcription time for interviewer. • Larger time commitment for interviewee. Our median e-interview time was 6 times our median f2f time. • Hmms, sentence fragments, and other place holders can make transcript choppy (see full transcript below). • Not a controlled environment. S304_post: “sorry about the pause, i'm in my apartment and the UPS guy came and I live on the third floor..” • Subject may convey information using gestures and/or tone of voice, which are not easily transcribed (see video, compare to transcript). • No ability to write or draw free-hand. Some “work-arounds” were developed, e.g., using keyboard symbols / \ - | to depict light’s path • Can run into conflicts if one interview runs over and has to be awkwardly curtailed when next interviewee shows up. • Interviewer cannot glean affective information from unintended reactions or unconscious responses of interviewee [1] Lillian C. McDermott, “Bridging the Gap Between Teaching and Learning: The Role of Research,” The Changing Role of Physics Departments in Modern Universities: Proceedings of ICUPE, ed. by E. F. Redish and J.S. Rigden. [2] E. Hunt and J. Minstrell, “A Cognitive Approach to the Teaching of Physics,” Classroom Lessons: Integrating Cognitive Theory and Classroom Practice, ed. by McGilly (MIT Press, 1994). [3] J. Piaget, The child’s conception of the world, trans. by J. & A. Tomlinson. (Littlefield, NJ, 1926/1972). RPI work supported in part by NSF CCLI Program under grant DUE-0089399. Thanks to Leo Schowalter for access to ScIT students, and to the rest of the ScIT advisory committee: Karen Cummings, Toh-Ming Lu, Saroj Nayak, Jim Napolitano, Peter Persans, and Wayne Roberge.