HyLighter Interactive Annotation System

1. HyLighter Interactive Annotation System Shine a New Light on Teaching and Learning FETC 2005 - Orlando, FL

2. HyLighter Presenters • Dale W. Lick, Ph.D. University Professor, Florida State University • David G. Lebow, Ph.D. President, HyLighter, Inc. • Hope Hartman, Ph.D. Professor of Educational Psychology, City College of New York & the City University of New York Graduate Center • Edie LeBas, M.A.T. English Instructional Team Leader, University of Miami Online High School

3. on-line annotation programs spreading across the digital landscape • Many types of annotation systems largely reflect the many purposes for reading itself. • Thousands of websites offer some form of annotation capability for public/social functions, collaboration, and education.

4. HyLighter supportslearning, assessment, and knowledge-building • 1. Makes thinking that is ordinarily hidden, become transparent and available for self-reflection and feedback. • 2. Allows users to continuously compare their developing understanding to others, monitor progress, and assess performance. • 3. Provides new, technology-enhanced ways to organize and synthesize ideas from multiple sources and create meaning from multiple perspectives.

5. The HyLighter Difference: • draws broadly on the learning sciences (e.g., expert performance, cognitive load theory, self-regulated learning) • merges electronic markings on digital text and other "information objects" made by any number of readers • generates composite displays showing convergence and divergence of highlighting and comments

6. highlight a section of text

7. add a comment

8. view a comment

9. click the Compare button

10. view “map” of group highlighting

11. view merged highlighting and comments: the "collective annotation model"

12. The Interactive Annotation Spreadsheet

13. Task Perspective Elements

15. Interactive Annotation forArgument & Persuasion • Introductory training sequence. • Participants read and annotate material. • Participants compare views and generate group views. • Instructor performs assessment activities, leads whole class discussion, models active reading strategies, provides feedback to students, and modifies instruction as necessary.

16. HyLighter andMetacognitive Teaching • Includes awareness of the sources and characteristics of students' misconceptions, selection of instructional strategies to overcome students' misconceptions, and monitoring/ evaluating progress.

17. HyLighter Interactive Annotation System Manager • Malleable to meet needs of different user groups • Integration with other technologies • Assessment instrumentation for complex cognitive skills • Research across the curriculum and training contexts

18. HyLighter Use by CCNY Grad Students inEd Psych Courses Fall 2004 Overview • Two different types of applications of HyLighter: Case Studies & Explanations of Answers on Multiple Choice Tests. • Students were working on their masters degrees in teaching. • Approximately half of the students were born abroad. Assignments in Preparation for Use of HyLighter • Text-based annotations preceded online annotations. • Online case annotations preceded multiple choice explanations. Data Sources • Case Studies of Beginning Teachers • Class Discussions • Multiple Choice Test Answer Explanations • End of Multiple Choice Test Comments • Essays on HyLighter

19. Case Study Results Many students commented on how using HyLighter made them more active readers & that the acts of annotating & comparing annotations increased meaningfulness & memorability of what they read. A few students didn’t like the highlighting but liked writing comments. Students thought about what they were highlighting for the first time. Students had thought that any and all highlighting was of equal value and there were no right or wrong ways to highlight - it was just a matter of style or personal preferences. One student didn’t highlight anything in the case because she viewed everything in it as equally important. She and others didn’t realize it was important to be selective.

20. Case Study Results cont. 6. Teachers’ and selected students’ annotations can be used as models to scaffold students’ development of effective highlighting skills. 7. Some students were relieved that they could highlight a section of text and didn’t have to make a comment despite the appearance of the dialogue box. 8. Students liked comparing their highlights & comments with other students & the teacher so they could see if they missed any important information and if they included irrelevant details. 9. One student failed to use HyLighter when she heard there was a bug in the program because she thought that meant you would get a virus by going there (from Sweden).

21. Case Study Results cont. 10. Several students complained about the small size of the comment box making it hard to monitor your own comment writing. 11. Students persisted and enjoyed the HyLighter experience despite initial problems using it because of browser or other problems. 12. I identified and corrected students’ misconceptions through analyzing & responding to comments in the case studies. 13. Students were fascinated by the differences between their highlights and comments 14. Small group discussions about using HyLighter and students’ background influences were extremely rich and enlightening, especially because about half of the class were born abroad.

22. % Credit for Explanations for Selected Wrong Answers • Item/Credit None Partial Full • #8 29 57 14 • #12 75 25 • #14 37.5 12.5 50 • #15 10 80 10 • #17 29 71 • #19 25 75 • Mean 30.08 45.08 24.83

23. Positive I thought this worked very well. I liked the fact that it made one think in depth about the answers & try to justify them. It allowed me to see gaps in my thinking, and knowledge, and I think it would have been nice to have had this opportunity on the first test. I like this a lot! I would definitely be excited to do something like this with my students. Negative I wasn't exactly sure to what extent we were to explain ourselves and that might have been helpful to know. One criticism is that the comment box is too small and very fiddly. If you lose your train of thought, you then have to mess around until you regain it The test took much longer than I had expected. I wonder if an essay section is even necessary after all this writing. Summary of Comments Re:HyLighter at the End of Multiple Choice Test

24. FREQUENT CATEGORIES 1. Explanation/Description 2. Elaboration 3. Examples 4. Problem with Terminology 5. Part/Whole 6. Misconception 7. Thought Report 8. Personal Opinion 9. Misapplication 10. Problematic Item FINDINGS Students had more knowledge about content targeted in test items than their wrong answers implied, showing flaws in test items or their interpretation. 90% of students in course benefited from analyzing their response choice explanations by receiving at least partial credit for some wrong answers. Almost 50% of these students got at least partial credit for ALL wrong answers in the set of 6 items analyzed. Explanations of Multiple Choice Selections

25. Essays on HyLighterAcross Three Graduate Courses • Impact of Using HyLighter on Attitudes about Technology • Impact of Using HyLighter on Thoughts about Technology • Cognitive and Metacognitive Benefits for Students • Highlighting as a Reading Strategy • Teaching • Assessment • Technology

26. Past, Present and Future of asynchronous discussions in an online school AP Lit class

27. Pedagogical Challenges Students live in different cities in different time zones so asynchronous discussions are their only ways to discuss literature with each other. Focused and interactive discussions are an integral part of the Advanced Placement Literature classroom to develop students’ understanding of texts and literary elements.

28. Past: Apex Stranded Discussion Forum • Students have productive discussions about characters and plot development in novels, modern plays, and short stories. • However, students lack focus in responding to discussion questions about a specific passage in a difficult text or in analyzing stylistic features of a text. • In addition, students voice difficulty with answering multiple-choice questions about poetry.

29. Sample“Cry for Help” in a “Stranded” Discussion Post about the Romantic Poets To be honest, I did not really enjoy this module. I like reading poems because it is easy to come up with your own interpretation of it. However, when it comes to answering the questions on the quiz, I get most of them wrong because I view things differently. It's only after I see the answer that I understand it completely. I just want to know your opinion on reading these poems and how well you do when taking the quizzes.

30. Before students begin study of Twelfth Night by Shakespeare I decide to ask the HyLighter people for help because of AP Lit students’ frustrations and difficulties with multiple-choice quizzes on Romantic poets, I receive approval from Apex Learning to use their multiple-choice quizzes on the HyLighter site.

31. How students use the four multiple-choice quizzes in HyLighter • Students highlight their first choices and submit a “thought report” about how they decided on their answers • Students email teacher their first choices for feedback as to whether right or wrong. Teacher responds quickly. • Students are allowed to select a second choice for each incorrect answer as long as they provide a thought report. • Students compare their thought reports with others and then respond to questions about their choices and their reports. Students make suggestions for improvement.

32. Sample Student Thought Reports for First Quiz • I think that the answer is 7A. Viola is referring to Olivia's physical complexion. Viola is comparing apples to apples. The only other choice that made sense to me was Choice D, but it compares two unlike things, apples and oranges, so i went with the choice A. • For this question I struggled to pick between A and D. I eliminated D as my choice though because I think the statement was talking more about her physical beauty and not how she acted. For that reason it was logical to pick A because Viola is complimenting Olivia, saying that even if she was as ugly as the devil, she would still be beautiful.

33. Thought Report for Third Quiz: Identifying a speaker of a passage Malvolia is thrilled to find out that his mistress has declared her love for him. Olivia supposedly sends him a letter (in reality, the letter is written by Maria) and in the letter she tells him that she has fallen in love and that she wishes him to wear the yellow stockings. To get to this conclusion, I disregarded answers B, D, and E. It is not B because Malvolio is simply Olivia's steward. It is not D because Maria is always finding ways to amuse herself by making fun of Malvolio, so there is no reason why he would want to get Maria's attention. The answer is not D because Malvolio puts on the yellow stockings before they put him in jail. Finally the answer is not A because Toby and Andrew are always making fun of Malvolio and not the other way around. The answer is C.

34. Discussion Post after Third Quiz Note below the change in thinking pattern of an “apple and oranges” student I didn't miss any questions the first time, but by re-reading the questions that I narrowed down to two choices I was able to better understand why I selected the correct answer. For example, question ten was pretty confusing for me. I narrowed it down to Feste and Toby. Feste was the fool, and I didn't know why Toby would consider himself a fool, and at first thought, I just guessed Toby because it didn't sound like Feste. After re-reading the quote and looking at the two choices, I understood not only the quote but also remembered the scene that it was from, and this helped me to understand the correct answer.

35. What I as a teacher observed from this first trial • On the first quiz, some students struggled with having to write a thought report and wanted to summarize the text • in order to give an answer. • By the second quiz, most students had begun to model their thought reports after viewing other students’ thought reports, especially the ones who had correct answers. • Students who started out using test-prep “apples and oranges” thinking began to reread the text and come up with their answers based on their reading a second time.

36. What students said about HyLighter • More interactive One of my favorite options of this new system is that we get to compare and see how our classmates might have had a different view. This is definitely more interactive and enhances the learning process. Instead of just having an overall discussion, the compare option gives it greater depth by being more specific. It also gives me the opportunity to see how different people might have seen it in a different way.

37. What students said about HyLighter • More focused • I think it helps without a doubt. I enjoy seeing how other people approach the questions as well as having the opportunity to actual learn how to approach the question so I don't just continue to make the same mistakes. Having to discuss each question helps too because it keeps the discussion focused so it ends up helping instead of talking about pointless things.

38. What students said about HyLighter • Shows errors in thinking • I didn't get any questions wrong on my first attempt at the quiz, but looking at how some of the other kids answered their questions wrong was interesting. It was helpful to see how they could interpret some of the quotes incorrectly, but be able to defend their reasoning well.

39. What students said about HyLighter • Showed more “efficient” way • to reach the right answer • I found it really interesting to read the response to question ten. This was the question I was having the most trouble with, and it helped me to learn that other students also had the same problem. It also helped me to read the way they reached their answer. By reading these answers, I was able to find a shorter and more efficient way to reach the right answer.

40. What students said about HyLighter • As part of their discussion questions, students were asked to make suggestions about how HyLighter could better serve their needs. • Based on their suggestions, HyLighter is now designing a new version to meet the needs expressed by University of Miami Online High School students on the following slides.

41. What students said to add to HyLighter • Provide direct quotes from text and have students explain them • The format overall is very helpful. I do think it could be beneficial to be able to post discussions on certain elements of the play that may be harder to understand. It's important for me to be able to see how my other classmates are thinking, and this becomes hard because we are not in a typical classroom-format. Any way that make taking AP English Lit online more like a classroom would be helpful to anyone.

42. What students said to add • Provide the freedom to write an overall comment about a passage • I like this format. Yes, I do think having difficult passages would be helpful not only to learn different approaches but also to see how different people interpret writing, especially Shakespeare. It's very personal and to see others perceptions would be nice. I think the best part of writing is that there is so much freedom in it. I miss that one part about a regular classroom so that feedback would be very helpful.

43. What students said to add • Provide a way to respond to other students’ ideas • I think it would be a good idea to create more questions that cover the style of writing and important soliloquies. It would also be a good idea to be able to respond to other students. Similar to the Apex system, HyLighter should include the option of being able to have discussions with our classmates on the play as a whole.

44. What students said to add • Ask questions about the author’s style in important passages • I think it would be a good idea to create more questions that cover the style of writing and important soliloquies. It would also be a good idea to be able to respond to other students. Similar to the Apex system, HyLighter should include the option of being able to have discussions with our classmates on the play as a whole.

45. What the teacher said to add • Based on the success of UMOHS students, HyLighter will provide the following • features in their new version • Computer-graded multiple choice tests • Computer-controlled feature that allows students to compare after completion of test, rather than wait on teacher to open up compare feature. This feature will be available on both multiple-choice tests and student discussions about the text.

46. Dale W. Lick Dale@hylighter.com David G. Lebow David@hylighter.com Hope Hartman gravity1@mindspring.com Edie LeBas elebas@umohs.org PowerPoint Presentation www.hylighter.com/FETC2005.ppt