Science Vocabulary Builder Project STUDENT LEARNING IMPACT PROJECT Evidence Document

1. Science Vocabulary Builder ProjectSTUDENT LEARNING IMPACT PROJECTEvidence Document Deb White Teacher Candidate Minnesota New Country School 20 November 2009

2. Project Purpose: To assess whether students using online flashcards to study vocabulary words experience larger gains than students using traditional paper flashcards. Objectives: Students who complete the Science Vocabulary Builder Project will… • Learn meanings associated with Latin and Greek root words frequently used in science terminology • Become more confident in their ability to define unfamiliar words • Develop increased awareness of their personal studying preferences • Learn about online flashcards, a new tool to help them build vocabulary knowledge for any subject

3. Strategy: Quasi-experimental design Non-Equivalent Groups Pretest/Posttest Treatment A Online Flashcards Vocabulary* Pre-Assessment Vocabulary Post-Assessment • Individual Study • Maintain Study Log Paper Flashcards Treatment B *Vocabulary = 66 Greek and Latin root words

4. Background: What are root words? • Root words are essentially language building blocks that are rearranged to form more complex terms. • Knowing root words (along with prefixes and suffixes) empowers learners to interpret unfamiliar terms by looking at each word’s parts (IRA/NCTE, 2007). • In science, Latin and Greek root words are essential for understanding many terms and concepts (White, 2009a). • For Example: • Photo: light • Tropo: turn; change • Syn: with; together • -thesis: an arranging • -ism: the process of

5. Background: Why should students study root words? • Knowledge of root words promotes talking and thinking like a scientist by helping students categorize and discuss new information (Snow, 2008). • A substantial body of research indicates that reading comprehension is directly related to vocabulary instruction (Pressley, 2001). • Learning root words improves spelling because patterns become recognizable (IRA/NCTE, 2007). • When learning vocabulary, direct instruction works. Seeing words in context isn’t sufficient for meaningful learning about concepts (Marzano, Pickering, & Pollock, 2001).

6. Background:Why use flashcards in the study? • Paper flashcards are a familiar method for learning vocabulary. Online flashcards are a relatively new way to study terms. • The students in this particular educational setting are accustomed to independent learning. Classroom-based vocabulary instruction wasn’t feasible, but using flashcards on an individual basis was reasonable. • Two characteristics of effective vocabulary instruction include learning parts of words and students playing with the words in a variety of activities (DeAngelis, 2007). Online flashcard web sites generally incorporate games, self-tests, and other activities to promote learning. This SLIP was designed to assess whether students using online flashcards would learn more root words than students who studied with traditional paper flashcards.

7. Learning Context Minnesota New Country School – A modern one-room schoolhouse! • Project-based; no letter grades, bells, or traditional classrooms • Approximately 115 students in grades 6-12 • Students grouped into eight advisories consisting of multiple developmental/age levels (up to 15 students per advisory) • Each student has a workspace and computer

8. Participant Profile Study participation was voluntary. Students who completed the Science Vocabulary Builder Project earned 0.1 science credit (I. History & Nature of Science: A. Scientific World View). • 30 students signed up; 28 students started the Project. • 23 students completed all requirements to earn credit for participating. Characteristics of the 23 participants: Ethnic Populations 20 White, non-Hispanic 2 Bi-racial (1 Blk/Wht; 1 unknown) 1 Hispanic Exceptionalities 1 Physical and Other Health Disability 2 Emotional/Behavioral Disorder 1 Autism Spectrum

9. Method Overview • PHASE ONE: Development • Literature Review • Preparation • Promotion and Sign-up • PHASE TWO: Data Gathering • 1st Meeting: Student Survey; Proposal Forms • 2nd Meeting: Pre-Assessment • Assign Treatment Groups; Distribute Study Materials • 3rd Meeting: Post-Assessment; Collect Study Logs and Proposal Forms • PHASE THREE: Data Analysis & Project Closure • Data Analysis • Return Assessments and Signed Proposal Forms to Students/Advisors • Recognition • Synthesis and Reflection

10. Phase One: Development • Literature Review • Personal interest and experience in teaching Greek and Latin root words for science literacy (college students) • Consulted Nichole Kotasek, Cooperating Teacher / Science Educator • Preparation • Developed list of root words • Wrote Student Survey, Pre-Assessment, Post-Assessment, and Study Log • Made flashcard sets (online at Quizlet.com; printed and cut paper sets from the online list). Terms and definitions were consistent with assessments. • Promotion and Sign-up • Town meeting, daily announcements (print and web), staff meeting, regular interactions with students. One week promotion and sign-up period.

11. Phase Two: Data Gathering • 1st Meeting • Student Surveys completed • Instructions for writing Proposal Forms (required for credit) • 2nd Meeting • Pre-Assessment completed • Distributed Study Materials and Study Logs • Students were semi-randomly assigned to either Treatment A (online) or Treatment B (paper). • 3rd Meeting • Post-Assessment completed • Study Logs and Proposal Forms collected

12. Treatment A: Online Flashcards Left: Note options for vocab games and tests Below: Flashcard, set to show both sides at once

13. Treatment B: Paper Flashcards

15. Phase Three: Data Analysis & Closure • Data Analysis • Scored Pre- and Post-Assessments • Entered data (scores, Study Logs) – MS Excel • Determined patterns and correlations • Credit for Participation • Signed off on Proposal Forms and returned them to advisors • Returned Pre- and Post-Assessments to students • Recognition during Presentation Night at MNCS • Two students received the “Vocabulary Science Builder Award” for demonstrating the most growth (% change). • Synthesis and Reflection

16. Assessment Measures Student Survey: Accessing student learning styles and attitudes toward learning vocabulary

17. Assessment Measures Pre-assessment: Determining existing vocabulary knowledge Matching items were developed using guidelines from Stiggins (2005).

18. Assessment Measures • Post-assessment: Quantifying change in vocabulary knowledge • The instructions, test structure, and presentation of the Post-Assessment were identical to those of the Pre-Assessment. • The terms that comprised each of the four lists remained the same. • However, the sequence in which the terms appeared on each list was modified for the Post-Assessment. • Note: Participants did not have access to their Pre-Assessment until after the Post-Assessment was completed and collected. Both assessments were scored at the end of the study to minimize potential researcher bias that could occur during daily interactions with students.

19. Learning Results Revisiting the Project Purpose: To assess whether students using online flashcards to study vocabulary words experience larger gains than students using traditional paper flashcards. Research Questions: Q1. Which study method was associated with the greatest growth in student scores on the Vocabulary Assessment? Q2. What was the relationship between students’ scores on the Pre-Assessment and the percent change they demonstrated at the end of the Project? Q3. What was the relationship between amount of study time reported and percent change in scores on the Vocabulary Assessments?

20. Q1. Which study method was associated with the greatest growth in student scores on the Vocabulary Assessment? On average, students who studied with online flashcards improved their scores 29.7% more than students who studied with paper flashcards. Students who reported that they did not study at all showed, on average, a 16% decrease in score. n=10 n=10 n=3 Mean Percent Change for All 23 Participants = +97.9%

21. Q2. What was the relationship between students’ scores on the Pre-Assessment and the percent change they demonstrated at the end of the Project? • Students with lower scores on the Pre-Assessment showed larger percent gainsthan students whose Pre-Assessment scores were higher.* • This is a weak correlation, however, because: • The scores of two students showing dramatic improvement > 400% were data outliers. • Students whose Pre-Assessment scores were in the middle range (24-40) tended to show very modest gains (mean=41.5%; n=8). *Note: Only one student achieved a perfect score (66) on the Post-Assessment, and none did so on the Pre-Assessment.

22. Q3. What was the relationship between amount of study time reported and percent change in scores on the Vocabulary Assessments? The data indicate no correlation between the amount of study time reported and percent change in vocabulary score. Self-reported data can be unreliable. Observations made during this Project indicated that a number of students did not record their study time accurately or consistently.

23. Analysis and Interpretation Average percent change in scores indicated that studying with online flashcards helped students learn more words than using paper flashcards. • BUT: Individual scores and percent gains for each treatment were highly erratic and showed no clear or statistically significant trends. • This is probably the result of uncontrolled (spurious) variables and sources of error such as: • Unstructured study methods within each treatment • Differences in actual study time or intensity • Studying with others (social learning) vs. studying independently • Differences in students’ developmental levels, abilities , or motivations • Inconsistencies in how/where/when the Assessments were completed (e.g., distractions, possible cheating)

24. Revisiting the Learning Objectives: Students who complete the Science Vocabulary Builder Project will… • Learn meanings associated with Latin and Greek root words frequently used in science terminology • Most students, regardless of the type of flashcard used, improved their score on the Assessment. On average, they nearly DOUBLED the number of word roots correctly defined (mean= +97.9%). • Become more confident in their ability to define unfamiliar words • See anecdotal evidence on the next slide.

25. A Vignette About New Confidence R. was a Science Vocabulary Builder Project participant in 7th grade. Our Science Reading Group read the following selection from a ChemMatters article* on Yellowstone geysers and hot springs: R. exclaimed: “Hey! Thermo means heat, and philum is…um, WAIT! I know this…philo…means love, so it’s bacteria that loves heat! And chlora sounds like chlorophyll, that green stuff or something with leaves, right? Then there’s acido…is that like acid?” HE GOT IT - Root words are powerful! *http://portal.acs.org/portal/PublicWebSite/education/resources/highschool/chemmatters/archive/WPCP_012409 Like other photosynthetic bacteria, plants, and algae, this new type of bacterium, called CandidatusChloracidobacteriumthermophilum, uses chlorophyll molecules to capture sunlight, but in a way different than other bacteria.

26. Revisiting the Learning Objectives: Students who complete the Science Vocabulary Builder Project will… • Learn meanings associated with Latin and Greek root words frequently used in science terminology • Become more confident in their ability to define unfamiliar words • Develop increased awareness of their personal studying preferences • Statements on students’ Study Logs reflected self-awareness about what methods were personally desirable or unappealing. For example: • “I’m so jealous – I wish that I was assigned to the computer flashcards!” • “I hate it - looking at the computer gives me headaches…I really don't like it. I have a really hard time learning them because I can't separate the ones I know from the ones I don't.”

27. Revisiting the Learning Objectives: Students who complete the Science Vocabulary Builder Project will… • Learn meanings associated with Latin and Greek root words frequently used in science terminology • Become more confident in their ability to define unfamiliar words • Develop increased awareness of their personal studying preferences • Learn about online flashcards, a new tool to help them build vocabulary knowledge for any subject • Based on their experiences in this Project, students from both treatment groups indicated a desire to use online flashcards and activities in the future. Direct instruction could be helpful for showing students how to use new techniques; this would encourage them to try new approaches that might improve learning.

29. Towards more rigorous research… To determine whether there is a causal relationship between study method utilized and increased knowledge of root words, the following changes are recommended: • Larger sample to permit true randomization. The current design involved some selection during treatment group assignment, a practice that threatens internal validity. • Increased control of spurious variables (via structure of research design) to strengthen internal validity. • Multiple types of measures (to determine convergent construct validity). • Re-administer existing Assessment at a later date to determine test-retest correlation (reliability/stability).

30. In a nutshell: • Most students learned Greek and Latin root words. • Some students have already recognized the value of this knowledge for interpreting unfamiliar terms. • Several students became motivated to continue developing their root word vocabulary independently. • IN PRACTICE, determining the “best” study method isn’t as important as providing students with a variety of choices for how they can learn. Teachers can promote learning by introducing different strategies and training students how to use them.

31. Reflections • It never ceases to amaze me how so many students are unwilling or unable to read and carefully follow a set of simple, straightforward instructions. Their minds seem so over-stimulated that they can’t focus long enough to comprehend, question, or pursue information that would help them, both immediately and in the long run. How can I encourage students to slow down enough to pay attention? (Jeez, I sound old.) • On that note…How am I going to keep students engaged? It’s not practical to offer whiz-bang-fun-yet-meaningful-and-standards-based learning activities each minute for every student. I’m afraid that I’ll get burned-out trying. It is both an instructional strength (that I can provide quality and engaging learning opportunities) and a weakness (the fact that I want to create an active learning environment for all students so badly that I might kill myself or go broke trying to do so)! • (Continued)

32. Reflections, cont’d • It’s hard to conduct (quasi)experimental research in an unstructured setting like MNCS! No matter how well-prepared, proactive, and flexible I am when planning instructional projects, there will always be unanticipated surprises. Sometimes happy developments, other times disappointments…these are the challenges that drive me to reflect, make changes, grow, and improve as an educator and role model. • As a teacher, my greatest reward is seeing kids becoming excited about using their new knowledge or skills to continue learning independently. The best instruction doesn’t stop at factual recall and content coverage: It teaches each learner ways to learn, gives him or her diverse tools for doing so, and fosters a passion for exploration that goes beyond the classroom. The Science Vocabulary Builder Project wasn’t a model of exemplary research. However, I know it was a success because some students have already put the power of their stronger vocabulary to work in their own education – and are proud of it!

33. References and Acknowledgments Borror, D. J. (1960). Dictionary of word roots and combining forms. Mountain View, CA: Mayfield Publishing. DeAngelis, M. (2007). Effective vocabulary instruction. In  B. Hoffman (Ed.), Encyclopedia of Educational Technology. Retrieved October 18, 2009, from http://coe.sdsu.edu/eet/articles/effectivevocab/start.htm IRA/NCTE. (2007). Prefix, root word, and suffix study sheet. Retrieved October 20, 2009, from http://www.readwritethink.org/lesson_images/lesson1042/study_sheet.pdf Latin roots and derivatives. (2009). Retrieved October 18, 2009, from http://www.vocabulary-lesson-plans.com/latin-roots.html Marzano, R. J., Pickering, D. J., & Pollock, J. E. (2001). Classroom instruction that works. Alexandria, VA: Association for Supervision and Curriculum Development. Pressley, M. (2001, September). Comprehension instruction: What makes sense now, what might make sense soon. Reading Online, 5(2). Retrieved October 20, 2009, from http://www.readingonline.org/articles/art_index.asp?HREF=/articles/ handbook/pressley/index.html

34. References and Acknowledgments, cont’d Root words used frequently in chemistry. (n.d.). Retrieved October 18, 2009, from http://www.csun.edu/science/books/sourcebook/chapters/1vocabulary/resources/chemistry_roots.pdf Snow, C. (2008). What is the vocabulary of science? In A. S. Rosebery & B. Warren (Eds.), Teaching science to English language learners (pp. 71-83). Arlington, VA: NSTA Press. Stiggins, R. J. (2005). Student-involved assessment for learning (4th ed.). Upper Saddle River, NJ: Pearson Education. White, D. S. (2009a). Discussion: It’s all Greek (and Latin) to me. Resource produced for BIOL 100 Teaching Assistants at Minnesota State University, Mankato during spring semester. White, D. S. (2009b). Science vocabulary builder project – MNCS. Quizlet online flashcard set created on October 28, 2009, at http://quizlet.com/1343630/ science-vocabulary- builder-project-mncs-flash-cards/ To the student participants, their advisors, Nichole Kotasek, and Rob Groebner…THANK YOU!