1 / 48

Donald J. Leu, Elena Forzani, Clint Kennedy, and Cheryl Burlingame Neag School Of Education University Of Connecticut

Online Research and Comprehension Assessment:  How Connecticut Students in Rich and Poor School Districts Perform on New Aspects of the Common Core State Standards. Donald J. Leu, Elena Forzani, Clint Kennedy, and Cheryl Burlingame Neag School Of Education University Of Connecticut.

skyler
Download Presentation

Donald J. Leu, Elena Forzani, Clint Kennedy, and Cheryl Burlingame Neag School Of Education University Of Connecticut

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Online Research and Comprehension Assessment: How Connecticut Students in Rich and Poor School Districts Perform on New Aspects of the Common Core State Standards Donald J. Leu, Elena Forzani, Clint Kennedy, and Cheryl Burlingame Neag School Of Education University Of Connecticut Transition to the Connecticut State Standards and System of Assessments Third Annual Connecticut Assessment Crown Plaza, Cromwell Rocky Hill, CT August, 2012 PowerPoint available at: http://www.education.uconn.edu/assessment/

  2. Common Core State Standards:Three Fundamental Changes to English Language Arts • Emphasis on Non-Fiction • Emphasis on Higher-Level Thinking • The First Appearance of Online Research and Comprehension Skills

  3. Online Research and Comprehension are Blended Into the ELA Standards • New Literacies: Online Research and Comprehension: • Identify a problem • Locate information • Evaluate information • Synthesize information • Communicate information “To be ready for college, workforce training, and life in a technological society, students need the ability to gather, comprehend, evaluate, synthesize, and report on information and ideas, to conduct original research in order to answer questions or solve problems, and to analyze and create a high volume and extensive range of print and nonprint texts in media forms old and new.” (Introduction, p. 4)

  4. Online Research and Comprehension Skills Appear in at Least Half of the Reading and Writing Anchor Standards

  5. CEO Why?The Nature of Work Has Changed Upper Level Management Upper Middle Level Management Middle Level Management Line Supervisors Workers The “General Motors” Model of Economic Management • Command and control • Lower levels of education required. • Wasted intellectual capital • Highly inefficient • Lower productivity • Little innovation • Little need for higher level and creative thinking. Wasted intellectual capital

  6. In a Flattened World: Opportunities Expand but Competition Increases How do economic units increase productivity? Flatten The Organization into Problem Solving Teams Team Team Team Team Team Greater Intellectual Capital Use = Greater Productivity These teams take full advantage of their intellectual capital to the extent their education system has prepared them for this. • Define problems • Locate information • Critically evaluate information • Synthesize and solve problems • Communicate solutions

  7. Which Tool Has Been Used By Economic Units To Increase Productivity And Compete? Recent productivity gains are due to using the Internet to share information, communicate, and solve problems (van Ark, Inklaar, & McGuckin, 2003; Friedman, 2005; Matteucci, O’Mahony, Robinson, & Zwick, 2005). The Internet Team Team Team Team Team Online Research and Comprehension Define problems Locate information Evaluate information Synthesize- solve problems Communicate solutions

  8. These New Literacies Appear at Every Grade Level in ELA in the CCSS Kindergarten (WS 6) With guidance and support from adults, explore a variety of digital tools to produce and publish writing, including in collaboration with peers. Grade 3 (WS 8) ...gather information from print and digital sources; take brief notes on sources and sort evidence into provided categories. 8

  9. These New Literacies Appear at Every Grade Level in ELA in the CCSS Grade 6 (RS 7) Integrate information presented in different media or formats (e.g., visually, quantitatively) as well as in words to develop a coherent understanding of a topic or issue. WritingAnchor Standard 6 Use technology, including the Internet, to produce and publish writing and to interact and collaborate with others. 9

  10. An Important Challenge:How Do We Assess Online Research and Comprehension?

  11. PIs Donald J. Leu, The University of Connecticut JonnaKulikowich, The Pennsylvania State University Nell Sedransk, National Institute of Statistical Sciences Julie Coiro, University of Rhode Island Graduate Research Assistants Elena Forzani, Clint Kennedy, and Cheryl Burlingame, The University of Connecticut Scientific Advisory Board P. David Pearson, The University of California, Berkeley Irwin Kirsch, Educational Testing Service Rand Spiro, Michigan State University Elizabeth Stage, Lawrence Hall of Science, Berkeley Glenn Kleimann, Friday Institute, NCSU

  12. Project Goals • Develop assessments of online research and reading comprehension in three different formats (ORCA-MC, ORCA – Open, ORCA-Closed). • Evaluate each instrument's internal assessment characteristics. • Evaluate the extent to which performance on each format is associated with various student variables. • Evaluate the practicality of each assessment format in the eyes of key education decision makers.

  13. ASSESSMENT TOPICS AND CONTEXTS

  14. SCORE POINT SYSTEM

  15. SCORE POINT SYSTEM

  16. Reading to Evaluate Information Online

  17. Reading to Synthesize Information Online

  18. Reading and Writing to Communicate Information Online

  19. Corresponding Multiple Choice Format

  20. Research Question • Is there an achievement gap in the ability to read on the Internet and conduct research between students who attend rich and poor school districts?

  21. Reading Achievement Gaps Between Rich and Poor (NAEP) % At or Above Proficient • In 2005 and 2011, 25 scale point gap among 8th graders - rich and poor. • In 2011 - “At or above proficient” • Rich = 55% Poor = 23% • In 2011 “Below basic” • Rich = 10% Poor = 32% 55% 23% 270 245 275 250 % Below basic 32% 10%

  22. 90/10 Income Achievement & Black-White Gaps in Reading, 1943-2001 Cohorts *Adapted from: Reardon, S.F. (2011). The widening academic achievement gap between the rich and the poor: New evidence and possible explanations. In R. Murnane & G. Duncan (Eds.), Whither Opportunity? Rising Inequality and the Uncertain Life Chances of Low-Income Children. New York: Russell Sage Foundation Press. Average Difference in S.D. Units on National Assessments Years

  23. A Central Problem for New Literacies Research:The Stability of Assessment Contexts ?

  24. Research in Online Reading Comprehension • A problem-based research and reading process • Elements: locate, evaluate, synthesize, communicate • Online reading ≠ offline reading • Leu, Castek, Hartman, Coiro, Henry, Kulikowich, & Lyver, 2005. • Coiro & Dobler, 2007 • Coiro, 2011.

  25. Research on Internet Access at Home and at School: Poor Students May Be Doubly Disadvantaged • Less home access among poor: • 32.1% adoption rate when family income less than $15,000 • 89.6% when more than $150,000 • (US Dept. of Commerce, 2011) • Instructional access at school may be affected by state tests: • Greater pressure in poor schools to teach to state tests - with no aspects of online reading comprehension included.

  26. Purpose • To determine if a significant online research and reading achievement gap existed between students attending rich and poor school districts. • Evaluate a new, more stable assessment format: ORCA-Closed • Part of the ORCA Project (IES)

  27. METHODS

  28. Participants • 256 7th-grade students attending a rich and a poor school district in CT. • West Town: a wealthy school district (Level B in CT District Reference Groups) • East Town: a poor school district (Level H in CT District Reference Groups)

  29. School District Differences

  30. Population Sample Students who completed both research activities: Asthma (wiki) and Energy Drinks (email)

  31. Internet Use Survey • Home Access • Do you have a computer at home? (Yes, No) • How many computers in your home are connected to the Internet? (0, 1, 2, 3 or more) • School Instruction • How often have you been required to use the Internet for a school assignment? (Never, Less than once a week, Once a week or more.)

  32. Offline Reading Comprehension • Connecticut Mastery Test (CMT) • Subtests • Reading for literary experience • Reading for information • Reading to perform a task • Degrees of Reading Power • Reliability = .94

  33. ORCA-Closed • Two types of topics in science (human body systems) • Learn more about: Are energy drinks heart healthy? • Take a position: Can Chihuahua dogs cure asthma?

  34. Scoring • 4 score points in each of four areas: locate, evaluate, synthesize, communicate. • Each area had 3 process score points and 1 product score points. • Process points were subjectively scaled prior to the assessment by a panel of experts. • A total of 16 score points for each topic. 32 points were possible.

  35. RESULTS

  36. The ORCA-Closed Is a Reliable Instrument • Combined Cronbach’s Alpha = .89 • Energy Drinks = .83 • Asthma = .79

  37. A Significant Achievement Gap Existed in Offline Reading(CT Mastery Test: Reading) • t = 13.81 p = .000 • eta squared = .511 (large) • West Town (Rich) Mean = 282.60(SD = 41.54) • East Town (Poor) Mean = 215.10(SD = 31.07)

  38. A Significant Achievement Gap Also Existed in Online Reading and Research: ORCA-Closed • t = 11.22, p = .000 • eta squared = .394 (large) • West Town Mean = 15.00 (SD=5.69) • East Town Mean = 7.65 (SD=4.39)

  39. ...Even When an ANCOVA Analysis Was ConductedCovariates: Offline Reading + Prior Knowledge • F (1,234) = 15.84, p = .000 • partial eta squared = .063 • (medium) • West Town (rich) adjusted mean = 12.96 • East Town (poor) adjusted mean = 10.27

  40. West Town Students Had More Computers Connected to the Internet at Home How many computers in your home are connected to the Internet? • Pearson Chi-Square = 23.40, df = 3, p = .000

  41. Differential Access to Instruction Appeared at SchoolHow often have you been required to use the Internet for a school assignment? • Pearson Chi-Square = 18.67, df = 2, p = .000

  42. DISCUSSION • Developed a reliable, new form of assessing online reading comprehension, ORCA-Closed, with greater stability. • The gap in reading between students attending rich and poor school districts appears even greater for online reading than for offline reading. • Mean scores nearly 2x greater for students in the richer district.

  43. DISCUSSION • An indication that the gap may be due to home access as well as, perhaps, unintended consequences of current state assessment practices, especially in poorer school districts.

  44. The Take Aways? • There appears to be an additional achievement gap, central to CCSS -- the ability to research and read online. • Any society that professes egalitarian ideals should be concerned about this gap in the 21st century. • We continue work on the ORCA Project and value partnerships with CSDE and school districts for this work.

  45. Online Research and Comprehension Assessment: How Connecticut Students in Rich and Poor School Districts Perform on New Aspects of the Common Core State Standards Donald J. Leu, Elena Forzani, Clint Kennedy, and Cheryl Burlingame Neag School Of Education University Of Connecticut Transition to the Connecticut State Standards and System of Assessments Third Annual Connecticut Assessment Crown Plaza, Cromwell Rocky Hill, CT August, 2012 PowerPoint available at: http://www.education.uconn.edu/assessment/

More Related