Should Students Be Allowed to Use a Word Processor for the State Writing Assessment?

Should Students Be Allowed to Use a Word Processor for the State Writing Assessment? PTE 893 – Leadership in Instructional Technology Final Research Project Nancy Edwards

Table of Contents • Trends • Reasons Computers Help Improve Writing • Issues in Teaching Writing • Effects of Writing with Computers • Importance of High Stakes Testing • What research Says • Issues Affecting Scores • Reasons to Use Computers in Testing • Transitioning • Ways to Bridge the Gap • References

Trends in Education • Trend #1: The presence of computers in schools has increased exponentially • Teachers use computers to teach writing • “Regular use of computers for writing over extended periods of time can lead to significant improvements in student writing skills” (Russell & Plati, 2001)

Trends in Education • Trend #2: State testing programs for increased accountability have been put in place, in some form, in all 50 states. • Most writing assessments ban the use of computers and word processors for the completion of these tests, except for special needs students (NE state update 10)

Trends in Education • Conclusion: These two trends seem to be working against each other

Trends in Education • As the use of computers in schools, homes, and businesses increases, at least two problems arise: • For students who do most of their writing via computer, writing longhand is an artificial rather than a real world task. • Paper and pencil tests may yield underestimates of the actual abilities of students who are accustomed to writing via computer

Trends in Education • “The situation is analogous to testing the accounting skills of modern accountants, but restricting them to the use of an abacus for calculations.” (Russell & Haney, 2000)

Reasons Why Computers May help Improve Student Writings • Computers help with • Generating text • Students can easily create neat, formatted text • Recopying is not necessary when making revisions • Students can focus on the flow of thoughts rather than on grammar, spelling, and punctuation.

Reasons Why Computers May help Improve Student Writings • Computers help with • Generating text • Revising Text • Neat appearance makes it easier to critique • Ability to alter text while preserving appearance • Impermanence helps students see writing as a fluid process • Students who write on computer return more often to their text and make more revisions (Russell and Plati, 2001)

Reasons Why Computers May help Improve Student Writings • Computers help with • Generating text • Revising Text • Collaboration and Audience • Easy to share writing on screen • Computer itself may act as an audience

Reasons Why Computers May help Improve Student Writings • Computers help with • Generating text • Revising Text • Collaboration and Audience • Feedback • Spelling and grammar checkers

Issues Which Deserve Attention from Those Who Design Writing Programs Utilizing New Technologies • According to The Educational Technology Journal , (1991) issues which deserve attention from those who design writing programs utilizing new technologies include:

Issues #1. Has the school identified in explicit terms why and how the word processor might radically alter the quality of student writing? • Word processor provides greater fluidity and flexibility than other writing technologies • Word processor makes thinking more powerful, as long as students are taught how to use it in that way

Issues #2. Is the staff prepared to employ the word processor as an idea processor? • In-services will probably be a requirement

Issues #3. Do students have sufficient access to computers so they can develop their ideas appropriately over time?

Issues #4. Is form more important than substance? • Stop confusing quantity with quality • Good writing is complete when the ideas have been expressed with clarity and impact

Issues #5. How will we know that writing is improving?

Effects of Using Computers to Write Compositions (as opposed to paper-and pencil) • According to a study by Russell (1999) • Compositions on computer produced longer responses that received higher scores • Students made more revisions • Compositions were of better quality • Students reported very positive attitudes toward computer based editing and writing

Importance of High Stakes Testing • High stakes testing is used to • Promote students to higher grades • Grant high school diplomas • Identify and sanction or reward low and high performing schools

What Research Says About Using Computers as Opposed to Pencil-and-Paper Testing for Writing • One study (Russell&Plati, 2002) found that using pencil and paper, for students accustomed to writing using a computer, underestimated their performance level for four to eight points on an eighty point scale.

What Research Says About Using Computers as Opposed to Pencil-and-Paper Testing for Writing • Another study (Russell& Haney, 2000) showed that when students wrote on paper, only 30% performed at a “passing” level; when they wrote on computer, 67% “passed.”

What Research Says About Using Computers as Opposed to Pencil-and-Paper Testing for Writing • Because nearly ten million students took some type of state sponsored writing test last year, and over half of the students nationwide use word processors in school, these results suggest that state paper and pencil tests may be underestimating the abilities of millions of students annually.

Issues Affecting Scores • Achievement scores on Writing Tests completed on the computer are affected by #1. How much practice students have had writing on the computer

Issues Affecting Scores • Achievement scores on Writing Tests completed on the computer are affected by #2. Keyboarding skills • For students who keyboard 20 wpm or more, performing writing tests on paper substantially underestimates their level of achievement • For slower keyboarders, performing these tests on computer adversely affects their performance (Russell, 1999)

Issues Affecting Scores • Students do best when testing most nearly matches normal practice • Until all students have access to and use computers regularly, open-ended tests administered via a single medium will likely underestimate performance of students accustomed to working in the alternate medium (Russell, 1999)

Reasons to Use Computers in Testing • Authenticity of testing would increase • Students would be better able to demonstrate their best work • Assessment would mirror real-world experience • More valid measures of student writing would be available to judge student and school achievement

Transitioning • As state level accountability tests begin to transition from paper administration to computer administration, several issues arise: (Russell & Plati, 2002) Issue #1. Until all students are accustomed to writing on computers, a better understanding of the extent to which the mode of administration affects student performance at different grade levels must be developed

Transitioning • As state level accountability tests begin to transition from paper administration to computer administration, several issues arise: (Russell & Plati, 2002) Issue #2. Given the many computing devices available, the effect of performing open ended items on desktop computers needs to be contrasted with performance on cheaper and more portable writing devices such as eMates and AlphaSmarts.

Transitioning • As state level accountability tests begin to transition from paper administration to computer administration, several issues arise: (Russell & Plati, 2002) Issue #3. Before testing programs offer students the option of performing tests on paper or on computer, the extent to which handwritten versus computer printed responses influence raters’ scores needs to be explored

Transitioning • As state level accountability tests begin to transition from paper administration to computer administration, several issues arise: (Russell & Plati, 2002) Issue #4. Administrative procedures for administering tests on computers in schools must be developed

Ways to Bridge the Gap • Schools could decrease the amount of time students spend working on computers so they do not become accustomed to writing on computers • Not likely to happen

Ways to Bridge the Gap • States could eliminate paper and pencil testing and have students perform open ended tests on computers • Not likely to happen soon • Could place some students at a disadvantage

Ways to Bridge the Gap • Administrators of tests could offer students the option of performing open ended tests on paper or on computer • Students would need to be able to recognize their own strengths • Would require additional efforts from those administering and scoring the tests

Ways to Bridge the Gap • Those interpreting scores should recognize the limitations of current testing programs • Probably the most reasonable solution in the short term • Does not “solve” the problem • Great emphasis is being placed on these tests by those who do not understand them

References McKenzie, J. (1991, June) From Now On, The educational Technology journal [On-line] 1 (6) Retrieved April 10 from http://www.fno.org/fnojun91.html

References (continued) • Russell, M. (1999, June 8) Testing on computers: A follow-up study comparing performance on computer and on paper. Education Policy Analysis Archives [On-line] 7 (20) Retrieved April 11, 2003 from http://epaa.asu.edu/epaa/v7n20

References (continued) Russell, M., & Haney, W., (1997, January 15) Testing writing on computers: An experiment comparing student performance on tests conducted via computer and via paper-and-pencil. Education Policy Analysis Archives [On-line], 5 (3) Retrieved April 11, 2003 from http://epaa.asu.edu/epaa/v5n3.html

References (continued) Russell, M., & Haney, W., (2000, March 28) Bridging the gap between testing and technology in schools. Education Policy Analysis Archives [On-line], 8 (19) Retrieved April 10, 2003 from http://epaa.asu.edu/v8n19.html

References (continued) Russell, M., & Plati, T. (2001) Effects of computer versus paper administration of a state-mandated writing assessment. [On-line] Retrieved April 9, 2003, from http://ericsys.uncg.edu/TCRecord%20-%20Writing%20Assessment.htm

References (continued) Russell, M., & Plati, T. (2002, May 29) Does it matter with what I write? Comparing performance on paper, computer and portable writing devices. Current Issues in Education [On-line], 5 (4) Retrieved April 10, 2003 from http://cie.ed.asu.edu/volume5/number4/index.html

References (continued) (2002, September) STARS, Update#10, Statewide writing assessment for 2002-2003 [On-line] Retrieved April 12, 2003 from http://www.nde.state.ne.us/stars/pdf/update10.pdf

Should Students Be Allowed to Use a Word Processor for the State Writing Assessment?