Chapter 10: Tests

1. Chapter 10:Tests Mitch Card EDLT 520: Thea Hand March 26, 2007

2. Chapter 10: Tests • Essentially, this chapter reviews the basics of using computers when developing tests • in a computer test center, not necessarily in a classroom, • The information is at least 5 years old, and • Is based on the use of Testing Software developed by major companies (like for the SAT, ACT, etc). • But, much of this is adaptable today to normal people like us. Things to be covered will include but not be limited to: • Construction • Administering • Content • Simulations • Test Security

3. Defined According to Wikipedia, the word test can be defined … • As a “test and experiment (that) form parts of the scientific method, to verify or falsify an expectation with an observation”. • As “a student assessment in education, (also called an "examination" or "exam") that serves to assess or measure students' performance, knowledge or skills”.

4. Before we begin – a word about NCLB and Testing If you would like to know more about NCLB and testing check-out the following: U.S. Department of Education site ERIC An interesting site: Fair Test: The National Center for Fair and Open Testing

5. An excerpt from NCLB:On what subjects are students tested and when? (This may be accomplished using computers in the next year or two) • No Child Left Behind requires that, by the 2005-06 school year, each state must measure every child's progress in reading and math in each of grades 3 through 8 and at least once during grades 10 through 12. In the meantime, each state must meet the requirements of the previous law reauthorizing ESEA (the Improving America's Schools Act of 1994) for assessments in reading and math at three grade spans (3-5; 6-9; and 10-12). By school year 2007-2008, states must also have in place science assessments to be administered at least once during grades 3-5; grades 6-9; and grades 10-12. Further, states must ensure that districts administer tests of English proficiency--to measure oral language, reading and writing skills in English--to all limited English proficient students, as of the 2002-03 school year. • Students may still undergo state assessments in other subject areas (i.e., history, geography and writing skills), if and when the state requires it. No Child Left Behind, however, requires assessments only in the areas of reading/language arts, math and science.

6. Computerized Test Construction • Advantages: • Tests can be designed to be printed or taken on-line, • Questions can be re-ordered for each individual so that no two tests have the same order of answers/questions, • Test pools (banks) can be generated by more than one teacher that can be accessed by many teachers at a time - for future reference, • Tests can be generated with little notice, • Most textbook publishers now supply teachers with test banks on CD that can be generated and placed on the Internet, and • The tests can be automatically graded and scores e-mailed to the teacher and the student (immediate feedback).

7. Computerized Test Construction • Disadvantages • Test security (cheating) becomes an important issue, • Teachers may not match appropriate questions to what has actually been taught, • Not all test questions have been produced with the same standards of quality, • Most graded tests must be restricted to a multiple choice, matching or short-answer format (they can not grade essays).

8. Computerized Test Administration Although helpful, there are problems with • having enough computers to test a class at a time, and • if the test structure is confusing or awkward in design, test anxiety could easily be increased, causing poorer results.

9. Factors to consider in any Test Preparation… • What is the purpose of the test, what will it test? • Does the test meet the needs of the test taker? • Does the test actually assess the target objectives of the curriculum? • Is the test length and time allowed appropriate to the level and age of the student?

10. Test Creator Software • Must be user friendly. • Allow the test generator as well as the test taker appropriate control. • Have a ‘safety net’ to prevent un-wanted deletion of answers and data from the program – provide affirmation phrases. • Label tests to make sure the right students are taking the right subject tests. • And, after completion, all new tests should be tested to find errors, unreasonable questions or that the entire testing process works correctly.

11. Basic Computerized Test Requirements • Instructions must be clear and cover all aspects of using the program and technology correctly. • There should be a practice page/question that allows the student to see what will happen while they take the test. • Be prepared to have problems with • Equipment failure, • Accidental termination of the test, and • Loss of answers in the data pool.

12. Simulations • Needs a computer to complete in the first place • Can be flexible enough to test skills - including: • Lab procedures, • Performance tasks, • Equipment operation & management.

13. Adaptive Testing-based on responses of the user. • Usually used after some sort of training or instruction. • Incorrect responses lead the testee to a review of the material. • Correct responses lead the testee to related, more in depth, questions.

14. Admissable Probability Testing • Essentially, the computer testing software has the capacity to grade answers that are only partially complete. • I am not sure why this is in here, but these tests are rarely used. • Examples: • Admissable Probability Measures Testing (APM) • Information Referenced Testing (IRT)

15. To ensure security of tests…. • Do not allow users to have anything to record answers on while taking the test, • Re-develop tests after they are used (using test banks) • Generate new questions and never use used questions again - a lot of work. • Take questions off-line (prevents hacking) • Require passwords to any collected test data

16. So…. So… • Stepping away from the chapter • Most new scholastic textbooks come with test generating software that includes test banks. • Generated tests can be converted to HTML and given on-line (see example– put your e-mail address in the space provided and get your graded test sent to you in your e-mail). • These tests are excellent to get students to prepare for class discussions, or written tests.

17. Sources – interestingly, there were no URL’s listed in the references for this chapter. • Alessi, S.M., and Trollip, S.R. (2001). Multimedia for learning: methods and development (3rd Ed.) Ma: Ally & Bacon. • U.S. Department of Education (n.d.) Stronger accountibility: Testing-frequently asked questions. Retrieved 3/11/07, from http://www.ed.gov/nclb/accountability/ayp/testing-faq.html • Wikipedia (n.d.) Test: from Wikipedia, the free on-line encyclopedia. Retrieved 3/11/07, from http://en.wikipedia.org/wiki/Test • Images and cartoons collected through Google ‘Images’ and the MS Picture Library.

18. Your Questions…. • Does it appear to you that on-line testing could be helpful in your teaching format? Why? • This chapter is mostly referring to computerized tests like the SAT. It has been proposed that state tests of all kinds should be tested on-line. How do you think that will affect test scores, especially for minority groups (ESL, LEP, SpEd)?