Critiquing Assessments Within & For Web-Based Curriculum

1. Critiquing Assessments Within & For Web-Based Curriculum EDT 560 – May 2, 2006 Applying Technology to Assessment in Education For Gail Garthwait ~ By Betsy Bass

2. Table of Contents • Table of Contents • Presentation Site Map I. Introduction II. What are Assessments and How do You Know if they are Good Enough – or Right – for You? III. Critiquing the Assessments Within and For Web-Based Curriculum – List of Process & Product Assessment Tools Index Page: to 4-Types of Web-Based Curriculum Projects A. WebQuests B. On-Line Collaborative Projects C. Student Publication D. Other Web-Based Curriculum Projects IV. Discussion Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

3. A. Build-in Assessment Strategies VS Stand-Alone Assessment Tools B. Rubrics 1. What are Rubrics 2. Where do you find them? 3. When should I use rubrics? 4. Should a rubric be used for everything taught? 5. Might Rubric use encourage kids to NOT do their best? 6. How would an educator build a rubric? C. What makes a good assessment? V. Conclusion (VI. References & VII. Appendix A. – ISTE Standards MetaRubric are in paper, not PowerPoint presentation.) Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

4. PowerPoint Presentation Site Map (Inspiration) Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

5. Introduction “Traditionally, assessment has been viewed as evaluation, although assessment can provide more information about students than just grades. Assessment becomes evaluation when judgments are made, judgments that determine, pass, or certify (Eisner, 1994 in Shambaugh & Magliaro, 2006, p. 120).” Where might we find curriculum and assessment tools today? Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

6. Curriculum & Assessment Tools Sources • Text Books • Educational Books, Magazines, and other Publications • Teacher-Created Material ~ Shared • School Districts • The World-Wide Web or Internet • This Presentation will review Assessments Within and For Web-Based Curriculum Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

7. 4 Types of Web-Based Curriculum • Review of four types of web-based curriculum ~ discussed in Critical Basics: Web-Based Curriculum Projects: Examples • & their built-in and created for assessments at http://www.mediaworkshop.org/lifesciences/ls_workshops/webbased.html • They are: web quests, online collaborative projects, student publications, and other web-based curriculum projects. • Teachers who do not have pre-planned evaluations: • Can use Web-based built-in assessments - if they exist • Can find many relevant and appropriate ones on the web • But whatever they choose, they must critique the tool for its quality first. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

8. Is the Assessment Tool that You Chose - The Best One for your Needs, Goals, & Students? • The next question - • Is the assessment tool of good quality? • And who decides that? • Critiques and reviews of company-developed assessments are on the market and many of the web-based curriculum has peer review that helps an educator make decisions. • Often however ~ the decision is left with the teacher and his or her colleagues to determine what is best for one’s own particular goals, needs, and students. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

9. What are Assessments Exactly? & What are some of the Assessment Tools? • The Glossary Definition of assessmentin Shambaugh and Magliaro (2006, p.307) states that: • “Assessment is “from the Latin word, assidere, meaning “to sit beside someone.” Assessment serves multiple purposes, providing teachers with a judgment on how students have preformed, are performing, or are likely to perform. Includes a family of methods to provide information on students, including placement, diagnostic, formative, and summative information (Linn & Gronlund, 2000).” Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

10. What are some of the tools used for formative and summative assessments? • Many of the tools in the “family of methods” can be found in textbooks, educational material or on the web. • A table -shown on the next slide – lists a variety of Process and Product Assessment Tools Source:http://eduscapes.com/sessions/ptools/assess.htm • Article:Engaging Technology Tools Projects by Annette Lamb (7/99. Updated 10/99). Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

11. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

12. 4 Types of Web-Based Curriculum • Review of four types of web-based curriculum ~ discussed in Critical Basics: Web-Based Curriculum Projects: Examples • & their built-in and created for assessments • They are: • webquests • online collaborative projects • student publications • other web-based curriculum projects • Discussion Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

13. Web-Based Curriculum Projects & Their Assessments • WebQuests - What are they? “An inquiry-oriented activity in which most or all of the information used by students is online. By providing links necessary to complete the quest, the student is able to focus on the material rather than spend time looking for it. The five-part WebQuest (Introduction, Task, Resources, Process, Evaluation, and Conclusion) promotes critical thinking at the levels of analysis, synthesis and evaluation. (4/19/2006)www.netnet.org/students/student%20glossary.htm.” Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

14. Webquests • One of the best WebQuests sites is: the WebQuest Portal by Bernie Dodge (http://webquest.org/). • Each WebQuest is developed with the Evaluation as part of the curriculum activity. • The WebQuest Portal lists peer review of each WebQuest: from one to five stars. • Will discuss: • Evaluation Section of a WQ Evaluation Rubric • Review 5-Star/5 WebQuest 1 • Review 5-Star/5 WebQuest 2 • Review 2-Star/5 WebQuest 3 • Review 1-Star/5 WebQuest 4 • Discussion Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

15. El Nino or El No-no (5 stars/5) by Keith Nuthall • http://www.powayschools.com/projects/elnino/ • This WQ included a grading rubric and was aligned to content standards. • The point range was Exemplary, Accomplished, Developing, and Beginning. • The topics covered were Focus/Organization, Support, and Conventions. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

16. Mrs. Christensen's Food Chain Gangs (5 stars/5) by Mrs. Christensen • http://teacherweb.com/IN/PNC/chris/ • The Food Chain WQ did include a rubric and the topics it reviewed were Graphics/Originality, Required Elements, Knowledge Gained, Content Accuracy, and Attractiveness. • Points ranged from 4 (best) to 1. • Aligned to standards. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

17. Journey Through Time (2 stars/5) by Amy Schneider • http://www.longwood.k12.ny.us/lhs/teach/webq/aschneidertime/ • Journey Through Time did not include a rubric in the same format as the two above. The task was to develop a travel brochure of a geologic period that they researched – discussing plants, animals and the environment. The evaluation rates the brochure as A- Excellent, B-Good Job, C and D – with two sentences per grade discussing what is required. For example A: is exemplary, contains accurate information, and wonderful illustrations and D is “Your brochure is missing some information. It has few or no illustrations.” • The Teacher Page included alignment to NYS Science Standards. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

18. Bay Quest (1 star out of 5) by Laurie Rosenberg and Lori Sheridan • http://cte.jhu.edu/techacademy/web/2000/cole/BayQuest/ • Even though this WQ rated lower on the average than #3 – it included an actual rubric with 0-36 points for Title, Link Construction, Link Evaluation, and Style (different value for different tasks). • However, it did not appear to be aligned with standards. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

19. Evaluation Section of Kathy Schrock’sWebquest Evaluation Rubric: http://webquest.sdsu.edu/webquestrubric.html Let’s use this criteria to critique the 4 WebQuests! Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

20. WebQuest Discussion 4-Types • I think, given Kathy Schrock’s definition of what a good evaluation should include, the star rating of the WebQuests by the peer evaluators were fairly reasonable. • The good ones were detailed and clearly expressed what the students were required to do, whereas the poorer-rated ones did not. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

21. Online collaborative projects 4-Types • This category represents any project where a student, class, or group of people works with others on a project. It may be as simple as all sampling water temperature on one day and comparing it – or being part of a major educational and/or environmental project. • In reviewing the projects discussed below under the online collaborative projects section of the Critical Basics paper, I realized that most online collaborative projects do not come with evaluations built-in. • It is the teacher’s responsibility to choose an appropriate evaluation for these endeavors. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

22. Collaboration Projects that I reviewed from the Critical Basics site are: 1. West Logotree: An alphabetical listing of online projects across all grade level ~ Join an On Line Project at http://www.siec.k12.in.us/%7Ewest/online/join.htm • I reviewed several of these projects, but no individual project came with a built-in rubric or evaluation suggestion. • West Logotree did offer: Evaluations: Three rubrics for assessing webpages by students. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

23. Online collaborative projects • 2. “Global Schoolhouse: Find online projects in a variety of content areas across all grade levels.” Once again these collaboration projects seem open-ended for teachers to decide how they want to evaluate their students’ progress. • 3. QESN Projects Around the World: Online projects around the world for students of all ages. Offered in English and French. • There is much help to find a project and get involved ~ but the evaluation sections again seem to be left to the individual teachers. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

24. An example is: “Down the drain: How much water do you use? NEW! (Gives description of project.) • http://www.k12science.org/curriculum/drainproj/information.html • Many projects list the standards with which the project aligns, but does not go as far as offering assessment options. Down the Drain offers the following, but not evaluation ideas. • “Material Available on this Site • Specific instructions for how to collect, submit, and analyze the data are given in the Student Activities section which is linked in the left-hand menu. Data is also available to be submitted and viewed through the Project Data section but please be sure to read the instructions first. Students and teachers are welcome to submit work for publication in the Student Area. The Teacher Areacontains a teacher's guide, a list of curriculum standards that are satisfied by the project, hands-on activities and other related project information. The Reference Material section has a lot of links related to water usage and conservation, and the Online Help section contains a list of online experts you can contact with water related questions as well as contact information for the CIESE project leader.” Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

25. Student publications 4-Types • The web is usedfor many educational uses in the classroom now, including publishing long- and short-term projects, portfolios, writing blogs, publishing magazines, or showcasing work, such as the Maine Lakes Conservancy Institute’s Student Portal, where students write under eight categories about their lake and its watershed during an educational lake ecology program. This can be viewed at: http://www.mlci.org/Students/default.aspx . • Since each participating school has their own program within their school, each teacher devises their own assessment methods. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

26. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

27. Student publications • I reviewed these two forms of student publications under the list for Critical Basics –III. Student Publications: • The Middle Ages: 7th Grade students report about what they learned • Digital Portfolios: High School student use the web to organize their best work, (including teacher's curriculum) Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

28. Student publications: The Middle Ages • The Middle Ages Curriculum had 3 projects. • All 3 projects listed evaluations processes – How the educator would assess the students. • Tools ranged from essays, oral presentations, research papers, and unit tests. • However - once again, for the three projects, no rubrics or assessment tools that would guide the students to do their best or to suggest tools for other teachers, were listed. Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

29. Student publications: Digital Portfolios • Digital Portfolios are a form of showcasing student’s work and their growth overtime ~ that is gaining popularity. • Many rubrics & basic checklists to get students started ~ can be easily found • The digital portfolios linked above (Digital Portfolios) are of a high school class creating their first portfolio and the use of animation and bright colors is predominant - maybe too predominant! Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

30. 4-Types Student publications: Digital Portfolios • Digital or electronic (e-) portfolios usually don’t have rubrics built-in. Using a rubric developed by someone else - or reviewing several and adapting it for one’s own use, seems to be a normal process. • The Meta-Rubric for ISTE Standards is a good one to use • A rubric for evaluating electronic portfolios from the University of Wisconsin (A+ Rubric) can be found at: http://www.uwstout.edu/soe/profdev/eportfoliorubric.html Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

31. Other web-based curriculum projects 4-Types IV. Other Web-based Curriculum Projects Other web-based curriculum projects include curriculum of all topics by teachers posted on the web and multimedia projects – which may include video and hypermedia projects. Theauthors of Critical Basics suggest that teachers watch for two main things: Convention and Content Considerations when critiquing on-line curriculum projects. Here are their suggestions >>> Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

32. Other web-based Curriculum projects • Once again, the assessment tools appear to be up to the educator to choose. • Some of the rubrics that I found that look useful are: • 1. Web Based Learning Environments Assessment Rubric • http://school.discovery.com/schrockguide/pdf/web_based_learning_rubric.pdf • 2. Multimedia Project Rubrichttp://www.ncsu.edu/midlink/rub.mmproj.htm Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

33. Other web-based Curriculum projects • 3.Alaska Department of Education & Early Development: • A Collection of Assessment Strategies • http://www.educ.state.ak.us/tls/frameworks/mathsci/ms5_2as1.htm • 4. Scoring Power Points by Jamie McKenzie • (About good Power Point Presentations – includes a rubric) http://fno.org/sept00/powerpoints.html Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

34. ScienceMysteries – for kids to solve! 4-Types • I also found a site that contained Science Mysteries for teachers to engage their students – especially those who do not usually like science! • Although they do not have assessments attached – they are a wonderful web-based educational source. • Once again, an educator must choose, develop, or adapt their own assessment tool. Look at the Writer Guy, Ken Eklund’s Website – for wonderful ways to get your students involved in solving a scientific mystery! (But create or find your own assessments!) • http://www.writerguy.com/sciencemystery/teachers.htm Contents EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

35. Discussion 4-Types • Some projects, such as WebQuests, include evaluation (rubrics often) as part of their development process, • But the majority of the projects reviewed contained lists of evaluation possibilities of varying quality or no assessment tool suggestions at all. • Collaboration Projects and Digital Portfolios rarely contained evaluation tools and educators depended on stand-alone tools for assessment. • On-line lesson plans and projects, created by teachers for their students, usually included lists of evaluation strategies – but no scoring guidelines such as rubrics. Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

36. Discussion: Built-In Assessment Strategies VS Stand-Alone Assessment Tools • A built-in assessment strategy is one where the curriculum product developer has created an assessment specifically for that product. • A good one includes formative and summative forms of evaluation along with some numerical or content guide such as a rubric or point value to a lab checklist. • If a built-in assessment doesn’t exist - an educator must use a “stand-alone” assessment tool - one of their own, a colleague’s, find one in a published book or manuscript, or download one from the web. • The educator may adapt one or develop a new one consisting of ideas from several that were reviewed. Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

37. Discussion - Rubrics 1. What are Rubrics? 2. Where do you find them? 3. When should I use rubrics? 4. Should a rubric be used for everything taught? 5. Might Rubric use encourage kids to NOT do their best? 6. How would an educator build a rubric? Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

38. Rubrics -1. What are Rubrics? Rubrics • “Web definitions forRubric • In general a rubric is a scoring guide used in subjective assessments. A rubric implies that a rule defining the criteria of an assessment system is followed in evaluation. A rubric can be an explicit description of performance characteristics corresponding to a point on a rating scale. A scoring rubric makes explicit expected qualities of performance on a rating scale or the definition of a single scoring point on a scale.serc.carleton.edu/introgeo/assessment/glossary.html - Definition in context” • The elements of a rubric, as defined in Kathy Schrock’s Rubric to Evaluate WebQuests, are: Overall Aesthetics, Introduction, Task, Process, Resources, and Evaluation – They work together. Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

39. Rubrics - 2. Where do you find them? Rubrics • Built-in to a lesson plan or curriculum project by the developer - who creates it specifically for project or • As a Stand-Alone Assessment Tool ~ which can be found in books, educational reference material, from other teachers, on the Web, or can be developed or adapted from many sources • Teachers can easily save rubrics to be reused and/or adapted for other lesson plans or projects later. • Rubrics can be developed to automatically calculate grades. • Rubrics are fairly versatile and can be easily adapted to fit an educator’s needs and clearly reflect his or her goals. Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

40. Rubrics - 3. When should I use rubrics? Rubrics • It is often helpful to use rubrics when: • it is important for students to be clear about your expectations concerning a particular project, lesson, assignment. • when engaging in project where the underlying concepts or tasks may be particularly complex or elusive to students; rubrics help to define the important components and the expectations for learning for each component. • students are asked to assess their own work or the work of others. • talking with students about their work; rubrics give both the teacher and the student a common language and place to think about how the work might be improved. • evaluating a final project or assignment; the more information a student receives about the level of his or her work, the more he she will learn from it. • when it is important to be able to communicate student progress not only to students, but also to parents, administrators, and the community.” • Source: http://www.thinkinggear.com/tools/using.cfm?t_id=1&ui=4 Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

41. Rubrics - 4. Should a rubric be used for everything taught? Rubrics • No –many different tools exist on paper and on the web to assess formative and summative deficiencies and accomplishments. • Some fit the needs, goals, and students’ abilities better than others. • An educator needs to make these choices. Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

42. Rubrics - 5. Might Rubric use encourage Kids to NOT do their best? (Mike’s Comment) Rubrics • Although most students seem to appreciate the guidance that rubrics provide, educators may find some students who feel that passing is good enough and that a rubric gives them the criteria to know what is needed - to just slide by. • However, the author’s opinion is that the students that may use a rubric to help them achieve the minimum and squeeze by with a just-passing grade, would probably have found another means to reach the same result. • These are my thoughts – what are others?! Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

43. Rubrics - 6. How would an educator build a rubric? Rubrics • Published workbook instruction, teacher-created work, or on Websites such as Creating a Rubric and Writing Rubrics (University of Wisconsin, Teacher Created Rubrics for Assessment, 2006). • Rubrics should include both qualitative and quantitative descriptors – Why? • How would a rubric developer decide on what to weight the most heavily - content versus technical items & why? Comments? Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

44. Discussion: What Makes A Good Assessment? • How do you know if it is a good one? • Who decides this and • For whom are they analyzing the assessment? • 1. Research the credibility of the source • 2. Read peer-reviews, published reviews, or talk to colleagues to learn how other people or organizations rate the product in question • 3. Review rubrics about evaluation of the projects, read magazines on technology and credibility of sources and where to find the best ones • 4. Use own experience to create the best material for his or her own use Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

45. Discussion: What Makes A Good Assessment? • Know before looking for and critiquing assessments for what types of tools and for what functions, one is looking! • The teacher must be the ultimate judge ofwhat is the best tool for his or her curriculum project and the information gleaned from the teacher’s own experience, that of colleagues’ and from reviews by peers and organization will be the supporting material in his or her final decision. Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

46. Conclusion … In the end, teachers must adapt web-based curriculum and their assessments… • To meet the needs of their own course and lesson plans, & • With the special needs of their own students in mind during the structuring of formative and summative evaluations. • Although rubrics may appear to be a panacea for all evaluations, educators must use whatever assessment format best fits theirneeds- Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

47. Conclusion Simple checklists can be a good guide for some things • The educator has • The responsibility of knowing his or her students well enough and his or her own goals for the project or unit well enough, to be able to choose, develop, or adapt appropriate assessments & • To be able to critique them to be able to choose, develop, or adapt the best ones. Contents Contents-p.2 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum