Evaluating Open Educational Resource (OER) Objects

1. Evaluating Open Educational Resource (OER) Objects Rubric VI: Quality of Instructional Tasks and Practice Exercises CC BY Achieve 2013

2. Open Educational Resources: [With Webster’s Definitions] O Open: Containing an open license - no restrictions on remixing or reusing [Having no enclosing or confining barrier; not restricted to a particular group or category of participants] E Educational: Used for teaching and learning [Pertaining to the action or process of educating or being educated] Resource: Object used to support an effort or task [A source of support or aid, especially one that can be readily drawn upon when needed] R

3. Open Educational Resources (OER) • What are OER? • OER are teaching, learning, and research resources that contain an open license. • They provide extraordinary opportunities for educators to freely share knowledge and resources. • They offer great potential for instructional innovation and networks for sharing best practices. • There are millions of OER objects available online.

4. Open Educational Resources (OER) • OER objects may include (but are not limited to): • Images • Applets • Games • Worksheets • Lesson plans • Original-source texts • Assessments • Units • Textbooks • Teacher/learner support materials • NOTE: Any smaller component of a more complex object, that can exist as a stand-alone, alsoqualifies as an “object” (e.g. a unit in a textbook, a lesson in a unit, or an activity in a lesson).

5. The Achieve OER Evaluation Rubrics • How and why were the Achieve OER Rubrics created? • Educators using the vast system of sharing need a method for filtering OER to meet the needs of their students. • The purpose of the rubrics is to provide a structure for evaluating an online resource in a systematic, purposeful and comprehensive way. • There are two ways of approaching the evaluation of a resource: • Holistically evaluating the object with a single rating, (similar to the star-rating systems used on Netflix, Yelpand Amazon) • Separately evaluating each of the components of quality, as with the Achieve OER Rubrics. • The rubric criteria are based on Achieve protocols used to assist states in alignment and quality review studies. • The Achieve OER Evaluation Tool, hosting the rubrics on OERCommons.org, allows a user to identify, evaluate and sort objects based on the specified essential elements of quality.

6. Applying the Achieve OER Rubrics • How do the Achieve OER Rubrics work? • The rubrics represent an evaluation system for objects found within Open Educational Resources. • They are hosted as an online evaluation tool on the repository, OERCommons.org, but can also be used independently. • They are applied to any content area. • NOTE: At this stage only Common Core Standards for English Language Arts/Literacy and Mathematics are available in the Achieve OER Evaluation Tool. Content standards for History, Social Studies, Sciences, and technical subjects are not yet available. • Each rubric is applied independently to an object. • They rate the potential, not the actual, effectiveness of an object. • They use a five-point scoring system to describe levels of quality.

7. The Achieve OER Evaluation Tool Rubrics • This presentation will focus on Rubric VI: • Rubric I. Degree of Alignment to Standards • Rubric II. Quality of Explanation of the Subject Matter • Rubric III. Utility of Materials Designed to Support Teaching • Rubric IV. Quality of Assessment • Rubric V. Quality of Technological Interactivity • Rubric VI. Quality of Instructional Tasks and Practice Exercises • Rubric VII. Opportunities for Deeper Learning • NOTE: Sometimes one or more of the rubrics may not be relevant to a particular object. In those cases, a rating of N/A is appropriate.

8. Rubric VI: Quality of Instructional Tasks and Practice Exercises • How and Why is Rubric VI Applied? • Applied to objects that contain instructional tasks and/or sets of practice exercises designed to strengthen skills and knowledge. • The purpose of these exercises is to deepen understanding and to routinize foundational skills and procedures. • When concepts/skills are introduced providing a sufficient number of practice exercises is critical. When integrating skills into complex tasks, the number of exercises is less important than their richness. • Applied to a group of exercises as a single object. • Not applicable if the object does not include practice exercises. Even if one can imagine ways to use the object as practice, if that is not the intention of the object, N/A is the correct rating.

9. Rubric VI: Quality of Instructional Tasks and Practice Exercises • 3: An object is rated superior only if all of the following are true: • The object offers an more than enough exercises needed for the average student to facilitate mastery of the targeted skills, as stated or implied in the object. For complex tasks, one or two rich exercises may be considered more than enough. • The exercises are clearly written and supported by accurate answer keys as applicable. • There are a variety of exercise types and/or the exercises are available in a variety of formats, as appropriate to the targeted concepts and skills. • For example a superior object for practice is a lesson addressing operations with polynomials includes an adequate number of practice problems using all four operations with polynomials. In addition there are extra practice sets designed for warm up, some for students needing remedial practice, and more complex problems for those needing a greater challenge. All practice sets provide varying formats for the problems and all include accurate solutions.

10. Rubric VI: Quality of Instructional Tasks and Practice Exercises 2: An object is ratedstrong if it offers only a sufficient number (for the average student) of well-written exercises to facilitate mastery of the targeted skills, supported by accurate answer keys or scoring guidelines, with little variety of exercise types or formats. For example a sixth grade object addressing conventions of standard English, capitalization and punctuation rules, includes a practice worksheet with an adequate number of exercises to allow for practice for students presently reading at grade level. Answer keys and scoring suggestions are included. This object is considered strong rather than superior because even though it offers adequate practice, it does not provide varying formats or provide for a variety of student learning needs.

11. Rubric VI: Quality of Instructional Tasks and Practice Exercises 1: An object is ratedlimited if it has some, but too few exercises to facilitate mastery of the targeted skills, is without answer keys, and provides no variation in format or type. For example an object is limited for practice in a lesson addressing two-digit multiplication which has a set of five practice exercises. That is too few to facilitate mastery of two-digit multiplicationbut enough to be considered practice.

12. Rubric VI: Quality of Instructional Tasks and Practice Exercises 0: An object is ratedvery weak or no value if exercises provided do not facilitate mastery of the targeted skills, contain errors, or are unsound for other reasons. For example an object addressing division of fractions and mixed numbers has included a set of practice problems. However of the ten problems, none require that the student convert a mixed number to an improper fraction. In addition the answer keys for the practice set includes three with incorrect answers and none show steps in the process. Since the teacher will likely need to create new practice exercises, this object is very weak for Rubric VI.

13. Rubric VI: Quality of Instructional Tasks and Practice Exercises • N/A:This rubric is not applicable(N/A) to an object that does not include practice exercises. • Even though one might imagine ways it could be used in this way, not applicable is the appropriate score if practice is not the intention of the object. • For example many objects are not intended to be used for practice of a skill but rather to introduce a concept, teach a subject, assess learning, etc. When practice is not the intention of the object, Rubric VI is not applicable.

14. Using the Achieve OER Evaluation Tool On OERCommons.org Rubric VI

15. Achieve OER Evaluation Toolon OERCommons.org: Rubric VI • OERCommons.org hosts the rubrics as the Achieve OER Evaluation Tool • NOTE: Specific instructions for Using the OERCommons Website to search for and evaluate objects, using all of the rubrics, can be found in another slide presentation of this training package.

16. Rubric VI: Quality of Instructional Tasks and Practice Exercises Achieve OER Evaluation Tool Using OERCommons.org to evaluate an object for its quality of instructional tasks and exercises: After you have evaluated an object using Rubrics I through V, you should be ready to apply Rubric VI. Begin by clicking on at the end of Rubric V or on the title line for Rubric VI.

17. Rubric VI: Quality of Instructional Tasks and Practice Exercises Achieve OER Evaluation Tool To view a video about Rubric VI click here…

18. Rubric VI: Quality of Instructional Tasks and Practice Exercises Achieve OER Evaluation Tool Hover over the rating numbers to see an abbreviated version of the rating descriptors. Review the object with your focus on the quality of the practice exercises included in the object and select the appropriate rating.

19. Rubric VI: Quality of Instructional Tasks and Practice Exercises Achieve OER Evaluation Tool To download a full text version of the rubrics click here:

20. Rubric VI: Quality of Instructional Tasks and Practice Exercises Achieve OER Evaluation Tool • Across the bottom of Rubric VI you see options to: • Leave a [+ Comment], • Clear rating, • [Save & Go to the Next Rubric] (You can also click on any rubric title to leave Rubric VI, but your ratings will not be saved.)

21. Achieve OER Evaluation Toolon OERCommons.org: Rubric VI Now you are ready to rate the object using the Achieve OER Evaluation Tool’s other rubrics. Rubrics can be used in any order and can be selected by simply clicking on the rubric title. Remember to click after each rubric to ensure that your work is saved. You can tell if your work was saved by the symbol at the beginning of each rubric title: indicates not yet rated or rating was not saved indicates your rating has been saved

22. Slides developed by Achieve: www.achieve.org 1400 16th Street, NW / Suite 510 Washington, DC 20036