Evaluating Open Educational Resource (OER) Objects

1. Evaluating Open Educational Resource (OER) Objects Rubric V: Quality of Technological Interactivity CC BY Achieve 2013

2. Open Educational Resources: [With Webster’s Definitions] O Open: Containing an open license - no restrictions on remixing or reusing [Having no enclosing or confining barrier; not restricted to a particular group or category of participants] E Educational: Used for teaching and learning [Pertaining to the action or process of educating or being educated] Resource: Object used to support an effort or task [A source of support or aid, especially one that can be readily drawn upon when needed] R

3. Open Educational Resources (OER) • What are OER? • OER are teaching, learning, and research resources that contain an open license. • They provide extraordinary opportunities for educators to freely share knowledge and resources. • They offer great potential for instructional innovation and networks for sharing best practices. • There are millions of OER objects available online.

4. Open Educational Resources (OER) • OER objects may include (but are not limited to): • Images • Applets • Games • Worksheets • Lesson plans • Original-source texts • Assessments • Units • Textbooks • Teacher/learner support materials • NOTE: Any smaller component of a more complex object, that can exist as a stand-alone, alsoqualifies as an “object” (e.g. a unit in a textbook, a lesson in a unit, or an activity in a lesson).

5. The Achieve OER Evaluation Rubrics • How and why were the Achieve OER Rubrics created? • Educators using the vast system of sharing need a method for filtering OER to meet the needs of their students. • The purpose of the rubrics is to provide a structure for evaluating an online resource in a systematic, purposeful and comprehensive way. • There are two ways of approaching the evaluation of a resource: • Holistically evaluating the object with a single rating, (similar to the star-rating systems used on Netflix, Yelpand Amazon) • Separately evaluating each of the components of quality, as with the Achieve OER Rubrics. • The rubric criteria are based on Achieve protocols used to assist states in alignment and quality review studies. • The Achieve OER Evaluation Tool, hosting the rubrics on OERCommons.org, allows a user to identify, evaluate and sort objects based on the specified essential elements of quality.

6. Applying the Achieve OER Rubrics • How do the Achieve OER Rubrics work? • The rubrics represent an evaluation system for objects found within Open Educational Resources. • They are hosted as an online evaluation tool on the repository, OERCommons.org, but can also be used independently. • They are applied to any content area. • NOTE: At this stage only Common Core Standards for English Language Arts/Literacy and Mathematics are available in the Achieve OER Evaluation Tool. Content standards for History, Social Studies, Sciences, and technical subjects are not yet available. • Each rubric is applied independently to an object. • They rate the potential, not the actual, effectiveness of an object. • They use a five-point scoring system to describe levels of quality.

7. The Achieve OER Evaluation Tool Rubrics • This presentation will focus on Rubric V: • Rubric I. Degree of Alignment to Standards • Rubric II. Quality of Explanation of the Subject Matter • Rubric III. Utility of Materials Designed to Support Teaching • Rubric IV. Quality of Assessment • Rubric V. Quality of Technological Interactivity • Rubric VI. Quality of Instructional Tasks and Practice Exercises • Rubric VII. Opportunities for Deeper Learning • NOTE: Sometimes one or more of the rubrics may not be relevant to a particular object. In those cases, a rating of N/A is appropriate.

8. Rubric V: Quality of Technological Interactivity • How and Why is Rubric V Applied? • “Interactivity” is used broadly to mean that the object responds to the user, or one that behaves differently based on what the user does. • Applied to objects that are designed with a technology-based interactive component, • Used to rate the degree and quality of the interactivity of that component (Not a rating for technology in general but for technological interactivity), • For interactive objects designed for student use, the rubric is applied to the entire object. • For objects designed for teacher use, for example lesson plans, which may include interactive components for use by students, the rubric is applied only to those interactive parts.

9. Rubric V: Quality of Technological Interactivity • When does Rubric V not apply? • Rubric V is not applicable if an object does not have an interactive element. • This is not a rating for technology in general, but only for technological interactivity between the user and the object. • The rubric does not apply to how students interact with each other, but rather how the technology responds to an individual student user. • Does not apply to objects where interaction with the object is limited to, for example, opening a PDF attachment.

10. Rubric V: Quality of Technological Interactivity • 3: An object is ratedsuperior only if all of the following are true: • The object, or interactive component of an object, is responsive to student input in a way that creates an individualized learning experience. This means the object adapts to the user based on what s/he does, or the object allows the user some flexibility or individual control during the learning experience. • The interactive component is purposeful and directly related to learning. • The interactive component of the object is well designed and easy touse, encouraging learner use. • The interactive component of the object appears to function flawlessly on the intended platform. • For example an interactive student-directed game is considered superior for Rubric V because it responds to individual students by increasing the level of challenge when students successfully complete sections of questions and offers hints when incorrect answers are given and moves the student to intervention questions when they are unable to give the correct answer.

11. Rubric V: Quality of Technological Interactivity 2: An object, or interactive component of an object, is ratedstrong if it has an interactive feature that is purposeful and directly related to learning, even if it does not provide an individualized learning experience. Similarly to the superior objects, strong interactive objects must be well designed, easy touse, and function flawlessly on the intended platform. For example an object has a component that reads aloud sections of text, making it easier for some students to stay focused or to hear the voice in a passage of literature. Even though this interactive feature is well designed, purposeful, and directly related to the learning, there is no personal feedback or individualized response. This object is considered strong for Rubric V.

12. Rubric V: Quality of Technological Interactivity 1: An object, or interactive component of an object, is ratedlimited if its interactive element neither relates to nor detracts from learning. This kind of interactive element might be designed to increase student motivation and to build content understanding by rewarding or entertaining the learner and extending the time the user engages with the content. For example even though the interactive element of a student-directed game may motivate the student to maintain the connection with the object, it is still considered limited for Rubric V. The game’s interactive feature includes earning points, leading to achieving levels, as a reward for correct answers. The interactivity component does not connect directly to the subject matter, provides no opportunity to learn from incorrect answers, or, in some cases, even to recognize when an incorrect answer was given.

13. Rubric V: Quality of Technological Interactivity 0: An object, or interactive component of an object, is ratedvery weak or no valueif it has interactive features that are poorly conceived and/or executed. The interactive features might fail to operate as intended, distract the user, or unnecessarily take up user time. For example in a student-directed game a group of clowns tumble across the screen with each correct answer, distracting the student and slowing their progress. This interactive feature is considered very weak or of no value.

14. Rubric V: Quality of Technological Interactivity N/A: This rubric is not applicable (N/A) for an object that does not have an interactive technological element. Rubric V is not applicable if there is no interactive feature in the object or if technological interaction with the object is limited to, for example, opening a user-selected PDF.

15. Using the Achieve OER Evaluation Tool On OERCommons.org Rubric V

16. Achieve OER Evaluation Toolon OERCommons.org • OERCommons.org hosts the rubrics as the Achieve OER Evaluation Tool • NOTE: Specific instructions for Using the OERCommons Website to search for and evaluate objects, using all of the rubrics, can be found in another slide presentation of this training package.

17. Rubric V: Quality of Technological Interactivity Achieve OER Evaluation Tool Using OERCommons.org to evaluate an object for its quality of technological interactivity: After you have evaluated an object using Rubrics I through IV, you should be ready to apply Rubric V. Begin by clicking on at the end of Rubric IV or on the title line for Rubric V.

18. Rubric V: Quality of Technological Interactivity Achieve OER Evaluation Tool To view a video about Rubric V click here…

19. Rubric V: Quality of Technological Interactivity Achieve OER Evaluation Tool Hover over the rating numbers to see an abbreviated version of the rating descriptors. Review the object with your focus on the quality of the technologically interactive feature(s) included in the object and select the appropriate rating.

20. Rubric V: Quality of Technological Interactivity Achieve OER Evaluation Tool To download a full text version of the rubrics click here:

21. Rubric V: Quality of Technological Interactivity Achieve OER Evaluation Tool • Across the bottom of Rubric V you see options to: • Leave a [+ Comment], • Clear rating, • [Save & Go to the Next Rubric] (You can also click on any rubric title to leave Rubric V, but your ratings will not be saved.)

22. Achieve OER Evaluation Toolon OERCommons.org: Rubric V Now you are ready to rate the object using the Achieve OER Evaluation Tool’s other rubrics. Rubrics can be used in any order and can be selected by simply clicking on the rubric title. Remember to click after each rubric to ensure that your work is saved. You can tell if your work was saved by the symbol at the beginning of each rubric title: indicates not yet rated or rating was not saved indicates your rating has been saved

23. Slides developed by Achieve: www.achieve.org 1400 16th Street, NW / Suite 510 Washington, DC 20036