Evaluating Open Educational Resource (OER) Objects

1. Evaluating Open Educational Resource (OER) Objects Rubric VII: Opportunities for Deeper Learning CC BY Achieve 2013

2. Open Educational Resources: [With Webster’s Definitions] O Open: Containing an open license - no restrictions on remixing or reusing [Having no enclosing or confining barrier; not restricted to a particular group or category of participants] E Educational: Used for teaching and learning [Pertaining to the action or process of educating or being educated] Resource: Object used to support an effort or task [A source of support or aid, especially one that can be readily drawn upon when needed] R

3. Open Educational Resources (OER) • What are OER? • OER are teaching, learning, and research resources that contain an open license. • They provide extraordinary opportunities for educators to freely share knowledge and resources. • They offer great potential for instructional innovation and networks for sharing best practices. • There are, literally, millions of OER objects available online.

4. Open Educational Resources (OER) • OER objects may include (but are not limited to): • Images • Applets • Games • Worksheets • Lesson plans • Original-source texts • Assessments • Units • Textbooks • Teacher/learner support materials • NOTE: Any smaller component of a more complex object, that can exist as a stand-alone, also qualifies as an “object” (e.g. a unit in a textbook, a lesson in a unit, or an activity in a lesson).

5. The Achieve OER Evaluation Rubrics • How and why were the Achieve OER Rubrics created? • Educators using the vast system of sharing need a method for filtering OER to meet the needs of their students. • The purpose of the rubrics is to provide a structure for evaluating an online resource in a systematic, purposeful and comprehensive way. • There are two ways of approaching the evaluation of a resource: • Holistically evaluating the object with a single rating, (similar to the star-rating systems used on Netflix, Yelpand Amazon) • Separately evaluating each of the components of quality, as with the Achieve OER Rubrics. • The rubric criteria are based on Achieve protocols used to assist states in alignment and quality review studies. • The Achieve OER Evaluation Tool, hosting the rubrics on OERCommons.org, allows a user to identify, evaluate and sort objects based on the specified essential elements of quality.

6. Applying the Achieve OER Rubrics • How do the Achieve OER Rubrics work? • The rubrics represent an evaluation system for objects found within Open Educational Resources. • They are hosted as an online evaluation tool on the repository, OERCommons.org, but can also be used independently. • They are applied to any content area. • NOTE: At this stage only Common Core Standards for English Language Arts/Literacy and Mathematics are available in the Achieve OER Evaluation Tool. Content standards for History, Social Studies, Sciences, and technical subjects are not yet available. • Each rubric is applied independently to an object. • They rate the potential, not the actual, effectiveness of an object. • They use a five-point scoring system to describe levels of quality.

7. The Achieve OER Evaluation Tool Rubrics • This presentation will focus on Rubric VII: • Rubric I. Degree of Alignment to Standards • Rubric II. Quality of Explanation of the Subject Matter • Rubric III. Utility of Materials Designed to Support Teaching • Rubric IV. Quality of Assessment • Rubric V. Quality of Technological Interactivity • Rubric VI. Quality of Instructional Tasks and Practice Exercises • Rubric VII. Opportunities for Deeper Learning • NOTE: Sometimes one or more of the rubrics may not be relevant to a particular object. In those cases, a rating of N/A is appropriate.

8. Rubric VII: Opportunities for Deeper Learning • Rubric VII is applied to objects designed to engage learners in at least one of the following deeper learning skills: • Think critically and solve complex problems • Work collaboratively • Communicate effectively • Learn how to learn • Reason abstractly • Construct viable, logical arguments and critique the reasoning of others • Apply discrete knowledge and skills to real-world situations • Construct, use, or analyze models • Not applicable when an object does not appear to be designed to provide the opportunity for deeper learning, even though one might imagine how it could be used to do this.

9. Rubric VII: Opportunities for Deeper Learning • 3: An object is ratedsuperior only if all of the following are true: • At least three of the deeper learning skills from the list are required in the object. • The object offers a range of cognitive demand that is appropriate and supportive of the material. • Appropriate scaffolding and direction are provided. • For example a superior object for deeper learning addresses peer conferencing in the process of revision of written drafts. It provides adequate scaffolding and direction for both the teacher and the student in its requirements that students work collaboratively, communicate their thinking effectively in sharing their work with peer partners, critique the reasoning of peer partners, and construct a logical argument supporting their final draft.

10. Rubric VII: Opportunities for Deeper Learning 2: An object is rated strongif it includes one or two deeper learning skills identified in this rubric. For example an eighth grade object is considered strong in its opportunities for deeper learning since it involves solving a complex mathematical problem that requires critical thinking and abstract reasoning skills to reach a solution.

11. Rubric VII: Opportunities for Deeper Learning 1: An object is ratedlimited if it includes one deeper learning skill identified in the rubric but is missing clear guidance on how to tap into the various aspects of deeper learning. For example as part of a fifth grade writing lesson, an object includes a provision for learners to collaborate, but the process and product are unclear. Since there is a suggestion to use collaboration, Rubric VII would apply but without clear explanation of how the collaboration is to take place, the object is limited in its opportunities for deeper learning.

12. Rubric VII: Opportunities for Deeper Learning • 0: An object is ratedvery weakif it appears to be designed to provide some of the deeper learning opportunities identified in this rubric, but it is not useful as presented. • For example an object describes an activity based on poorly formulated problems and has unclear directions. It is unlikely that this activity will lead to skills like critical thinking, abstract reasoning, constructing arguments, or modeling. It would be considered very weak for Rubric VII.

13. Rubric VII: Opportunities for Deeper Learning N/A: This rubric is not applicable(N/A) to an object that does not appear to be designed to provide the opportunity for deeper learning, even if one might imagine how it could be used to do this. For example an object focuses on practicing long division with two-digit divisors through the use of multiple exercises. Because it does not clearly describe the process to be used in presenting the exercises, the object does not provide clear opportunities for deeper learning and Rubric VII would be not applicable.

14. Using the Achieve OER Evaluation Tool On OERCommons.org Rubric VII

15. Achieve OER Evaluation Toolon OERCommons.org • OERCommons.org hosts the rubrics as the Achieve OER Evaluation Tool • NOTE: Specific instructions for Using the OERCommons Website to search for and evaluate objects, using all of the rubrics, can be found in another slide presentation of this training package.

16. Rubric VII: Opportunities for Deeper Learning Achieve OER Evaluation Tool Using OERCommons.org to evaluate an object for its opportunities for deeper learning: After you have evaluated an object using Rubrics I through VI, you should be ready to apply Rubric VII. Begin by clicking on the title line for Rubric VII or on at the end of Rubric VI.

17. Rubric VII: Opportunities for Deeper Learning Achieve OER Evaluation Tool To view a video about Rubric VII click here…

18. Rubric VII: Opportunities for Deeper Learning Achieve OER Evaluation Tool Hover over the rating numbers to see an abbreviated version of the rating descriptors. Review the object with your focus on opportunities deeper learning and select the appropriate rating.

19. Rubric VII: Opportunities for Deeper Learning Achieve OER Evaluation Tool To download a full text version of the rubrics click here:

20. Rubric VII: Opportunities for Deeper Learning Achieve OER Evaluation Tool • At the bottom of Rubric VII you see options to: • Leave a [+ Comment], • Clear rating

21. Rubric VII: Opportunities for Deeper Learning Achieve OER Evaluation Tool Make sure you have rated all the rubrics in the Evaluation Tool. Return to any rubrics by simply clicking on the rubric title. You can tell if any are not saved by the symbols at the left of the rubric title. indicates not yet rated or rating was not saved indicates your rating has been entered and saved

22. Rubric VII: Opportunities for Deeper Learning Achieve OER Evaluation Tool When all the rubrics have been rated saved you can view the final results by clicking on [Save & View Results] for Rubric VII or on [View Results].

23. Achieve OER Evaluation Tool: Finalize Your Review • After all seven rubrics have been rated and saved: • You have clicked on [Save & View Results] at the end of Rubric VII, or • You have clicked on [View Results] at the bottom of the Evaluation Tool • You will see your ratings and the overall average ratings for all previous users.

24. Achieve OER Evaluation Toolon OERCommons.org: Rubric VII If you are satisfied click on [Finalize OER Review]. If you want to change any ratings you can click on [Go Back & Change Your Scores]. Always FINALIZE your results when you are finished rating all seven rubrics.

25. Slides developed by Achieve: www.achieve.org 1400 16th Street, NW / Suite 510 Washington, DC 20036