Creating Open Educational Resources

Once you've shared your OER, it's available for anyone to find and use. You might be interested in tracking that usage, whether for your own personal interest or perhaps to reinforce the importance of the resource for annual review or promotion and tenure. While there aren't easily comparable metrics, like the journal impact factor, there are ways for you to track how many people are viewing, downloading, or adopting your resource.

Downloads, Views, and other metrics

Certain repositories/databases will provide metrics for all items deposited on the platform. For example, the VCU Institutional Repository, Scholars Compass, provides information on how many times your resource page was viewed and how many times your OER was downloaded. It also provides information on where those views and downloads originated from, so you can demonstrate that your resource has global impact. We recommend including an entry for your OER in Scholars Compass even if it is also included elsewhere to help track those metrics.

Each repository will vary on what it tracks, so we recommend checking the platform to learn more about what they provide. If the platform does not provide any metrics services, you could consider enrolling in external metrics-tracking services like Google Analytics or Altmetric. You'll have to check with both the service and platform to see what is possible.

Self-Reported Adoptions

Another way to track usage of your OER is to ask instructors to share when they've adopted the text. While this does require relying on others to self-report, most in the OER community are happy to take the small steps to share their usage. 

To help encourage self-reporting of adoptions and usage, you could include the request with an email address or perhaps a short form on the homepage of the website and/or in the front matter of the work.

Before beginning the assessment of your OER, you should consider what factors you wish to study and what data collection method will be needed to assess those factors. For example, if your research question is...

• How has the OER impacted student learning outcomes? Your assessment could entail pre- and post-testing of students using the OER and those using the traditional textbook and see how those compare. 
• How has OER usage impacted student success? Your assessment could entail gathering and comparing grades from students in OER and non-OER classes. Alternatively, you could compare DFW (drop, fail, withdrawal) rates for OER and non-OER classes.
• What do students think of the OER? How does their perception compare to commercial textbooks? Your assessment could entail a survey asking students about their perceptions of the resource.

IRB

If you are assessing your resource purely for your own purposes, you do not need to obtain IRB approval. However, if you are interested in publishing and/or presenting the outcomes of your assessment, you will likely need to apply for IRB approval. Contact VCU IRB for more information.

Assessment resources

Most research on OER impact assessed OER based on factors in the COUP Framework as set out by the Open Education Group: cost of education, student success outcomes, patterns of usage of OER, and perceptions of OER.

These resources can help faculty assess the impact of their OER usage: 

Examples of OER Research

Cost savings is often an important part of calculating the impact of your resource. Cost savings, or cost avoidance, is the amount of money students will avoid spending by using your OER instead of a commercial textbook. 

To calculate cost savings, you will need to know: 

• The number of sections using the resource
• The number of students enrolled in those sections. Alternately, the number of students enrolled can replace these first two numbers.
• The cost of the textbook that the OER originally replaced. Most often, the cost of purchasing a new book from the bookstore, or the list price on Amazon, is used. Some in the OER community use $100 as a standard amount for savings to account for the fact that even though a book might cost more than that, not all students would have purchased the title.

Cost Savings for 1 semester = (number of sections) x (average number of students per section) x (cost of book being replaced).

OR

Cost Savings for 1 semester = (cost of students using the OER that semester) x (cost of book being replaced)

Add up each semester's savings for cumulative savings over time. You do not need to adjust the cost of the book, even though the price might change at the bookstore.

The OER Adoption Impact Calculator

The OER Adoption Impact Calculator allows users to estimate the impact of their OER use on both student cost savings and other relevant calculations such as course throughput rate and additional tuition revenue from increased enrollment intensity.