Subscription Scoring

Subscription Scoring

The subscription review process uses a variety of criteria to capture a holistic snapshot of each resource’s value. These criteria include both quantitative and qualitative factors early in the process—to generate a preliminary, draft cancellation list—and extensive qualitative criteria later to shape final cancellation decisions.

Traditionally, libraries have relied almost exclusively on quantitative metrics like cost-per-use to make cancellation decisions. However, as important as the bottom line can be, there’s more to exploration, research, and scholarship than economics. Higher education is also about values like equity, diversity, and openness.

In 2019-20, the Western Libraries implemented a new, multi-pronged methodology to better evaluate our subscriptions in alignment with the university’s values. The first prong involves building a portfolio of criteria that go beyond cost-per-use to capture a more complete picture of each subscription. These criteria include quantitative metrics like annual price increase, as well as qualitative metrics like accessibility to users with disabilities, vendor privacy policies, and publisher commitment to Open Access. Together, these criteria provide a more balanced basis for decision-making, in line with Western’s liberal arts mission. 

The second prong of our new methodology entails acknowledging and accounting for the differences in format and media among our many subscriptions. A 2,000-title journal package is very different from a collection of streaming videos, or an ebook package, or a statistical dataset. To compare them, apples-to-apples, isn’t appropriate. Instead, each format deserves its own custom criteria and scoring rubrics. To that end, Western’s librarians have developed separate scorecards for each of our nine formats of subscription. Each scorecard is different, but each adds up to 100 points--allowing us to make apples-to-oranges comparisons across hundreds of diverse subscriptions.

Throughout summer and fall, Libraries personnel collect data on Western’s hundreds of subscriptions. Each component of the data contributes a number of points towards a resource’s final score. Librarians use the lowest-scoring resources to generate a preliminary, draft cancellation list, accumulating enough titles to offset whatever new subscriptions have been requested. Each subscription’s score will be included on the draft cancellation list.

These scores are just the first stage in our decision-making process and will be supplemented by the critical third prong of our holistic methodology: qualitative feedback gathered throughout fall and winter in consultation with university stakeholders.

A complete list of the quantitative and qualitative criteria used to generate the preliminary scores can be found at the bottom of this page, along with a list of the format categories and an example scorecard.


Subscription Evaluation Criteria

Format Categories