Subscription Scoring

This year, the Libraries reviewed 400+ subscriptions using a variety of criteria to capture a holistic snapshot of each resource’s value.

Traditionally, libraries have relied on quantitative metrics like cost-per-use to make cancellation decisions. However, as important as the bottom line can be, there’s more to exploration, research, and scholarship than economics. Higher education is also about values like equity, diversity, and openness.

Throughout the past year, librarians at Western have been developing a two-pronged methodology to better evaluate our subscriptions in alignment with the university’s values. The first prong involves building a portfolio of criteria that go beyond cost-per-use to capture a more complete picture of each subscription’s value. These criteria include quantitative metrics like annual price increase, as well as qualitative metrics like accessibility to users with disabilities, vendor privacy policies, and publisher commitment to Open Access. Together, these criteria provide a more balanced basis for decision-making, in line with Western’s liberal arts mission.

The second prong of our new methodology entails acknowledging and accounting for the differences in format and media among our many subscriptions. A 2,000-title journal package is very different from a collection of streaming videos, or an ebook package, or a statistical dataset. To compare them, apples-to-apples, isn’t really fair. Instead, each format deserves its own custom criteria and scoring rubrics. To that end, Western’s librarians have developed separate scorecards for each of our nine formats of subscription. Each scorecard is different, but each adds up to 100 points--allowing us to make apples-to-oranges comparisons across 400+ diverse subscriptions.

Throughout summer and fall, the Subscription Task Force collected data on Western’s 400+ subscriptions. Each component of the data contributed a number of points towards a resource’s final score. Ultimately, scores ranged from 6.25 to 96 out of 100 points. The Task Force used the lowest-scoring resources to generate the draft cancellation list, accumulating enough titles to meet the $330,000 reduction goal plus a margin for departmental retention requests. The majority of subscriptions on the list fall below 48 out of 100 points. You can see each subscription’s score in the “Total Review Score” column of the draft cancellation list, as well as the scores for each individual criterion broken out.

A complete list of the quantitative and qualitative criteria can be found at the bottom of this page, along with a list of the format categories, an example scorecard, and graphs showing how Western’s subscriptions scored.