2020-21 Subscription Scoring

2020-21 Subscription Scoring

This year, the Libraries is reviewing more than 100 databases and large packages, 200 single-subscription journals, and over 2500 journals from publisher packages. This review uses a variety of criteria to capture a holistic snapshot of each resource’s value. These criteria include both quantitative and qualitative factors early in the process—to generate a preliminary, draft cancellation list—and extensive qualitative criteria later to shape final cancellation decisions.

Traditionally, libraries have relied almost exclusively on quantitative metrics like cost-per-use to make cancellation decisions. However, as important as the bottom line can be, there’s more to exploration, research, and scholarship than economics. Higher education is also about values like equity, diversity, and openness.

Last year, the Western Libraries implemented a new, multi-pronged methodology to better evaluate our subscriptions in alignment with the university’s values. The first prong involves building a portfolio of criteria that go beyond cost-per-use to capture a more complete picture of each subscription. These criteria include quantitative metrics like annual price increase, as well as qualitative metrics like accessibility to users with disabilities, vendor privacy policies, and publisher commitment to Open Access. Together, these criteria provide a more balanced basis for decision-making, in line with Western’s liberal arts mission.

The second prong of our new methodology entails acknowledging and accounting for the differences in format and media among our many subscriptions. A 2,000-title journal package is very different from a collection of streaming videos, or an ebook package, or a statistical dataset. To compare them, apples-to-apples, isn’t appropriate. Instead, each format deserves its own custom criteria and scoring rubrics. To that end, Western’s librarians have developed separate scorecards for each of our nine formats of subscription. Each scorecard is different, but each adds up to 100 points--allowing us to make apples-to-oranges comparisons across hundreds of diverse subscriptions.

Throughout summer and fall, Libraries personnel collected data on Western’s hundreds of subscriptions. Each component of the data contributed a number of points towards a resource’s final score. Librarians will use the lowest-scoring resources to generate a preliminary, draft cancellation list, accumulating enough titles to meet the $164,000 reduction goal plus a margin for departmental retention requests. Each subscription’s score will be included on the draft cancellation list.

These scores are just the first stage in our decision-making process and will be supplemented by the critical third prong of our holistic methodology: qualitative feedback gathered throughout fall and winter in consultation with university stakeholders.

A complete list of the quantitative and qualitative criteria used to generate the preliminary scores can be found at the bottom of this page, along with a list of the format categories and an example scorecard.

Subscription Evaluation Criteria

Format Categories

Example Scorecard