Subscription Scoring
About Subscription Scoring
The subscription review process uses a variety of criteria to capture a holistic snapshot of each resource’s value. These criteria include both quantitative and qualitative factors early in the process—to generate a preliminary, draft cancellation list—and extensive qualitative criteria later to shape final cancellation decisions.
Traditionally, libraries have relied almost exclusively on quantitative metrics like cost-per-use to make cancellation decisions. However, as important as the bottom line can be, there’s more to exploration, research, and scholarship than economics. Higher education is also about values like equity, diversity, and openness.
In the past few years, the Western Libraries implemented a new, multi-pronged methodology to better evaluate our subscriptions in alignment with the university’s values. The first prong involves building a portfolio of criteria that go beyond cost-per-use to capture a more complete picture of each subscription. These criteria include quantitative metrics like annual price increase, as well as qualitative metrics like accessibility to users with disabilities, vendor privacy policies, and publisher commitment to Open Access. Together, these criteria provide a more balanced basis for decision-making, in line with Western’s liberal arts mission.
The second prong of our methodology entails acknowledging and accounting for the differences in format and media among our many subscriptions. A 2,000-title journal package is very different from a collection of streaming videos, or an ebook package, or a statistical dataset. To compare them, apples-to-apples, isn’t appropriate. Instead, each format deserves its own custom criteria and scoring rubrics. To that end, Western’s librarians have developed separate scorecards for each of our nine formats of subscription. Each scorecard is different, but each adds up to 100 points--allowing us to make apples-to-oranges comparisons across hundreds of diverse subscriptions.
Throughout summer and fall, Libraries personnel collect data on Western’s hundreds of subscriptions. Each component of the data contributes a number of points towards a resource’s final score. Librarians use the lowest-scoring resources to generate a preliminary, draft cancellation list, accumulating enough titles to offset whatever new subscriptions have been requested. Each subscription’s score will be included on the draft cancellation list.
These scores are just the first stage in our decision-making process and will be supplemented by the critical third prong of our holistic methodology: qualitative feedback gathered throughout fall and winter in consultation with university stakeholders.
A complete list of the quantitative and qualitative criteria used to generate the preliminary scores can be found on this page, along with a list of the format categories.
Subscription Evaluation Criteria
CPU is a commonly used indicator of value for library subscriptions. Calculated using three years of use and the most recent cost data. CPU constitutes 50 out of 100 points on each e-resource scorecard.
This criterion protects material that cannot be obtained reliably via means other than a subscription. Availability via ILL is not a criterion across all categories.
Curricular relevance is a core value of Western’s library collections; use of materials for course reserves is an indicator of curricular relevance. Use for course reserves is not a criterion across all categories.
Annual increase is a significant factor in the sustainability of library subscriptions, calculated over three years.
Library resources should be accessible to all users, regardless of ability, with minimal mediation; this criterion operationalizes that value by holding vendors accountable for ongoing accessibility improvements.
Some e-resources limit access to a fixed number of concurrent users, interrupting access to our full user community and providing poor user experience; this criterion holds vendors accountable for providing broader access. Limitations on seats is not a criterion across all categories.
This criterion boosts resources that are highly interdisciplinary or support large departments or programs.
This criterion refers to use restrictions on e-book platforms, which include limitations on printing, sharing, and downloading, as well as proprietary software and the requirement that users create usernames and passwords to access content.
This criterion privileges resources with largely unique content, while flagging e-resources with a high degree of overlap. Duplication is not a criterion across all categories.
As library subscriptions become increasingly unsustainable, Western wishes to acknowledge and reward publishers taking real steps toward a more open scholarly landscape. This criterion gives a small boost to e-resources from publishers who make at least some content truly open. Commitment to Open Access is not a criterion across all categories.
This criterion holds vendors accountable for having transparent privacy policies.
The ability of users to discover content by searching in OneSearch is a key, albeit small, piece of each e-resource’s value. This criterion acknowledges the vendors who make their content easy to find. Indexing is not a criterion across all categories.
This criterion represents the extent to which the resource or vendor operates for the greater good in a marketplace that has increasingly seen the intrusion of corporate market values. Does the resource or provider actively advance—or is it in conflict with—the Libraries’ missions and goals of social justice and equitable access to knowledge? Does the provider demonstrate positive engagement with library values, beyond performative statements? This criterion is not applied across all formats or resources, but rather if and as relevant circumstances emerge. It has the potential to add or remove up to 10 points from any resource’s total score. For more information about the kinds of things that influence the X Factor score, contact one of our subject teams.
Format Categories
Subscription databases containing exclusively e-books. Content is leased year to year, so upon cancellation all titles are lost. These are distinct from owned e-books, which may be purchased individually or in packages and represent a one-time expenditure.
Packages or databases containing exclusively e-journals. Content is owned, so each year’s subscription fees in effect purchase a new year of content. Upon cancellation, content up to the cancellation date is retained (except for any content from prior to the original subscription start date).
Subscription databases containing exclusively full-text content, whether journals, monographs, conference proceedings, gray literature, etc. Content is leased year to year, so upon cancellation all titles are lost.
Subscription databases containing a mix of full-text content and abstracting/indexing, whether journals, monographs, conference proceedings, gray literature, etc. Content is leased year to year, so upon cancellation all titles, including abstracts/indexing, are lost.
Subscription databases containing exclusively abstracting/indexing, i.e. that provide a means of discovering content but not the content itself. This can include abstracting/indexing for journals, monographs, conference proceedings, gray literature, etc. The database is leased year to year, so upon cancellation all discovery is lost.
Databases that provide full-text access to a single title or reference work, e.g. an online encyclopedia, dictionary, or handbook. Purchase models for full-text reference works vary: in some cases, content is leased year to year and upon cancellation all material is lost; in other cases, content is owned and each year’s payment secures perpetual access to another year of material.
Databases consisting primarily of streaming music or video. Content is leased year to year, so upon cancellation all content is lost.
Databases that contain non-textual (or other non-traditional) scholarly material, e.g. images, chemical structures, data, etc. Because it does not follow standard bibliographic patterns, the content from these e-resources is more difficult to compare (or replace) on a one-to-one basis, and may not be available through ILL. Content is typically leased year to year, so upon cancellation all titles are lost.
Single e-journal titles purchased outside of packages. Content is owned, so each year’s subscription fees in effect purchase a new year of content. Upon cancellation, content up to the cancellation d ate is retained (except for any content from prior to the original subscription start date).