Subscription Evaluation

About Subscription Evaluation

The subscription review process uses a variety of criteria to capture a holistic snapshot of each resource’s value. These criteria include both quantitative and qualitative factors early in the process—to generate a preliminary, draft cancellation list—and extensive qualitative criteria later to shape final cancellation decisions. 

In 2020, the Western Libraries implemented a new, multi-pronged methodology to better evaluate our subscriptions in alignment with the university’s values. This year, we are launching an updated subscription review process which will allow us to respond more efficiently to significant subscription price increases and enable us to better review our large journal packages. The timeline and mechanisms for university-wide feedback on the annual subscription review list will remain the same as in previous years.

Throughout summer and fall, Libraries personnel collect and review data on Western’s hundreds of subscriptions, including cost per use, average annual price increase, and overlap with other resources. Additional qualitative and quantitative data, such as usage for course reserves and support for Open Access, can be collected for subscriptions identified for more in-depth evaluation. Librarians use the data and their subject-area expertise to generate a preliminary, draft cancellation list, based on the target reduction amount in addition to the amount needed to offset whatever new subscriptions have been requested.

This draft list is the first stage in our decision-making process and will be supplemented by qualitative feedback gathered throughout fall and winter in consultation with university stakeholders. 

A list of the quantitative and qualitative criteria used to assess our subscriptions can be found on this page, along with a list of the format categories. 

Subscription Evaluation Criteria

CPU is a commonly used indicator of value for library subscriptions. Calculated using three years of use and the most recent cost data. CPU constitutes 50 out of 100 points on each e-resource scorecard.

Annual increase is a significant factor in the sustainability of library subscriptions, calculated over three years. 

This criterion privileges resources with largely unique content, while flagging e-resources with a high degree of overlap. Duplication is not a criterion across all categories.

Additional Data

 This criterion protects material that cannot be obtained reliably via means other than a subscription. Availability via ILL is not a criterion across all categories.

Curricular relevance is a core value of Western’s library collections; use of materials for course reserves is an indicator of curricular relevance. Use for course reserves is not a criterion across all categories.

Library resources should be accessible to all users, regardless of ability, with minimal mediation; this criterion operationalizes that value by holding vendors accountable for ongoing accessibility improvements. 

Some e-resources limit access to a fixed number of concurrent users, interrupting access to our full user community and providing poor user experience; this criterion holds vendors accountable for providing broader access. Limitations on seats is not a criterion across all categories.

This criterion boosts resources that are highly interdisciplinary or support large departments or programs. 

This criterion refers to use restrictions on e-book platforms, which include limitations on printing, sharing, and downloading, as well as proprietary software and the requirement that users create usernames and passwords to access content. 

As library subscriptions become increasingly unsustainable, Western wishes to acknowledge and reward publishers taking real steps toward a more open scholarly landscape. This criterion gives a small boost to e-resources from publishers who make at least some content truly open. Commitment to Open Access is not a criterion across all categories.

This criterion holds vendors accountable for having transparent privacy policies.

The ability of users to discover content by searching in OneSearch is a key, albeit small, piece of each e-resource’s value. This criterion acknowledges the vendors who make their content easy to find. Indexing is not a criterion across all categories.

This criterion represents the extent to which the resource or vendor operates for the greater good in a marketplace that has increasingly seen the intrusion of corporate market values. Does the resource or provider actively advance—or is it in conflict with—the Libraries’ missions and goals of social justice and equitable access to knowledge? Does the provider demonstrate positive engagement with library values, beyond performative statements? This criterion is not applied across all formats or resources, but rather if and as relevant circumstances emerge. It has the potential to add or remove up to 10 points from any resource’s total score. For more information about the kinds of things that influence the X Factor score, contact one of our subject teams.  

Format Categories

Subscription databases containing exclusively e-books. Content is leased year to year, so upon cancellation all titles are lost. These are distinct from owned e-books, which may be purchased individually or in packages and represent a one-time expenditure.

Packages or databases containing exclusively e-journals. Content is owned, so each year’s subscription fees in effect purchase a new year of content. Upon cancellation, content up to the cancellation date is retained (except for any content from prior to the original subscription start date).

Subscription databases containing exclusively full-text content, whether journals, monographs, conference proceedings, gray literature, etc. Content is leased year to year, so upon cancellation all titles are lost.

Subscription databases containing a mix of full-text content and abstracting/indexing, whether journals, monographs, conference proceedings, gray literature, etc. Content is leased year to year, so upon cancellation all titles, including abstracts/indexing, are lost.

Subscription databases containing exclusively abstracting/indexing, i.e. that provide a means of discovering content but not the content itself. This can include abstracting/indexing for journals, monographs, conference proceedings, gray literature, etc. The database is leased year to year, so upon cancellation all discovery is lost.

Databases that provide full-text access to a single title or reference work, e.g. an online encyclopedia, dictionary, or handbook. Purchase models for full-text reference works vary: in some cases, content is leased year to year and upon cancellation all material is lost; in other cases, content is owned and each year’s payment secures perpetual access to another year of material.

Databases consisting primarily of streaming music or video. Content is leased year to year, so upon cancellation all content is lost.

Databases that contain non-textual (or other non-traditional) scholarly material, e.g. images, chemical structures, data, etc. Because it does not follow standard bibliographic patterns, the content from these e-resources is more difficult to compare (or replace) on a one-to-one basis, and may not be available through ILL. Content is typically leased year to year, so upon cancellation all titles are lost.

Single e-journal titles purchased outside of packages. Content is owned, so each year’s subscription fees in effect purchase a new year of content. Upon cancellation, content up to the cancellation d ate is retained (except for any content from prior to the original subscription start date).