1. Upgrade notes

2. New Features

2.1. Administration

2.1.1. New Latency Tester Tool

The Evergreen Web Staff Client now includes a link to a section under Administration → Workstation called Tests. This page houses a simple tool that can be used to test the latency of the websocket connection between the client and the server (via the opensrf.echo service).

This page displays which Evergreen host server is being queried. Upon hitting the blue "Start Test" button for the first time, it will issue 10 sequentially fired requests in order to get a solid initial average. Clicking the button a second time will just take one more measurement and recalculate the average latency. The results can be copied to clipboard for troubleshooting purposes and also cleared from display.

2.1.2. marc_export --uris option

The marc_export support script now has a --uris option (short form: -u) to export records with located URIs. When used by itself, it will export only records that have located URIs. When used in conjunction with --items, it will add records with located URIs but no items/copies to the output. If combined with a --library or --descendants option, this option will limit its output to those records with URIs at the designated libraries. The best way to use this option is in combination with the --items and one of the --library or --descendants options to export all of a library’s holdings both real and electronic.

2.2. Architecture

2.2.1. Sample Data Includes Surveys

The Concerto sample data set now includes patron surveys, questions, answers, and responses.

2.2.2. Virtual Index Definitions

The practical purpose of Virtual Index Definitions is to supply an Evergreen administrator with the ability to control the weighting and field inclusion of values in the general keyword index, commonly referred to as "the blob," without requiring tricky configuration that has subtle semantics, an over-abundance of index definitions which can slow search generally, or the need to reingest all records on a regular basis as experiments are performed and the configuration refined. Significant results of recasting keyword indexes as a set of one or more Virtual Index Definitions will be simpler search configuration management, faster search speed overall, and more practical reconfiguration and adjustment as needed.

Previous to this commit, in order to provide field-specific weighting to keyword matches against titles or authors, an administrator must duplicate many other index definitions and supply overriding weights to those duplicates. This not only complicates configuration, but slows down record ingest as well as search. It is also fairly ineffective at achieving the goal of weighted keyword fields. Virtual Index Definitions will substantially alleviate the need for these workarounds and their consequences.

  • A Virtual Index Definition does not require any configuration for extracting bibliographic data from records, but instead can become a sink for data collected by other index definitions, which is then colocated together to supply a search target made up of the separately extracted data. Virtual Index Definitions are effectively treated as aggregate definitions, matching across all values extracted from constituent non-virtual index definitions. They can further make use of the Combined class functionality to colocate all values in a class together for matching even across virtual fields.

  • Configuration allows for weighting of constituent index definitions that participate in a Virtual Index Definition. This weighting is separate from the weighting supplied when the index definition itself is a search target.

  • The Evergreen QueryParser driver returns the list of fields actually searched using every user-supplied term set, including constituent expansion when a Virtual Index Definition is searched. In particular, this will facilitate Search Term Highlighting described below.

  • Stock configuration changes make use of pre-existing, non-virtual index definitions mapped to new a Virtual Index Definition that implements the functionality provided by the keyword|keyword index definition. The keyword|keyword definition is left in place for the time being, until more data can be gathered about the real-world effect of removing it entirely and replacing it with Virtual Index Definition mappings.

  • New system administration functions will be created to facilitate modification of Virtual Index Definition mapping, avoiding the need for a full reingest when existing index definitions are added or removed from a virtual field.

Increased use of Metabib Display Fields

We use Metabib Display Fields (newly available in 3.0) to render catalog search results, intermediate metarecord results, and record detail pages. This requires the addition of several new Metabib Display Field definitions, as well as Perl services to gather and render the data.

We also use more Metabib Display Fields in the client. As a result, bibliographic fields will display in proper case in more client interfaces and in Evergreen reports.

Search Term Highlighting

This commit enables Search Term Highlighting in the OPAC on the main search results page, the record detail page, and intermediate pages such as metarecord grouped results page. Highlighting search terms will help the user determine why a particular record (or set of records) was retrieved.

Highlighting of matched terms uses the same stemming used to accomplish the search, as configured per field and class.

This feature will help the user more quickly determine the relevance of a particular record by calling their attention to search terms in context. Lastly, it will help familiarize the user with how records are searched, including which fields are searched as well as exposing concepts like stemming.


A new AngularJS "MARC Search/Facet Fields" interface has been created to replace the Dojo version, and both have been extended to support Virtual Index Definition data supplier mapping and weighting.

Settings & Permissions

The new Virtual Index Definition data supplier mapping table, config.metabib_field_virtual_map, requires the same permissions as the MARC Search/Facet Fields interface: CREATE_METABIB_FIELD, UPDATE_METABIB_FIELD, DELETE_METABIB_FIELD, or ADMIN_METABIB_FIELD for all actions

There is a new template-level global configuration variable in config.tt2 called search.no_highlight which disables highlighting for users of that config.tt2 instance.

Public Catalog

The public and staff catalog will make use of new APIs to identify and display highlight-augmented values for those Display Fields used to render the search result pages, intermediate metarecord constituent pages, and record detail pages. Highlighting of terms will be performed using the application of Template::Toolkit-driven CSS. A generic CSS class identifying a highlighted term, along with CSS classes identifying the search class and each search field will be available for use for customization of the highlighting. A stock CSS template is provided as a baseline upon which sites may expand.

When highlighting is generally enabled, it may be turned on or off on a per-page basis through the use of a UI component which will request the page again without highlighting.


There now exist several new database tables and functions primarily in support of search highlighting. Additionally, the QueryParser driver for Evergreen has been augmented to be able to return a data structure describing how the search was performed, in a way that allows a separate support API to gather a highlighted version of the Display Field data for a given record.

Default Weights

By default, the following fields will be weighted more heavily in keyword searches. Administrators can change these defaults by changing the values in the "All searchable fields" virtual index in the "MARC Search/Facet Fields" interface.

  • Title proper

  • Main title (a new index limited to the words in the 245a)

  • Personal author

  • All subjects

In addition, note indexes and the physical description index will receive less weight in default keyword searches.

Re-ingest or Indexing Dependencies

With the addition and modification of many Index Definitions, a full reingest is recommended. However, search will continue to work as it did before the changes in this commit for those records that have not yet been reingested during that process. Therefore a slow, rolling reingest is recommended.

Performance Implications or Concerns

Because the Metabib Display Fields infrastructure will eventually replace functionality that is significantly more CPU-intensive in the various forms of XML parsing, XSLT transformation, XPath calculation, and Metabib Virtual Record construction, it is expected that the overall CPU load will be reduced by this development, and ideally the overall time required to perform and render a search will likewise drop. It is unlikely that the speed increase will be visible to users on a per-search basis, but that search in aggregate will become a smaller consumer of resources.

2.3. Cataloging

2.3.1. Track Record Merges

When 2 or more bib records are merged, all records involved are stamped with a new merge_date value. For any bib record, this field indicates the last time it was involved in a merge. At the same time, all subordinate records (i.e. those deleted as a product of the merge) are stamped with a merged_to value indicating which bib record the source record was merged with.

In the browser client bib record display, a warning alert now appears along the top of the page (below the Deleted alert) indicating when a record was used in a merge, when it was merged, and which record it was merge with, rendered as a link to the target record.

2.4. Circulation

2.4.1. Alternate Patron Hold Pickup

This feature adds a bit of convenience to a common task: checking out an item on hold to another patron (typically a family member or helper).

When you checkout the item, you will get a pop-up window with warnings associated with this item. The "ITEM_ON_HOLDS_SHELF" message is now expanded to

  • Let you know the name of the person who had placed the hold.

  • Give you the option (in the form of a checkbox) of cancelling the hold placed by the above-named patron. (Checked = Cancel the hold; Unchecked = Leave the hold in place)

The initial value of the checkbox is derived from the circ.clear_hold_on_checkout organizational setting.

If the operator has CANCEL_HOLD privilege, then if the checkbox is checked and the checkout is allowed to proceed, the hold will be cancelled with a note that the item was checked out to another patron.

This feature is available in the browser-based staff client.

2.4.2. New Patron Billing Statement

The Evergreen web staff client now includes a patron billing statement, which summarizes a patron’s bills, credits and payments in a familiar layout. This can be found on the "Statement" tab of the Patron Bill Details page. (From the Patron Bills page, double-click a row to view its details, or choose "Full Details" from the Actions menu.)

2.4.3. Enhanced Billing Timestamp Support

Previously, billings had to make do with a single timestamp attempting to fill two different roles. In the case of an overdue fine, the timestamp represented the end of the fine period for that billing, while for all other fines, the timestamp was merely the time the bill was created. This setup generally worked, but not without confusion, and limited our ability to understand and process the data.

Billings will now have up to three timestamps: a create date, and when applicable, a fine period start and a fine period end. This clarifies and simplifies things like backdating, retrospective fine generation, account balancing for negative balance avoidance, and billing timeline views.

2.4.4. Copy Alerts and Suppression Matrix

The Copy Alerts feature allows library staff to add customized alert messages to copies. The copy alerts will appear when a specific event takes place, such as when the copy is checked in, checked out, or renewed. Alerts can be temporary or persistent: temporary alerts will be disabled after the initial alert and acknowledgement from staff, while persistent alerts will display each time the alert event takes place. Copy Alerts can be configured to display at the circulating or owning library only or, alternatively, when the library at which the alert event takes place is not the circulating or owning library. Copy Alerts can also be configured to provide options for the next copy status that should be applied to an item. Library administrators will have the ability to create and customize Copy Alert Types and to suppress copy alerts at specific org units.

Copy alerts can be added via the volume/creator and the check in, check out, and renew pages. Copy alerts can also be managed at the item status page.

Copy alert types can be managed via the Copy Alert Types page in Local Administration, and suppression of them can be administered via the Copy Alert Suppression page under Local Administration.

2.4.5. Place Multiple Holds At Once

Users with the appropriate permissions now have the ability to place multiple title/metarecords holds at once. This feature is especially beneficial for book clubs and reading groups, which need to place holds on multiple copies of a title.

In order to use the feature:

  • Set the Maximum number of duplicate holds allowed Library Setting (circ.holds.max_duplicate_holds) to a number higher than 1

  • Log in as a user with the CREATE_DUPLICATE_HOLDS

When placing a title or metarecord hold, a Number of copies field will display for these users. This field is not available when placing part, volume or copy holds.

This feature does not change the way in which the system fills holds. The multiple holds will fill in the same way that they would if the user had placed multiple holds separately.

2.4.6. New Notice Columns in Items Out Grid

The grid in the patron "items out" page in the Evergreen web staff client has two new columns indicating the number of notifications generated for a given loan and the date of the most recent notification. These columns will allow circulation staff to better respond to patron questions about whether they were sent notification about an overdue item.

The columns are based on the number of completed Action Trigger events on the loan that have a checkout.due hook. In other words, they would include overdue and courtesy notices.

A new library setting, "Exclude Courtesy Notices from Patrons Itemsout Notices Count", if set will cause the notice count and date fields to exclude courtesy notices.

2.4.7. Patron Email Addresses Now Clickable In Web Staff Client

Adds a mailto link to the patron’s email in their profile so it can be clicked to send and email to the patron. No new settings or permissions are included in this feature.

2.5. OPAC

2.5.1. Copy Location Filter Displays for System Searches

The Shelving Location filter now displays on the advanced search page when a search is scoped to a library system, not just to an individual branch. If a library system is selected as the Search Library, the shelving location limiter will display any shelving location that is owned by the selected system or by the consortium. It will NOT display shelving locations owned by child branches.

2.5.2. Multi-source Attributes

We now allow record attribute definitions to extract data using more than one strategy (XPath, tag+subfield, fixed field, etc.) as long as the values from various sources would, after normalization, have the same shape.

This change allows us to configure multilingual search, by extracting values from both the 008 controlfield and the 041 datafield. Because the values in each can be normalized to the same controlled list (and, in practice, are already from the same normalized value set), OPAC searches can now use normal boolean search semantics to find records with various combinations of language attributes.

E.g., in the concerto test data:

  • keyword: piano item_lang(eng) item_lang(ita)

2.5.3. Optional Display of Badges in OPAC

A new setting controls whether badges (popularity, etc.) are displayed in the OPAC. If you do not wish badges to be displayed, set the ctx.hide_badge_scores setting to "true" in config.tt2.

3. Miscellaneous

3.1. Fixes to patron name/username search indexes

When using pg_restore to restore an Evergreen database, some of the indexes used to speed up patron searches on names and usernames could be lost.

This release fixes the underlying issue and re-creates the indexes in question.

3.1.1. Details

When using pg_restore to restore an affected database, the "unaccent" indexes on actor.usr would not be created due to an unqualified function reference in evergreen.unaccent_and_squash.

The function will be replaced to resolve the search path issue, and the following indexes on actor.usr will be dropped and then re-created:

  • actor_usr_first_given_name_unaccent_idx;

  • actor_usr_second_given_name_unaccent_idx;

  • actor_usr_family_name_unaccent_idx;

  • actor_usr_usrname_unaccent_idx;

This will be done even if the indexes are already present, and may take a few minutes on a database with many patrons.

4. Acknowledgments

The Evergreen project would like to acknowledge the following organizations that commissioned developments in this release of Evergreen:


We would also like to thank the following individuals who contributed code, translations, documentations patches and tests to this release of Evergreen:


We also thank the following organizations whose employees contributed patches:


We regret any omissions. If a contributor has been inadvertently missed, please open a bug at http://bugs.launchpad.net/evergreen/ with a correction.