1. Evergreen 3.11.6

This release contains bug fixes improving on Evergreen 3.11.5.

This includes fixes for a critical-importance security issue and two high-importance security issues. Users are advised to upgrade as soon as possible.

1.1. Upgrade notes

The security patches for Bug 2069959 and Bug 2019157 both involve changes to OPAC Template Toolkit templates. If you have customized these templates, perhaps as branding for a specific org unit, please review your customized version to ensure that:

  • the loc_value variable in misc_util.tt2 has non-numeric charcters removed, and

  • the blimit variable in browse.tt2 has the html filter applied.

1.2. Security

  • Patch Insecure direct object reference (IDOR) vulnerability for action trigger output in OPAC list printing feature. (Bug 2070078)

  • Remediates a reflected Cross-site Scripting (XSS) vulnerability in the public catalog browse feature. (Bug 2069959)

  • Mitigate a reflected cross-site scripting (XSS) vulnerability in the public catalog. (Bug 2019157)

1.3. Further reading

To learn more about the mechanics and impact of IDOR and XSS vulnerabilities:

1.4. Contributors

  • Galen Charlton

  • Mike Rylander

  • Jane Sandberg

  • Jason Stephenson

2. Evergreen 3.11.5

This release contains bug fixes improving on Evergreen 3.11.4.

2.1. Upgrade Notes

2.2. Accessibility

  • Increases the visibility of focus outlines in the Angular staff client (Bug 1828463)

2.3. Acquisitions

  • Fixes for SFTP Transfer of EDI Order Data - This repairs the SFTP transfer mechanism so that it should work with most vendors who require usernames and passwords for authentication. To switch from FTP to SFTP, edit the EDI account’s host entry to begin with "sftp://" instead of "ftp://". Check with your EDI vendor before making this change. They may have additional requirements. This fix requires two new Perl modules; see Upgrade Notes above. (Bug 2040514)

  • Fixes EDI error when PO ordering agency has no mailing address (Bug 2016188)

2.4. Administration

  • Fixes bug that could cause the Cash Reports page to display payments for the wrong day. (Bug 2051599)

  • Adds HTML::defang to the opac.patron.custom_css Library Setting (Bug 1869971)

  • Fixes issue with Patrons with Negative Balances interface where a deleted patron with a negative balance would break the interface (Bug 2039725)

  • Sets a useful group of default columns in the Patrons with Negative Balances interface (Bug 2047704)

  • Teaches marc_export to generate an error if given an empty ID file (Bug1329872)

  • Makes it possible to display the org unit ID as a number on Angular record editor forms for editing org units (Bug 2051944)

  • Displays Org Unit ID in Org Config interface. (Bug 2051879)

  • Improves description of the "How to set default owning library for auto-created line item items" Library Setting (Bug 2028095)

  • Silences some "Use of uninitialized value" log entries from catalog search (Bug 2043045)

2.5. Catalog

  • Show the More/Less toggle on facet display in the staff catalog only when a facet has more than five entries. (Bug 2046974)

2.6. Cataloging

  • Fixes problem where "Form" value could not be saved in MARC editor for electronic resources. (Bug 2056204)

  • Improves performance of item refresh after batch editing (Bug 1821094)

2.7. Circulation

  • Adds privilege expiration date column to Group Member Details table (Bug 1779743)

  • Fixes annotate payment when using keyboard navigation (Bug 2047158)

  • Adds help button for Convert change to patron credit on patron bills (Bug 1929596)

2.8. Documentation

  • Improves documentation of Fiscal Propagation and Rollover (Bug 2049774)

  • Updates the version of Antora used to build the documentation (Bug 2036328)

  • Updates documentation to include information about strict barcode (Bug 2053050)

  • Adds documentation for the Angular staff catalog, based on documentation produced by Indiana Evergreen.

  • Allow Windows users to generate the Evergreen manual locally. (Bug 1930099)

2.9. OPAC

  • Adds 245$n and 245$p to the title field in public catalog list CSV download, to better distinguish between multiple titles in the same series. (Bug 1909585)

  • Changes "Account Information and Preferences" in areas of the OPAC to "Personal Information and Preferences" (Bug 1980138)

  • Clarify button text in public catalog New List interface. (Bug 2047589)

  • Fixes placement of Save Notes button in public catalog My Lists page (Bug 2047588)

  • Changes button order in OPAC My Lists (Bug 2047592)

2.10. SIP

  • Adds code to flesh part level holds and issuance holds information in SIP/Patron.pm (Bug 1525394)

2.11. Staff Client

  • Fixes issue with logging out of multiple tab in AngularJS client pages (Bug 2034617)

  • Ensures that both AngularJS and Angular grids use a gear icon for the grid settings menu. (Bug 1803788)

  • Fixes issue where Reports interface would not load if the BitWarden browser plugin is installed (Bug 2052567)

2.11.1. Third Party Services

  • Adds idempotency to Stripe to prevent duplicate payments (Bug 2057948)

2.12. Acknowledgements

We would like to thank the following individuals who contributed code, testing, documentation, and patches to the 3.11.5 point release of Evergreen:

  • Andrea Buntz Neiman

  • Bill Erickson

  • Blake Graham-Henderson

  • Brett French

  • Chris Sharp

  • Christine Morgan

  • Debbie Luchenbill

  • Eva Cerniňáková

  • Galen Charlton

  • Gina Monti

  • Jane Sandberg

  • Jason Boyer

  • Jason Stephenson

  • Jeff Davis

  • Jennifer Pringle

  • Josh Stompro

  • Ken Cox

  • Lena Hernandez

  • Michele Morgan

  • Mike Rylander

  • Robin Fitch

  • Rogan Hamby

  • Spencer Pennington

  • Stephanie Leary

  • Steven Mayo

  • Susan Morrison

  • Terran McCanna

  • Jennifer Weston

3. Evergreen 3.11.4

This release contains bug fixes improving on Evergreen 3.11.3.

3.1. Upgrade Notes

3.2. Bug Fixes

3.2.1. Accessibility

  • Removes extra tab stops when navigating bib record actions in staff client using keyboard (Bug 2052960)

3.2.2. Acquisitions

  • The fund dropdowns for line items and direct charges on purchase orders now display funds that user has permission to use, fixing a regression (Bug 2040637)

3.2.3. Administration

  • Adds a new org unit setting that configures the discovery layer URL opened by the Patron View button on a staff catalog record (Bug 2019207)

  • Fixes misconfigured delete dialogs and adds dialogs where they were missing (Bug 2043508)

3.2.4. Circulation

  • Patch ensures that when "Require Monographic Part when Present" is in effect, that deleted monograph parts are not taken into account when checking whether a title-level hold is possible (Bug 2051557)

  • Fixes bug that allowed one checkout after a patron had reached a group penalty threshold, for example PATRON_EXCEEDS_OVERDUE_COUNT or PATRON_EXCEEDS_CHECKOUT_COUNT (Bug 1890822)

3.2.5. Documentation

3.2.6. General

3.2.7. Public catalog

  • Restores ability to submit basic OPAC search by hitting enter in search input (Bug 2053035)

  • Closes autosuggest dropdown in the public catalog when it loses focus (Big 2054128)

3.2.8. Staff catalog

  • Adds a "Clear Added Content Cache" item to the Other Actions menu in the staff catalog record page (Bug 1939162)

3.3. Acknowledgements

We would like to thank the following individuals who contributed code, testing, documentation, and patches to the 3.11.4 point release of Evergreen:

  • Jason Boyer

  • Dan Briem

  • Galen Charlton

  • Elizabeth Davis

  • Ruth Frasur Davis

  • Jason Etheridge

  • Blake Graham-Henderson

  • Stephanie Leary

  • Tiffany Little

  • Llewellyn Marshall

  • Stephen Mayo

  • Terran McCanna

  • Gina Monti

  • Susan Morrison

  • Andrea Buntz Neiman

  • Jane Sandberg

  • Chris Sharp

  • Jason Stephenson

We would also like to thank the following organizations that sponsored development in this point release:

  • Pennsylvania Integrated Library System (PaILS)

4. Evergreen 3.11.3

This release contains bug fixes improving on Evergreen 3.11.2.

4.1. Upgrade Notes

4.2. Bug Fixes

4.2.1. Accessibility

  • Restores search result pagination link button padding in staff catalog (Bug 2042358)

4.2.2. Acquisitions

  • Fixes issue with Expand All button on Purchase Order pages (Bug 2049654)

  • Fixes ability to use enter key to submit the Acquisitions General Search form (Bug 2049780)

4.2.3. Administration

  • Fixes an erroneous constraint on asset.copy_template (Bug 1384796)

  • Fixes copy stat cat fleshing in SuperCat (Bug 2047587)

  • A fix to reduce size of release tarball by not shipping the Angular build cache (Bug 2048907)

  • Improves the performance of the marc_export support script, particularly when items are included and adds a --batch-size option to the marc_export support script to better control resource usage (Bug 2041364)

4.2.4. Cataloging

  • Fixes an issue in MARC Batch Import / Export where queue data was fetched in parallel, causing excessive pcrud calls (Bug 1945003)

  • Restores the <NONE> selection to prefix and suffix dropdowns in Angular holdings editor Batch Actions (Bug 1998413)

4.2.5. Circulation

  • Adds publication year to Angular Pull List (Bug 2049673)

  • Fixes issue with alerts not displaying upon the initial load of the Patron interface (Bug 1980273)

  • Holds grid can now print / download the Hold Status column (Bug 2051038)

  • Enables clearing the default pickup location in the patron editor (Bug 1939154)

4.2.6. Client

  • Fixes a printing issue on Patrons With Negative Balances admin page (Bug 20471668)

  • Fixes cropping on Reports icon in splash page (Bug 2046970)

  • Adds ability to save the column settings on the patron and item stat cat entries (Bug 2046575)

  • Improvements to Hours of Operation notes field (Bug 2036296)

4.2.7. Developer

  • Removes make_release -x option to build XUL client; make_release now builds the browser client by default (Bug 2051370)

4.2.8. Documentation

4.2.9. General

  • Fixes an issue where the progress bar would not close in Firefox (Bug 1739638)

4.2.10. Public catalog

  • Removes non-functional staff-only "Locate Z39.50 Matches" buttons from OPAC templates (Bug 2021903)

4.2.11. Staff catalog

  • Makes the Hold Status, Current Item, and Requested Item Columns non-sortable on Angular holds grids to avoid errors (Bug 1889133)

  • Fixes a tab display error in the Traditional Staff Catalog (Bug 2047714)

  • Fixes crash when displaying Staff View for a deleted record that has no metarecord mappings (Bug 2039229)

  • Improves speed of searching for and displaying titles that are members of large metarecord sets (Bug 2051708)

4.3. Acknowledgements

We would like to thank the following individuals who contributed code, testing, documentation, and patches to the 3.11.3 point release of Evergreen:

  • Jason Boyer

  • Dan Briem

  • Galen Charlton

  • Garry Collum

  • Jeff Davis

  • Ruth Frasur Davis

  • Bill Erickson

  • Blake Graham-Henderson

  • Stephanie Leary

  • Shula Link

  • Tiffany Little

  • Steven Mayo

  • Terran McCanna

  • Gina Monti

  • Michele Morgan

  • Susan Morrison

  • Andrea Buntz Neiman

  • Mike Rylander

  • Jane Sandberg

  • Chris Sharp

  • Jason Stephenson

  • Josh Stompro

  • Jessica Woolford

5. Evergreen 3.11.2

This release contains bug fixes improving on Evergreen 3.11.1.

5.1. Bug Fixes

5.1.1. Accessibility

  • Auto suggest causes significant accessibility issues for using basic search in some browsers (Bug 1187993)

  • Web Staff Client - accessibility and button names (Bug 1615714)

  • Screen readers skip Angular grid checkbox, row number, and flair icon cells (Bug 2038230)

  • The icon column (status-column) in the patron bills interface needs to convey its meaning to assistive technologies too (Bug 1818086)

  • add_circle_outline and remove_circle_outline icons in Marc Search tab need text alternatives (Bug 2042492)

  • Accessibility Improvements Needed in the Catalog (Bug 1965985)

  • form labels needed in edit-org-unit-setting-dialog (Bug 2009853)

  • Search Preferences: labels and form fields are not associated with each other (Bug 2036313)

  • Report output modal - visual accessibility issues (Bug 2037666)

  • ARIA labels needed in date select, datetime select (Bug 2043421)

  • Increase color contrast on Angular staff tab links (Bug 2043238)

  • Line item checkbox IDs are numeric; need prefix (Bug 2019031)

  • Add aria-describedby when multiple links have identical text (Bug 2016343)

5.1.2. Acquisitions

  • Better way to ID funds at warning or stop percentages (Bug 1984007)

  • Line Item Alert Types Not Scoped in Purchase Orders (Bug 2030820)

  • Alert Type Drop Down Duplicated (Bug 2030821)

  • Unable to search by Line Item- Evergreen Bib ID (Bug 1914297)

  • Actually install the edi pusher and fetcher scripts (Bug 2034969)

  • legacy acq search: lineitem search results can prevent editing copies (Bug 2036840)

  • When Adding a Brief Record If You Double Click "Add Record" Two Line Items Are Created (Bug 2040336)

  • Line item deleting silently fails if selection list is owned by another user (Bug 1966096)

  • Line item alert comments and note text have the same id (Bug 2009093)

  • Drop Downs Don’t Work Well in View/Place Orders (Bug 2040319)

5.1.3. Administration

  • Single Day Emergency Closings Fail to Update Due Dates Correctly (Bug 1818912)

  • Fixed issue loading some AngularJS interfaces when hostname starts with staff or eg (Bug 1862834)

  • Ability to filter out deleted shelving locations in Shelving Locations Editor (Bug 1917092)

  • Single Sign On (Shibboleth) + Bootstrap OPAC (Bug 1917083)

  • Missing IDL field for stop_blocked_user on config.hold_matrix_matchpoint (Bug 2028012)

  • Library Settings Editor - History Link Missing Cursor Change (Bug 2039306)

  • Shelving location ID 1 cannot be modified (Bug 2023314)

  • Edit Survey Q&A button styles have gone awry (Bug 2040186)

  • eg_db_config can fail depending on ~/.psqlrc contents (Bug 2023418)

  • Rename New Statistical Categories Editors (Bug 2023579)

5.1.4. Booking

  • Booking: attempting to create a reservation for a single item freezes the browser (Bug 2032717)

  • Booking: Overlapping bookings allowed (Bug 1804066)

5.1.5. Carousels

  • Carousels - Carousels Can’t be Created or Edited (Bug 2039612)

  • The "prev" and "next" navigation buttons in carousels are not translated. (Bug 2033067)

  • Carousels - Add buttons in New Carousels Very Large (Bug 2039606)

5.1.6. Cataloging

  • Angular Holdings Editor uses old terminology (Bug 1983424)

  • Fixed Fields Grid in Enhanced MARC Editor Not Updated on Save (Bug 2015163)

  • Fast Item Add Not Working from MARC Edit (Bug 1986706)

  • setting Default Search Pane fails in Angular 3.11 catalogue (Bug 2007603)

  • New Holdings Editor Ignores "Default Classification Scheme" Library Setting (Bug 1960885)

  • WebClient - Create MARC Record - Keyboard Shortcut (Bug 2031040)

  • WebClient - Create MARC Record - Select Template Focus and Page Name (Bug 2031043)

  • Create MARC Record - focus on item add and call number (Bug 2031114)

  • Create MARC Record - Hide help button for flat editor (Bug 2031123)

  • Create MARC Record - Flat Editor - Keyboard Shortcut for Saving (Bug 2031162)

  • Create MARC Record - Jump to Flat Editor - Keyboard Shortcut (Bug 2031177)

  • MARC Batch Import/Export Queue - Some Actions No Longer Show as Links (Bug 2039310)

  • angular MARC editor tab does not display record source value (Bug 1927870)

  • Enable spellcheck for angular MARC edit screens (Bug 1947906)

  • Angular: can no longer double click on item to open editor (Bug 1908568)

  • Stack Subfields are not stacking properly anymore (Bug 2040528)

  • MARC Batch Import/Export Queue: Links to the Staff Catalogue should open in a new tab (Bug 2040305)

  • Record Match Sets: Buttons Out of Alignment When Creating a New Match Set (Bug 2040303)

  • Wide buttons in Record Match Sets (Bug 2043134)

  • Reapplying item template with alert or note results in multiple alerts and/or notes (Bug 1855144)

5.1.7. Circulation

  • Placing holds fails unintuitively when preferred pickup location is disabled via org unit setting opac.holds.org_unit_not_pickup_lib (Bug 1477154)

  • Preferred name not listed as available to receipts (Bug 1841635)

  • Make more strings available for translation in the Mark Damaged and Mark Missing dialogs (Bug 1840990)

  • Check Out Fails Silently if Operating Hours of Operation Set to Closed 7 Days a Week (Bug 1944601)

  • One Hour Gap in Default Autorenewal Delays (Bug 1899976)

  • Sort direction for selection depth wrong when doing best-hold selection (Bug 2023338)

  • Cash Reports allows start date after end date (Bug 2002343)

  • Autorenewal Can Overwhelm open-ils.trigger Service Drones (Bug 2030915)

  • Cash Reports - Label Totals Wrapping Unnecessarily Early (Bug 2039311)

  • Display of survey results in patron account formatted incorrectly (Bug 2040184)

  • Concerns about functionality of Mark item Missing from Items Out (Bug 1998605)

  • Circulation→Retrieve Recent Patrons can have duplicate entries (Bug 2009281)

  • View Holds: Need To be Able To Tell Where The Item Is Coming From (Bug 2040312)

5.1.8. Course materials

  • Browse for course not working (Bug 1913815)

  • Blank or Wildcard Search for Course by Instructor Fails (Bug 1968754)

  • OPAC course reserves link display shouldn’t depend on search library (Bug 2035389)

5.1.9. Client

  • Logging out on a page with a pcrud call floods browser with errors (Bug 2002693)

  • Web staff client does not work properly when Czech is switched on (Bug 2032753)

  • angular: add keyboard support to eg-grid options menu (Bug 1828575)

  • Staff Client eg grid not sorting alphabetically (Bug 1912840)

  • Link/button issue in clipboard dialog component (Bug 2043424)

5.1.10. Documentation

5.1.11. General

  • open-ils.actor.container.retrieve_by_class doesn’t properly handle missing bucketOwnerId (Bug 2036265)

  • Unusual strings in POEditor (Bug 2045078)

  • 2023-06 package-lock.json updates (main and rel_3_11) (Bug 2022939)

  • docs/package.json should be .gitignored (Bug 2035383)

5.1.12. OAI-PMH

5.1.13. Public catalog

  • Request a Card link missing on login form (Bug 2039114)

  • Button in Patron Messages interface in OPAC are not translatable (Bug 1919501)

  • Bootstrap Opac: Personal Information Page contains Links as Buttons (Bug 2040314)

  • Marking org unit as non-visible in the OPAC defaults patrons' preferred pickup locations to the first org unit (Bug 2043127)

5.1.14. Reports

5.1.15. Search

  • Search suggestions can make searches very slow in 3.11 (Bug 2038472)

  • Bootstrap OPAC: Only show current addresses (Bug 1939309)

5.1.16. Staff catalog

  • Angular Staff Catalogue: More Link Missing from Facets (Bug 1934018)

  • Exclude Electronic Resources Check Box Can’t Be Selected in Staff Catalogue (Bug 2036297)

  • Staff Catalog: Default Search and Preferred Library settings are deleted when Search Preference page is loaded (Bug 2037685)

  • Use table for shelving locations in staff catalog (Bug 2016742)

  • Placeholders in search form in staff catalog appears untranslated (Bug 1920126)

  • Some components of the staff client search result interface appears untranslated (Bug 1920230)

  • Copy count highlight color contrast in staff catalog search results (Bug 2043847)

5.2. Further details on bug fixes

5.2.1. Change in AutoRenew Event Definition Default Delay

The delay for the AutoRenew event has been changed from -23 hours to -24 hours and 1 minute. The previous values of -23 hours for the delay and -1 minute for the max_delay left a gap of approximately 1 hour where items would not auto-renew if they fell due during that time. Depending upon the time that the AutoRenew event runner is scheduled to run, this gap may never turn up. However, all it takes is a misconfigured client (i.e. an incorrect timezone setting) or a manually edited due date on a circulation for this to turn up. The new interval settings guarantee that all circulations for a given 24 hour period are selected with no gap.

A database upgrade script is provided to alter any event definitions using the Circ::Autorenew reactor and the previous default delay values to the new settings. If you have customized or added any event definitions using this reactor, you should double check that they are correct after an upgrade.

5.2.2. Shelving Locations Editor: Hide deleted shelving locations by default

In the Shelving Locations Editor under Local Administration, a filter to hide deleted locations is applied by default. Clicking the Remove Filters button or Clearing the filter on the Is Deleted column will reveal the deleted locations.

5.3. Acknowledgements

We would like to thank the following individuals who contributed code, testing and documentation patches to the 3.11.2 point release of Evergreen:

  • Scott Angel

  • Jason Boyer

  • Eva Cerniňáková

  • Galen Charlton

  • Jeff Davis

  • Dan Briem

  • Andrea Buntz Neiman

  • Garry Collum

  • Elizabeth Davis

  • Ruth Davis

  • Bill Erickson

  • Robin Fitch

  • Blake Graham-Henderson

  • Lena Hernandez

  • Kyle Huckins

  • Linda Jansova

  • Brian Kennedy

  • Angela Kilsdonk

  • Stephanie Leary

  • Mary Llewellyn

  • Llewellyn Marshall

  • Steven Mayo

  • Terran McCanna

  • Gina Monti

  • Christine Morgan

  • Michele Morgan

  • Susan Morrison

  • Lauren Mous

  • Jennifer Pringle

  • Simone Rauscher

  • Mike Rylander

  • Jane Sandberg

  • Chris Sharp

  • Jason Stephenson

  • Josh Stompro

  • Elizabeth Thomsen

  • Beth Willis

  • Carol Witt

6. Evergreen 3.11.1

This release contains bug fixes improving on Evergreen 3.11.0.

This includes a fix for a critical security issue. Users are advised to upgrade as soon as possible.

6.1. Upgrade notes

6.2. Bug Fixes

6.2.1. Security ===

  • Fixes an issue in open-ils.fielder that could enable unauthenticated remote SQL injection attacks.

6.2.2. Accessibility

6.2.3. Acquisitions

  • Adds keyboard support for links in purchase order line item list (Bug 2019032)

6.2.4. Administration

6.2.5. Circulation

  • Fixes issues with place hold from patron record in Angular and AngularJS (Bug 1996818)

6.2.6. Documentation

  • Fixes GitHub actions docs build errors (Bug 2022366)

  • Further updating references from master to main in documentation and comments

  • Fixes to Standing Penalties docs & release notes (Bug 2022962)

6.2.7. Reports

  • Fixes an issue where enabling Shibboleth broke reports output access (Bug 2008252)

6.3. Acknowledgements

We would like to thank the following individuals who contributed code, testing, and documentation to the 3.11.1 point release of Evergreen:

  • John Amundson

  • Jason Boyer

  • Dan Briem

  • Galen Charlton

  • Jeff Davis

  • Stephanie Leary

  • Andrea Buntz Neiman

  • Jane Sandberg

  • Jason Stephenson

7. Evergreen 3.11.0

7.1. Acquisitions

7.1.1. Lineitem Detail Audit Table

The default schema has added an audit table for the acq.lineitem_detail table. The audit table is NOT created during database upgrade. If you wish to add the audit table to your Evergreen installation, you can run the following SQL in your database:

SELECT acq.create_acq_auditor ( 'acq', 'lineitem_detail' );
CREATE INDEX acq_lineitem_detail_hist_id_idx ON acq.acq_lineitem_detail_history( id );
CREATE INDEX acq_lineitem_detail_hist_lineitem_idx ON acq.acq_lineitem_detail_history( lineitem );
CREATE INDEX acq_lineitem_detail_hist_fund_debit_idx ON acq.acq_lineitem_detail_history( fund_debit );

7.2. Administration

7.2.1. Enhanced Concerto dataset

An alternative sample dataset called "Enhanced Concerto" is now available. This dataset includes a more realistic organizational unit structure and additional data with which to test Evergreen functionality.

The "Enhanced Concerto" dataset supports human-driven testing and community demonstrations. However, at present, automated tests, particularly the ones found in Open-ILS/src/sql/Pg/live_t and Open-ILS/src/perlmods/live_t, are not guaranteed to pass with the "Enhanced Concerto" dataset; instead, they remain targeted to working with the original "Concerto" dataset.

Installing the dataset

The dataset can be loaded when installing Evergreen by using the --load-concerto-enhanced option of eg_db_config. For example:

perl Open-ILS/src/support-scripts/eg_db_config --update-config \
       --service all --create-database --create-schema --create-offline \
       --user <user> --password <password> --hostname <hostname> --port <port> \
       --database <dbname> --admin-user <admin-user> --admin-pass <admin-pass> \
       --load-concerto-enhanced

The --load-concerto-enhanced switch is mutually exclusive with the --load-all-sample switch to load the full original "Concerto" dataset and the --load-concerto switch to load just the bibs, items, and authorities from the original "Concerto" dataset.

Updating the dataset

An experimental script called make_concerto_from_evergreen_db.pl is included to support

  • updating the dataset as Evergreen’s schema evolves

  • incorporating changes to the dataset from a running Evergreen system

The script compares data between different versions of an Evergreen database depending on how it is invoked. This script has known bugs and its output must be manually reviewed before being committed.

7.2.2. --pipe option for marc_export

Add a --pipe option to marc_export to force reading of record ids from standard input when one or more of the --library, or --descendants options are used. This permits more flexibility when exporting records with holdings for given libraries or subsets of a collection.

It is an error to specify the --all or --since and --pipe options in the same command line.

        --pipe

Some examples:

        --pipe --library BR1   [Filter the list of bib ids by those
                                with BR1 as the Owning Library]

        --pipe --descendants BR1 [Filters the list of bib ids by those
                                  with BR1 or descendants as the Owning Library]

7.2.3. --strip option for marc_export

The --strip option is used to suppress generation of specified elements from the marc_export output. The option, which can be specified more than once, is in one if these forms:

        --strip <field RE>/<subfield RE>
        --strip /<subfield RE>
        --strip <field RE>

Some examples:

        --strip 856/0   [Delete subfield 0's in fields with tag 856.]

Regular expressions are accepted:

        --strip 8../0   [Delete subfield 0's in fields with tag 800-899.]

If the field is omitted, it is as if you specified "..." for the field RE.

    --strip /0      [Delete subfield 0's in all fields.]
    --strip /[abc]  [Delete subfield a, b or c in all fields.]

If the slash and subfield are omitted, it means to delete the given fields.

    -strip 856     [Delete fields with tag 856]

If the slash is present, but the subfield is omitted, it means "all subfields"

    --strip 856/    [Delete all subfields with tag 856]

7.2.4. Applying Missing Database Upgrades

This patch fixes a situation where an Evergreen database that had been been upgraded to 3.6.0 at some point in its past using the 3.5.1-3.6.0 DB update script may be missing some DB revisions.

The following bugfixes are affected:

7.2.5. Improvements to Mapping Permissions to Groups

The dialog to add permission mappings to a group in the Permission Groups administrative interface now allows multiple permissions to be added in one invocation. This reduces the number of clicks and keyboard interactions required when adding a large number of permission mappings.

7.2.6. Custom Penalties

This development creates a set of Library Settings that a staff member with appropriate permissions can use to set a custom value for a stock penalty according to the following workflow:

  1. Create a new penalty in Standing Penalties

  2. Set the desired penalty threshold in Group Penalty Thresholds

  3. Use the Library Setting to associate your new penalty with a stock system penalty and organizational unit context.

The custom penalty will be automatically applied in place of the stock penalty at the specified org units, when the patrons' account meets the penalty criteria.

System penalties that can be overridden:

  • PATRON_EXCEEDS_FINES

  • PATRON_EXCEEDS_OVERDUE_COUNT

  • PATRON_EXCEEDS_CHECKOUT_COUNT

  • PATRON_EXCEEDS_LOST_COUNT

  • PATRON_EXCEEDS_LONGOVERDUE_COUNT

  • PATRON_EXCEEDS_COLLECTIONS_WARNING

  • PATRON_IN_COLLECTIONS

7.3. Architecture

7.3.1. Malicious Search Protection

Evergreen sometimes sees some "novel" query strings in the wild that cause the search backend to time out or worse. These are sometimes malicious and sometimes accidental, but the effect on users is the same.

The changes here improve query compilation in several respects in order to reduce the chances of an overly complex query causing problems for the search subsystem.

More work is done up front to simplify and combine parts of the resulting SQL, allowing more work to be done closer to the data. This change allows Evergreen to handle many more tested or chained boolean expressions, and negated terms are now handled directly in line with other adjacent terms. Phrases (exact matches) are now searched for using Postgres' adjacency tsearch operator.

All of these changes work together to improve performance by getting more search work done in fewer database operations while protecting against certain query constructs that have caused problems in the past.

7.3.2. Angular and Bootstrap Updates for Staff Interface

The Angular staff interface now uses Angular 15 and Bootstrap 5. This introduces various changes to how the staff interface styling is done; developers should take note.

7.3.3. Fix SQL Injection Vulnernability

An SQL injection vulnernability related to the implementation of search term highlights is now closed.

7.3.4. Queued Ingest

This feature allows for the separation of bib and authority record updates and the search (and other) indexing that occurs when a record is modified in some way. Prior to this feature, bib and authority records would be indexed immediately upon an update.

While individual record ingest has not become a problem with regard to system performance or interface usability, there exist several batch operations which aggregate many inserts or updates and whose aggregate ingest time cost can be significant. These include, but are not limited to, reingest caused by authority control propagation, reingest required by the addition or modification of indexing configuration, cataloging and acquisitions record import and overlay from the staff interface, and upgrade-time reingest required by structural changes to the underlying indexing and search system.

New Utility

When Queued Ingest is enabled, a new control script, ingest_ctl, is available to perform several functions:

  • Run in the background to process the queues of indexing requests

  • Display statistics of queued ingest activity

  • Specify that a set of records should be reindexed.

Here are some examples of how it is used:

# Enqueue records 1-500000 for reingest later, just one worker for the queue
/openils/bin/ingest_ctl --queue-threads 1
    --queue-type biblio
    --queue-run-at tomorrow
    --queue-owner admin
    --queue-name "slowly updating records due to new RDA attributes"
    --start-id 1 --end-id 500000

# Start the background worker
/openils/bin/ingest_ctl --coordinator --max-child 20

# Stop the background worker
/openils/bin/ingest_ctl --coordinator --stop

# Process whatever you can Right Now
/openils/bin/ingest_ctl --max-child 20

# Process a single queue Right Now
/openils/bin/ingest_ctl --queue 1234 --max-child 20

# Stats on Queued Ingest processing so far today
/openils/bin/ingest_ctl --stats --since today --totals-only

This script also requires the following switches (or environment variables) in order to connect to the database:

  • --db_user (or environment variable PGUSER)

  • --db (or environment variable PGDATABASE)

  • --dbpw (or environment variable PGPASSWORD)

  • --db_port (or environment variable PGPORT)

New Settings

This feature adds several new global flags:

Global Flag

Enabled

Queued Ingest: Abort transaction on ingest error rather than simply logging an error

no

Queued Ingest: Queue all bib record updates on authority change propagation, even if bib queuing is not generally enabled

no

Queued Ingest: Use Queued Ingest for bib record ingest on insert and undelete

no

Queued Ingest: Use Queued Ingest for authority record ingest on insert and undelete

no

Queued Ingest: Use Queued Ingest for bib record ingest on update

no

Queued Ingest: Use Queued Ingest for authority record ingest on update

no

Queued Ingest: Use Queued Ingest for bib record ingest on delete

no

Queued Ingest: Use Queued Ingest for authority record ingest on delete

no

Queued Ingest: Maximum number of database workers allowed for queued ingest processes

yes; default value 20

Queued Ingest: Use Queued Ingest for all bib record ingest

no

Queued Ingest: Use Queued Ingest for all bib and authority record ingest

no

Queued Ingest: Do NOT use Queued Ingest when creating a new bib, or undeleting a bib, via the MARC editor

yes

Queued Ingest: Use Queued Ingest for all authority record ingest

no

Queued Ingest: Do NOT Use Queued Ingest when editing bib records via the MARC Editor

yes

This feature does not add any new library settings or permissions.

Upgrade Notes

Queued Ingest is not automatically turned on upon upgrade. to enable it, at minimum the following actions should be taken:

  • Enable at least the "Use Queued Ingest for all bib and authority record ingest" global flag

  • Ensure that ingest_ctl is running with the --coordinator flag.

7.4. Cataloging

7.4.1. Catalog Staff View

Adds a Staff View tab to the record details page in the staff catalog. In addition to showing various fields from the bib record, it also shows a count of hold requests and a break-down of available items versus total items, and it shows a breakdown of similar records by formats and language editions, based on the associated metarecord.

7.5. Circulation

7.5.1. Adjust Hold Cancel Reasons

The dialog to confirm canceling a hold in the staff interface now lists only hold cancel reasons that can be manually applied, excluding the ones that can only be automatically applied.

In addition, two more hold cancel reasons are added:

  • Patron via email

  • Patron via SMS

7.5.2. New Permission for Viewing Holds Pull List

Access to the holds pull list is now restricted to users with the VIEW_HOLD_PULL_LIST permission. By default, this new permission is granted to all users with the VIEW_HOLDS permission.

7.6. Client

7.6.1. Angular navbar keyboard navigation

Adds basic keyboard navigation support to the Angular web client navbar as follows:

  • Use the Tab key to move between the top-level menu items.

  • Press Enter, Space, or the down arrow on a top-level item to open its submenu. Pressing Esc will close it.

  • Shift-tab from the first submenu item back to the top level, and use Enter or Space to toggle the button again and close the submenu.

  • Within the submenu, use either Tab and shift-Tab to move up and down, or the up/down arrow keys.

7.7. OPAC

7.7.1. Multi-term Did You Mean search suggestions

Expanding on the previous single-class, single-term search suggestion development, this feature provides suggestions for single-class searches with multiple terms.

  • The Library Settings that were previously used to control the global behavior of search suggestions have been moved to search class configuration fields. This was done because the data in each search class benefits from different setting values. If the Library Settings had been set, they will be used to update the corresponding search class configuration fields. Regardless, Evergreen administrators are recommended to review the settings.

  • If a patron’s search matches a variant or non-preferred heading from an authority record, if the main heading of that authority is linked to at least bibliographic record, the system will provide that main heading as a suggestion as well, along with spelling-corrected suggestions.

  • Quoted phrases in user input require strict term order and adjacency for the phrase portion of the suggestion generated for the phrase(s), whereas unquoted input (or the portion that is not quoted) does not.

MARC Search/Facet Class field additions

  • variant_authority_suggestion Whether this class should attempt variant authority suggestions based on search-class/browse-axis mapping

  • symspell_transfer_case Whether suggestions should retain user-supplied letter case

  • symspell_skip_correct Only supply suggestions to misspelled words

  • symspell_suggestion_verbosity Setting that controls the amount of effort, and therefore time, spent on suggestion generation

  • max_phrase_edit_distance Maximum average per-word edit distance when evaluating suggestions

  • suggestion_word_option_count Maximum alternate suggestions per word

  • max_suggestions Maximum suggstions to present

  • low_result_threshold Maximum hit count beyond which suggestions are not provided

  • min_suggestion_use_threshold Minimum number of times a suggestion must exist in the corpus

  • pg_trgm_weight Weight of the trigram similarity metric; 0 avoids calculation costs

  • soundex_weight Weight of the soundex similarity metric; 0 avoids calculation costs

  • keyboard_distance_weight Weight of the keyboard distance similarity metric; 0 avoids calculation costs

Upgrade

If the databse has authority records that are linked to bilbiographic records, a reingest of the search suggestion dictionary is recommended.

Instructions for performing that reingest are included in the database update scripts and will be output to the log when those scripts are run.

7.7.2. Restrict login redirect

As a security best-practice, Evergreen should not allow arbitrary redirection on successful login, but instead limit redirection to local links or configured domains and schemes.

This feature is controlled by a new global flag called opac.login_redirect_domains which must contain a comma-separated list of domains. All hostnames under each domain is allowed for redirect, and the scheme of the redirect URL must be one of http, https, ftp, or ftps.

7.7.3. Accepting payments with SmartPAY

SmartPAY is a payment processing service that lets sites take credit card payments without payment card information ever touching the sites' own servers.

Library Settings

The following settings need to be set at the appropriate org level for sites wanting to use SmartPAY.

  • "Allow Credit Card Payments" (should be true) 

    credit.payments.allow

  • "Enable SmartPAY payments" (should be true) 

    credit.processor.smartpay.enabled

  • "SmartPAY location ID" (value provided by Comprise) 

    credit.processor.smartpay.location_id

  • "SmartPAY customer ID" (value provided by Comprise) 

    credit.processor.smartpay.customer_id

  • "SmartPAY login name" (value provided by Comprise) 

    credit.processor.smartpay.login

  • "SmartPAY password" (value provided by Comprise) 

    credit.processor.smartpay.password

  • "SmartPAY API key" (value provided by Comprise) 

    credit.processor.smartpay.api_key

  • "SmartPAY server name" (value provided by Comprise) 

    credit.processor.smartpay.server

  • "SmartPAY server port" (value provided by Comprise) 

    credit.processor.smartpay.port

  • "Name default credit processor" (should be SmartPAY) 

    credit.processor.default

7.8. Miscellaneous

  • Add patron home library code as a column to the View Holds grid in the staff catalog record details page (LP#1991726)

  • Include template ID in the template table in the Reporter (LP#1998386)

  • Remove the pub flag from the biblio.record_note table (LP#1978978)

  • Add the publication date to the Staff Catalog’s Shelf Browse (LP#1999432)

  • Resolve search performance degradation with PostgreSQL version 12 and up (LP#1999274)

  • Improved styling of paid line items in acquisitions screens (LP#1999270)

  • Improved styling of the keyboard shortcut info modal (LP#1999955)

  • (Developer) Add Emacs mode to fm_IDL.xml (LP#1914625)

  • autogen.sh can now accept a -c switch to specify the location of opensrf_core.xml. This is useful for certain multi-tenant setups of Evergreen. (LP#2003707)

  • Better organization of acquisitions line item alert fields (LP#2002977)

  • Prevent templates from applying or changing magical status in angular holdings editor (LP#1999401)

  • Prevent directly editing the shelving location deleted field in the Shelving Locations Editor (LP#2002435)

  • The "Strict Barcode" checkbox is now closer to the barcode input on the Check Out, Check In, and Renew Items pages (LP#1990968)

  • LP1929593 UPDATE_COPY_BARCODE permission

    • This adds the permission UPDATE_COPY_BARCODE and a new API call, 

      open-ils.cat.update_copy_barcode

    • which explicitly tests for both UPDATE_COPY_BARCODE and UPDATE_COPY, with either being sufficient for allowing a barcode change. Existing Replace Barcode UI’s in both Angular and AngularJS have been modified to use this API call instead of the pcrud service. One side-effect of this has been better surfacing of errors, as errors in pcrud were uncaught and bypassing the normal error handling. This addresses LP1951469.

    • The upgrade script gives any permission groups that already have the UPDATE_COPY permission the new UPDATE_COPY_BARCODE permission at the same depth, though it’s technically not needed.

  • Patron and staff login forms now include a button to reveal the password input. (LP#1977554)

  • Adds new Local Administration entries for Item Statistical Categories Editor and Patron Statistical Categories Editor, which are angularized interfaces.

  • Tweaks eg-grids to underline hyperlinks within cells. This potentially affects multiple interfaces.

  • eg-org-family-select now supports persistKey

  • LP1965446 Option to Disable Title-Level Holds on Bib Records with Parts

    • This feature adds one global flag and one library setting, respectively:

      • circ.holds.api_require_monographic_part_when_present Holds: Require Monographic Part When Present for hold check.

      • circ.holds.ui_require_monographic_part_when_present Require Monographic Part when Present

    • Normally the selection of a monographic part during hold placement is optional if there is at least one copy on the bib without a monographic part. A true value for this setting for any involved owning library for the bib or for the global flag will require part selection even under this condition. This essentially removes the All/Any Parts option from the part selection drop-down, for both versions of the public catalog (TPAC and BOOPAC), and for the Angular staff catalog interface. It should be noted that if the library setting is set below the consortium level, Title level holds may be allowed for some libraries and not others.

    • At the API level, we consider just the global flag and will throw a TITLE_HOLD_WHEN_MONOGRAPHIC_PART_REQUIRED event for a title hold request when there are items with monographic parts on the bib. It is possible for the library settings and the global flag to differ, but the global flag will catch every instance of hold placement including those by third party callers, SIP, etc.

  • Links the lineitem id link in Acquisitions search results to the lineitem detail page rather than the purchase order. (LP#2003946)

8. Acknowledgments

The Evergreen project would like to acknowledge the following organizations that commissioned developments in this release of Evergreen:

  • CW MARS

  • Evergreen Community Development Initiative

  • Evergreen Indiana

  • King County Library System

  • Pennsylvania Integrated Library System

  • Westchester Library System

We would also like to thank the following individuals who contributed code, translations, documentations patches and tests to this release of Evergreen:

  • John Amundson

  • Scott Angel

  • Jason Boyer

  • Dan Briem

  • Andrea Buntz Neiman

  • Eva Cerninakova

  • Galen Charlton

  • Garry Collum

  • Elizabeth Davis

  • Jeff Davis

  • Britta Dorsey

  • Bill Erickson

  • Jason Etheridge

  • Ruth Frasur

  • Jeff Godin

  • Blake Graham-Henderson

  • Rogan Hamby

  • Elaine Hardy

  • Stephanie Leary

  • Clayton Liddell

  • Shula Link

  • Tiffany Little

  • Mary Llewellyn

  • Debbie Luchenbill

  • Karen MacDonald

  • Llewellyn Marshall

  • Terran McCanna

  • Chrystal Messam

  • Gina Monti

  • Christine Morgan

  • Michele Morgan

  • Susan Morrison

  • Susasn Morrison

  • Dan Pearl

  • Jennifer Pringle

  • Mike Risher

  • Mike Rylander

  • Jane Sandberg

  • Chris Sharp

  • Ben Shum

  • Jason Stephenson

  • Josh Stompro

  • Elizabeth Thomsen

  • Jennifer Weston

  • Beth Willis

  • Carol Witt

  • Adam Woolford

  • Jessica Woolford

We also thank the following organizations whose employees contributed patches:

  • BC Libraries Coop

  • Bibliomation

  • Catalyte

  • CW MARS

  • Equinox Open Library Initiative

  • Evergreen Indiana

  • Georgia Public Library Service

  • Kenton County Library

  • King County Library System

  • Lake Agassiz Regional Library

  • LibraryMarket

  • Linn Benton Community College

  • MOBIUS

  • NC Cardinal

  • NOBLE

  • Princeton University

  • Sigio

  • Westchester Library System

We regret any omissions. If a contributor has been inadvertently missed, please open a bug at http://bugs.launchpad.net/evergreen/ with a correction.