evergreen-ils blog

Yesterday I was explaining to some librarians how Evergreen relates Items (the actual physical barcoded material that circulates) to Volumes (where the Call Number lives) to Bib Records (which contains the MARC), and one person was curious and asked, “why do you do it that way?” The short answer is because it’s good design, but the question momentarily threw me for a loop because the implication is that they’re used to systems which do not do it that way.

Imagine that the structure used for items in the database for your ILS is laid out something like this:

Item Table
———-
Internal ID
Creation Date
Barcode
Call Number
…other fields

In some automation systems, that Call Number field will be a free-text label. If you want to change the Call Number for an Item, then that field will necessarily get changed. If you want to change the Call Number for a large grouping of Items, then you will have to change the field for all those items (sometimes one at a time).

In Evergreen, that Call Number field will actually be a foreign key to another table in the database, one that looks something like this:

Volume
——-
Internal ID
Owning Library
Call Number Label
…other fields

This allows a Call Number to be shared amongst a group of items, and be modified with a single edit.

This is also an example of database normalization.

Why is this important? It has to do with redundancy of information (and I’m not talking about backup storage, RAID’s, etc).

Human beings love redundancy when it comes to communication; our meanings get emphasized by our body language, and the syntax and structure of our languages, both spoken and written, encode information in multiple ways. But redundancy opens the door to discrepancy, and discrepancy leads to ambiguity. Human beings can deal with ambiguity, but computers (and most software developers, and maybe catalogers) can’t abide it. “Time flies like an arrow, but fruit flies like a banana”.

A computer might resolve ambiguity in a non-obvious way, or worse, it may resolve ambiguity in a manner that coincidentally matches a human’s expectation, only to cause trouble later when you start getting data anomalies.

Normalization is a best practice design technique for designing relational database schemas, though there are cases when you want to make a trade-off and de-normalize certain data structures. But those cases are usually optimizations that should occur after you already have a good design as your starting point.

Relational databases are powerful enough that they have sunk into the consciousness of librarians, to the extent that they put down “Must be built on a relational database” in their RFP’s. But most legacy ILS products actually use hierarchal databases, which are good for some things, but not for others. The design decisions made when you go with a hierarchal database are very different than the ones you make when you start with a relational database, and I worry about the legacy products that have “tacked on” relational databases for buzzword and RFP-compliance. Legacy automation may now have relational databases, but are they actually using them as a relational database? Some are, but I know of at least one that isn’t.

Let’s return to Evergreen for a moment. Because Call Number Volumes are represented as their own entities, you have the option of thinking of them differently. For example, you can move a set of items to a different “call number”, one which may be associated with a different Bib Record altogether (or you could move a volume itself to a new bib record). You could even change which library “owns” a volume, and suddenly all the items attached to that volume have a new owner. And you’re able to place a hold on a specific “volume”, in addition to title-level and copy-level holds (and in Evergreen, meta-record holds across editions and formats).

But you can also do what you may already be used to, and change the Call Number for an item (or a batch of items) from an Item Editing interface, and not have to know about the more flexible structures that are being manipulated underneath.

Because we already have Volumes, we’re also in a better place for adding Serials.

There are fewer widespread repercussions to changing an end-user interface than there are with changing your database schema, so that’s why it’s important to have a good database design from the beginning, so that your interfaces have more options. To steal a sentiment from Mr. Miyagi, Evergreen has strong root.

– Jason

I’ve been to my share of library conferences over the last couple of years. They all have something to offer in their own way: networking, schmoozing, wacky vendor displays, swag, etc. It’s been my experience, though, that as a geek/developer, I’m often a little disappointed at the content — it’s just not geared toward someone like me. This is why I love Code4lib. The conference is densely packed with interesting technical presentations. They are the kind of talks that not only inspire you, they give you something concrete to take back home. In addition to the presentations, you are surrounded by an interesting gang of library technologists, who, I get the sense, all have tricks up their sleeves. It makes for a fulfilling week, to say the least.

Some highlights

My week started with the Evergreen pre-conference, skillfully lead by Dan Scott. Dan blogged about it here. Apart from some technical woes, a lot was seen and (hopefully) learned by all. We performed the install, imported bib and holdings data, implemented new functionality (new OpenSRF method and OPAC UI) to email a user their password if they have forgotten it, and took a quick look at some of the staff client interfaces.

Conference day 1 began with Brewster Kahle’s keynote address on his work with archive.org and openlibrary.org. Brewster’s vision and chutzpah is refreshing and inspiring. The goal of “One wiki page per book” sounds so simple (and obvious) when inserted into a slide show, but the implementation will require tremendous work and resources. I applaud their efforts, not only in principle, but because I think it will lead to better information and resource sharing in the long run.

After lunch, Winona Salesky and Michael Park did a joint presentation called “XForms for Metadata Creation”, which included two different MODS editors developed with XForms. I liked this talk in particular for two reasons. For starters, the demos I saw were that of a simple (yet powerful) bibliographic data creation interface, which I see as potentially useful for any ILS. Additionally, the use of XForms, which I only have a basic understanding of, gave me a chance to see some new (for me) technology in action. My interest is piqued.

I skipped the breakout sessions that day to put together a (5 minute) lightning talk on Pylons. One of the interesting challenges of lightning talks is the use of a shared PC and what amounts to a moratorium on web access for the sake of efficiency. My presentation was a set of images and screenshots, the first a crane on a sunny Portland day, which seemed apropos of the topic. I discussed one particular aspect of Pylons, which allows you to easily plug in custom pre and post-processing middleware applications for your web apps. I demonstrated a simple XML validator and a highlighting plugin which bolded pre-defined terms in the HTML on its way to the client. These gave me the chance to dig a little deeper into Pylons, an architecture which I think has a lot of interesting potential.

Wednesday morning I learned a lot about RDA. Later, I thought the discussion on of the ILS Discovery Interface Task Force segued well into Ross Singer’s lightning talk on Jangle, which, as Ross pointed out, could be used to implement the proposed standardized ILS interface.

I was not able to attend the last day, so I missed a chunk of the conference, including Dan’s CouchDB talk. Glad it went well, Dan!

Wow, now I realize why most folks blog about conferences as they are happening It’s a lot to digest, especially for a conference as dense as code4lib. There were a lot of great presentations, breakout sessions, and lightning talks this year and this post only comments on a few of them. Next year’s conference will be held in Providence, RI, and if you can, I would recommend checking it out.

-bill

I was curious about how many circulations PINES had on the biggest day on record. And that led to our asking: which hour and which minute had the most circs?

Maximum circs by DAY:
96,326 – May 29, 2007

Maximum circs by HOUR:
11,305 – May 29, 2007, 11 AM

Maximum circs by MINUTE:
548 – May 2, 2007, 9:21 AM

I understand that the day after Memorial Day in 2007 is remembered as a very busy day for reasons that are clear.

For a bit of perspective, the FY 2005 NCES/NCLIS data for all US public libraries give the following annual figures for total circulations by library:

Q3 (third quartile–the libraries above this number are in the top 25%):
144,663

Q2 (median–the middle value):
43,912

Q1 (libraries below this number are the bottom 25%):
13,113

The arithmetic mean is 229,685.

That maximum number of circs for PINES for one day is greater than the median annual value for all public libraries.

Why is the mean so much higher than the middle value? Because the distribution of all these numbers is highly skewed–in the Statistical sense. And that fascinating topic can be the subject for compelling blog post at another time.

Bob Molyneux

We get this question often at Equinox Software and we see it on occasion on the Evergreen community mailing lists and in the library blogosphere.

I am a single-branch public library with about 25,000 titles. I’ve looked at the options and I really like Evergreen. But isn’t it simply ‘too big’ for my situation?

It is true Evergreen was built from the ground up to handle the largest of library systems: it’s highly scalable, fault-tolerant (meaning it can sustain hardware failures and keep running), and has capabilities that make it a perfect selection for large libraries and consortia. Those attributes/goals are clearly stated on the open-ils community website.

However, it is important to realize this does not mean Evergreen is not a good solution for smaller libraries as well. Evergreen does in fact “scale down” elegantly. We know this because there are developers in our community that run Evergreen on their laptops. That’s right folks–they can run the entire server-side software stack on their laptops. Another example: Bill Erickson, at the last code4lib conference, ran an “Evergreen install in 20 minutes” breakout session in which Evergreen was installed on a laptop.

Evergreen was designed to be able to scale to multiple servers of varying sizes and purposes. But it can also scale to a single server (or laptop) if that’s what the library desires and what makes sense in the cost-to-uptime ratio.

Also, the ability for Evergreen to “scale down” from a functional standpoint is something that’s apparent in Kent County Library’s recent migration announcement. Kent County Library serves a population of approximately 19,500 though a main library and 2 smaller branches. There is more information about the decision to go with Evergreen on the Kent County Library blog.

I hope this information puts the rumor to rest that Evergreen is just for big libraries and consortia. The fact is Evergreen can work quite well in smaller situations.

Lorcan Dempsey’s post Evergreen and Pines refers to a Georgia Public Library Service (GPLS) report Use of Georgia’s public libraries continues to rise in Internet Age and echoes one I made earlier here entitled Save the time of the reader. Briefly, all three deal with increases in the use of Georgia Public libraries generally and PINES particularly. But Dempsey makes a point of mentioning the consortial aspect of PINES and I have been thinking about this very point since my post a few weeks ago.

PINES, of course, runs on Evergreen and although Evergreen is running individual systems, the fact is that Evergreen is the most consortially-aware ILS—open source or proprietary—has resulted in some fascinating dynamics which these cited posts allude to.

In my post, Save the time of the reader, I reported on my attempt to measure the change year over year in three PINES use measures that could be compared before and after the switch from the legacy vendor to Evergreen. There are dramatic increases which I attributed largely to Ranganathan’s law it referred to because the reader’s time was saved as a result of better design of the interface, particularly of holds.

However, there is a measurement problem that brings out something more important that had implications I didn’t understand a month ago. Not all public libraries in Georgia are in PINES and the number changed over the period of my analysis. If we want apples to apples, as the saying goes, we should analytically remove the libraries that were added during the year so that we would have circs, holds, and such at the same set of libraries, that is, so that increases reported are not because of the addition of new libraries but rather because of easier to use software with better features. People, though, are just a bit complicated and that simple statistical manipulation would miss something key.

Georgia has a universal borrower card so if your local library is not in PINES, you can go to one that is and borrow from the PINES system. And what library users are doing is pretty clear, they are making the drive: they want access to the 2 million bibliographic entities and the over 9 million items owned by the libraries in PINES. As a result, the obvious statistical treatment itself would miss this dynamic aspect of people’s behavior. They don’t hold still and they react to changes in their environment.

To come full circle, I offer this speculation: that some of the growth in the number of PINES systems (44 to 49) since Evergreen went live in 2006, may be a result of this behavior.

I was graduated from library school in 1971 and I have seen various fashions about how best to do library service: big regionals or small local and intimate libraries. Back and forth. Back and forth. I believe the jury is now in. Library users are voting with their feet; they want the big, resource rich library.

Chris Jowaisis’s Texas Library Systems wiki has an ILS Discussion in which he offered an opinion I thought insightful about the Georgia experience: “(CJ opinion) political agreements are as impressive as the technology achievements of Evergreen.” I have quoted Chris’s observation often. Now, after seeing the experience in Georgia up close, I think that while we librarians and politicians care about the politics, the people who use libraries don’t. If you build the resource sharing consortium, they will come.

Bob Molyneux

Evergreen went live over the Labor Day weekend in 2006. How are the users of PINES adapting to the new ILS? The results of this preliminary analysis show rapid growth in use of PINES libraries.

We only have fragmentary data from before the change but the following table shows changes in selected measures of use of the PINES system. Year over year changes are the best way to calculate this kind of change because there are seasonal effects—for instance December is generally a slow month—and I have chosen to use percentages rather than raw numbers to make comparisons easier. In the table, the row “September 2007″ lists the percent increases in the various raw figures of September 2007 over September 2006. Thus, September 2007 saw an increase in PINES libraries’ circulations of 10.1% over the September 2006 figures. Blanks in the table indicate there are no matching data to calculate percentage increases.

The table below has percent increases in circulations, total holds, and unique patrons requesting holds. Holds are those place by users in the library when some cataloged item is not available. Although Evergreen is used to measure the in-house use of both of cataloged and of uncataloged items, these types of circulations are not included here. The circs are by the standard definition—these are items going out the door.

We can see that circs have increased a bit over 10% for the four months we have matching data. Holds increased a good bit more and we have data for a longer period. The range of increases of these holds is from a bit under 30% increase to over 50%, depending on which month one examines. The change in holds is dramatic.

% increase in



% increase
% increase in
Unique Patrons



in Circulations
Total Holds
Requesting Holds


September, 2006
 
 
 


October, 2006
 
 
 


November, 2006
 
 
 


December, 2006
 
 
 


January, 2007
 
35.2
 


February, 2007
 
40.8
 


March, 2007
 
39.2
 


April, 2007
 
50.0
 


May, 2007
 
45.9
 


June, 2007
 
38.5
 


July, 2007
 
52.0
 


August, 2007
 
38.4
 


September, 2007
10.1
51.4
39.5


October, 2007
15.0
36.5
30.4


November, 2007
10.0
29.7
25.0


December, 2007
11.1
28.9
25.5

The right column shows the increase in unique patrons requesting holds. This column presents an interesting set of statistics because we see a striking increase in the number of patrons using this service.

So, the number of people requesting holds is going up, too. The number of holds per person (not shown) is up only slightly averaging fewer than 4.

Why these increases? I think we are seeing an “Evergreen Effect” which I would define as increases in library use brought on by saving the time of the reader. I don’t believe Ranganathan would be surprised by these data. Those readers are responding to a lower bar between them and materials they are seeking by using the PINES system and materials from its constituent libraries more. Evergreen was designed from the ground up with close consultation with librarians, Georgia Public Library Services staff, and library users. Modern notions about presentation of data were included from the beginning, not bolted on to an aging system. Hence, it is easy to use and intuitive.

These figures overstate the changes slightly. The number of systems at the start of PINES was 44 (including the State Library) and by December, 2007, there were 47. We did look at backing the data from the newer libraries out and that would decrease the raw numbers and percentages about 4%, we estimate.

But we ran into a problem and that was that people living in the service areas of libraries not served by PINES have been using PINES libraries in order to get access to the large, virtual library. There are currently 1.82 million titles in PINES libraries with 9.4 million copies. As I said: a large virtual library and one with many resources to share to library patrons with a PINES card. Any Georgia citizen can get one and we know that library users from non-PINES are availing themselves of these resources and 1.82 million titles is a powerful draw and, not inconceivably, a draw for some of the non-PINES libraries to join. In any case, if we took the effect of the newer libraries out, many of the users in the new systems were already users of PINES. There is an apples to oranges comparison but the currently unmeasurable effect is likely small—probably lower than 4% but we just can’t measure it. So, for December 2007 increase in circulations over December 2006 might be as low as 7%. The holds might also be about 4% lower, that is, perhaps 25% increase in total holds for December 2007 over December 2006. Teasing out a better number is going to be tricky.

Did you know that there’s a Facebook group dedicated to your favorite ILS? The Evergreen “Open-ILS” Automated Library system group is an informal virtual social gathering of Evergreen fans, devotees, and users.  If you have a Facebook account (or are looking for a reason to set one up), we invite you to join the group.  See you there!

Equinox hosted a small Evergreen lunch party/gathering yesterday, and it seemed a perfect opportunity to get a picture of many folks who work together all the time, but rarely see each other. Sadly, one of our number had to take the picture–thanks to Lamar Veatch, Georgia State Librarian for performing the picture-taking duty.

Happy Holidays from the Evergreen gang (clicky for a larger image).

Pictured, from the left:

Tim Daniels, GPLS

Bob Molyneux, Equinox

Don McMorris, Equinox

Hilary LaJeunesse, with Will LaJeunesse

Brad LaJeunesse, Equinox

Elizabeth McKinney de Garcia, GPLS

Art Rhyno, University of Windsor

Bin Lin, GPLS

Jason Etheridge, Equinox

Bill Erickson, Equinox (squatting)

Sally Murphy, GPLS

Brandon Uhlman, British Columbia, Ministry of Education, Pines Project

Dan Scott, Laurentian University

(behind Dan is Dilbert, but he was shy)

Mike Rylander, with Avery Rylander

Sara Rylander, with Baby #2 Rylander (in the oven)

Lamar Veatch, GPLS (behind the camera)

ACQFest is just finishing up, everyone is safely stowed on or near planes for their trip back home, and I’m trying to catch up on more than a week of backlog. I took a quick break to take the book quiz pointed out by Mark Linder and it seems … well, I’ll let you, kind reader, be the judge of its accuracy.

You’re A Prayer for Owen Meany!
by John Irving
Despite humble and perhaps literally small beginnings, you inspire faith in almost everyone you know. You are an agent of higher powers, and you manifest this fact in mysterious and loud ways. A sense of destiny pervades your every waking moment, and you prepare with great detail for destiny fulfilled. When you speak, IT SOUNDS LIKE THIS!
Take the Book Quiz at the Blue Pyramid.