Imagine that the structure used for items in the database for your ILS is laid out something like this:

Item Table
———-
Internal ID
Creation Date
Barcode
Call Number
…other fields

In some automation systems, that Call Number field will be a free-text label. If you want to change the Call Number for an Item, then that field will necessarily get changed. If you want to change the Call Number for a large grouping of Items, then you will have to change the field for all those items (sometimes one at a time).

In Evergreen, that Call Number field will actually be a foreign key to another table in the database, one that looks something like this:

Volume
——-
Internal ID
Owning Library
Call Number Label
…other fields

This allows a Call Number to be shared amongst a group of items, and be modified with a single edit.

This is also an example of database normalization.

Why is this important? It has to do with redundancy of information (and I’m not talking about backup storage, RAID’s, etc).

Human beings love redundancy when it comes to communication; our meanings get emphasized by our body language, and the syntax and structure of our languages, both spoken and written, encode information in multiple ways. But redundancy opens the door to discrepancy, and discrepancy leads to ambiguity. Human beings can deal with ambiguity, but computers (and most software developers, and maybe catalogers) can’t abide it. “Time flies like an arrow, but fruit flies like a banana”.

A computer might resolve ambiguity in a non-obvious way, or worse, it may resolve ambiguity in a manner that coincidentally matches a human’s expectation, only to cause trouble later when you start getting data anomalies.

Normalization is a best practice design technique for designing relational database schemas, though there are cases when you want to make a trade-off and de-normalize certain data structures. But those cases are usually optimizations that should occur after you already have a good design as your starting point.

Relational databases are powerful enough that they have sunk into the consciousness of librarians, to the extent that they put down “Must be built on a relational database” in their RFP’s. But most legacy ILS products actually use hierarchal databases, which are good for some things, but not for others. The design decisions made when you go with a hierarchal database are very different than the ones you make when you start with a relational database, and I worry about the legacy products that have “tacked on” relational databases for buzzword and RFP-compliance. Legacy automation may now have relational databases, but are they actually using them as a relational database? Some are, but I know of at least one that isn’t.

Let’s return to Evergreen for a moment. Because Call Number Volumes are represented as their own entities, you have the option of thinking of them differently. For example, you can move a set of items to a different “call number”, one which may be associated with a different Bib Record altogether (or you could move a volume itself to a new bib record). You could even change which library “owns” a volume, and suddenly all the items attached to that volume have a new owner. And you’re able to place a hold on a specific “volume”, in addition to title-level and copy-level holds (and in Evergreen, meta-record holds across editions and formats).

But you can also do what you may already be used to, and change the Call Number for an item (or a batch of items) from an Item Editing interface, and not have to know about the more flexible structures that are being manipulated underneath.

Because we already have Volumes, we’re also in a better place for adding Serials.

There are fewer widespread repercussions to changing an end-user interface than there are with changing your database schema, so that’s why it’s important to have a good database design from the beginning, so that your interfaces have more options. To steal a sentiment from Mr. Miyagi, Evergreen has strong root.

– Jason

Next up, a pair of kudos to Koha.

Over the past weekend they added, at a mailing list member’s request, a call number browser inspired by Evergreen’s, which we call Shelf Browse. In Evergreen, because it supports a hierarchical organization of libraries, you can actually browse an entire system or even consortium as one huge virtual shelf! It’s a very nifty feature, and one that we know the PINES patrons have been making good use of (to the tune of 66,965 and counting so far this year, and about 300,000 times in 2007) since Evergreen launched in September of 2006. Now Koha will have a similar feature at the request of a small church Library! This, my friends, is Open Source at work.

By way of evidence from our users, I’ll mention that Evergreen provides call number / shelf browse as a “Quick Search” from the advanced search interface, which is useful to Evergreen users and may be useful for Koha patrons as well. In any case, good work.

I also noticed that Koha has incorporated, as of November of last year according to their source repository’s timestamps, the SIP2 code that David Fiander and Bill Erickson wrote for Evergreen. We’re glad to see the code that GPLS funded is going to good use in and inspiring other projects!

Three and a half years ago, when I first joined PINES and the Evergreen team, there was a dream and a small test server. Now we’ve written more than a quarter of a million lines of code, and that code runs the day-to-day operations of one state-wide library consortium (biggest in the world, he bragged ) with at least two more in the works, and is helping to build a province-wide consortium in Canada — and let’s not forget the Laurentian/McMaster/Windsor “Unholy Trinity.” These are amazing, and they fill my heart with a satisfaction that is difficult to describe, but none of those things, even as possibilities, are why I signed up. I joined this effort because I believe in Open Source software. I believe whole-heartedly that it is a force for positive change in an industry I love, and fits perfectly with the mission of libraries.

Again, congrats to both NewGenLib and Koha, and let’s keep the cross-pollination going.

–miker

Some highlights

My week started with the Evergreen pre-conference, skillfully lead by Dan Scott. Dan blogged about it here. Apart from some technical woes, a lot was seen and (hopefully) learned by all. We performed the install, imported bib and holdings data, implemented new functionality (new OpenSRF method and OPAC UI) to email a user their password if they have forgotten it, and took a quick look at some of the staff client interfaces.

Conference day 1 began with Brewster Kahle’s keynote address on his work with archive.org and openlibrary.org. Brewster’s vision and chutzpah is refreshing and inspiring. The goal of “One wiki page per book” sounds so simple (and obvious) when inserted into a slide show, but the implementation will require tremendous work and resources. I applaud their efforts, not only in principle, but because I think it will lead to better information and resource sharing in the long run.

After lunch, Winona Salesky and Michael Park did a joint presentation called “XForms for Metadata Creation”, which included two different MODS editors developed with XForms. I liked this talk in particular for two reasons. For starters, the demos I saw were that of a simple (yet powerful) bibliographic data creation interface, which I see as potentially useful for any ILS. Additionally, the use of XForms, which I only have a basic understanding of, gave me a chance to see some new (for me) technology in action. My interest is piqued.

I skipped the breakout sessions that day to put together a (5 minute) lightning talk on Pylons. One of the interesting challenges of lightning talks is the use of a shared PC and what amounts to a moratorium on web access for the sake of efficiency. My presentation was a set of images and screenshots, the first a crane on a sunny Portland day, which seemed apropos of the topic. I discussed one particular aspect of Pylons, which allows you to easily plug in custom pre and post-processing middleware applications for your web apps. I demonstrated a simple XML validator and a highlighting plugin which bolded pre-defined terms in the HTML on its way to the client. These gave me the chance to dig a little deeper into Pylons, an architecture which I think has a lot of interesting potential.

Wednesday morning I learned a lot about RDA. Later, I thought the discussion on of the ILS Discovery Interface Task Force segued well into Ross Singer’s lightning talk on Jangle, which, as Ross pointed out, could be used to implement the proposed standardized ILS interface.

I was not able to attend the last day, so I missed a chunk of the conference, including Dan’s CouchDB talk. Glad it went well, Dan!

Wow, now I realize why most folks blog about conferences as they are happening It’s a lot to digest, especially for a conference as dense as code4lib. There were a lot of great presentations, breakout sessions, and lightning talks this year and this post only comments on a few of them. Next year’s conference will be held in Providence, RI, and if you can, I would recommend checking it out.

-bill

Maximum circs by DAY:
96,326 – May 29, 2007

Maximum circs by HOUR:
11,305 – May 29, 2007, 11 AM

Maximum circs by MINUTE:
548 – May 2, 2007, 9:21 AM

I understand that the day after Memorial Day in 2007 is remembered as a very busy day for reasons that are clear.

For a bit of perspective, the FY 2005 NCES/NCLIS data for all US public libraries give the following annual figures for total circulations by library:

Q3 (third quartile–the libraries above this number are in the top 25%):
144,663

Q2 (median–the middle value):
43,912

Q1 (libraries below this number are the bottom 25%):
13,113

The arithmetic mean is 229,685.

That maximum number of circs for PINES for one day is greater than the median annual value for all public libraries.

Why is the mean so much higher than the middle value? Because the distribution of all these numbers is highly skewed–in the Statistical sense. And that fascinating topic can be the subject for compelling blog post at another time.

Bob Molyneux

I am a single-branch public library with about 25,000 titles. I’ve looked at the options and I really like Evergreen. But isn’t it simply ‘too big’ for my situation?

It is true Evergreen was built from the ground up to handle the largest of library systems: it’s highly scalable, fault-tolerant (meaning it can sustain hardware failures and keep running), and has capabilities that make it a perfect selection for large libraries and consortia. Those attributes/goals are clearly stated on the open-ils community website.

However, it is important to realize this does not mean Evergreen is not a good solution for smaller libraries as well. Evergreen does in fact “scale down” elegantly. We know this because there are developers in our community that run Evergreen on their laptops. That’s right folks–they can run the entire server-side software stack on their laptops. Another example: Bill Erickson, at the last code4lib conference, ran an “Evergreen install in 20 minutes” breakout session in which Evergreen was installed on a laptop.

Evergreen was designed to be able to scale to multiple servers of varying sizes and purposes. But it can also scale to a single server (or laptop) if that’s what the library desires and what makes sense in the cost-to-uptime ratio.

Also, the ability for Evergreen to “scale down” from a functional standpoint is something that’s apparent in Kent County Library’s recent migration announcement. Kent County Library serves a population of approximately 19,500 though a main library and 2 smaller branches. There is more information about the decision to go with Evergreen on the Kent County Library blog.

I hope this information puts the rumor to rest that Evergreen is just for big libraries and consortia. The fact is Evergreen can work quite well in smaller situations.

PINES, of course, runs on Evergreen and although Evergreen is running individual systems, the fact is that Evergreen is the most consortially-aware ILS—open source or proprietary—has resulted in some fascinating dynamics which these cited posts allude to.

In my post, Save the time of the reader, I reported on my attempt to measure the change year over year in three PINES use measures that could be compared before and after the switch from the legacy vendor to Evergreen. There are dramatic increases which I attributed largely to Ranganathan’s law it referred to because the reader’s time was saved as a result of better design of the interface, particularly of holds.

However, there is a measurement problem that brings out something more important that had implications I didn’t understand a month ago. Not all public libraries in Georgia are in PINES and the number changed over the period of my analysis. If we want apples to apples, as the saying goes, we should analytically remove the libraries that were added during the year so that we would have circs, holds, and such at the same set of libraries, that is, so that increases reported are not because of the addition of new libraries but rather because of easier to use software with better features. People, though, are just a bit complicated and that simple statistical manipulation would miss something key.

Georgia has a universal borrower card so if your local library is not in PINES, you can go to one that is and borrow from the PINES system. And what library users are doing is pretty clear, they are making the drive: they want access to the 2 million bibliographic entities and the over 9 million items owned by the libraries in PINES. As a result, the obvious statistical treatment itself would miss this dynamic aspect of people’s behavior. They don’t hold still and they react to changes in their environment.

To come full circle, I offer this speculation: that some of the growth in the number of PINES systems (44 to 49) since Evergreen went live in 2006, may be a result of this behavior.

I was graduated from library school in 1971 and I have seen various fashions about how best to do library service: big regionals or small local and intimate libraries. Back and forth. Back and forth. I believe the jury is now in. Library users are voting with their feet; they want the big, resource rich library.

Chris Jowaisis’s Texas Library Systems wiki has an ILS Discussion in which he offered an opinion I thought insightful about the Georgia experience: “(CJ opinion) political agreements are as impressive as the technology achievements of Evergreen.” I have quoted Chris’s observation often. Now, after seeing the experience in Georgia up close, I think that while we librarians and politicians care about the politics, the people who use libraries don’t. If you build the resource sharing consortium, they will come.

Bob Molyneux

I was wondering if any of you would be interested in gathering together to talk about Evergreen during the Public Library Association’s National Conference (http://www.placonference.org/) in March? Most of the Evergreen developers will be there and we can arrange for a meeting space. The notion is that anyone interested in the project (software, community, vendors, etc.) could be invited, and we could style it as a Birds of a Feather-type gathering, or maybe even the nucleus of an independent Evergreen Users Group.

What do you think? What sort of topics or activities would you be interested in? Would you be interested in something like this at other times or places? Is anyone interested in helping to organize this?

Thanks!

– Jason

LOCATION UPDATE:March 27, 5:30pm – 7:30pm at the Minneapolis Marriot City Center, Gray & Wayzata rooms (8th floor)

We only have fragmentary data from before the change but the following table shows changes in selected measures of use of the PINES system. Year over year changes are the best way to calculate this kind of change because there are seasonal effects—for instance December is generally a slow month—and I have chosen to use percentages rather than raw numbers to make comparisons easier. In the table, the row “September 2007″ lists the percent increases in the various raw figures of September 2007 over September 2006. Thus, September 2007 saw an increase in PINES libraries’ circulations of 10.1% over the September 2006 figures. Blanks in the table indicate there are no matching data to calculate percentage increases.

The table below has percent increases in circulations, total holds, and unique patrons requesting holds. Holds are those place by users in the library when some cataloged item is not available. Although Evergreen is used to measure the in-house use of both of cataloged and of uncataloged items, these types of circulations are not included here. The circs are by the standard definition—these are items going out the door.

We can see that circs have increased a bit over 10% for the four months we have matching data. Holds increased a good bit more and we have data for a longer period. The range of increases of these holds is from a bit under 30% increase to over 50%, depending on which month one examines. The change in holds is dramatic.

% increase in



% increase
% increase in
Unique Patrons



in Circulations
Total Holds
Requesting Holds


September, 2006
 
 
 


October, 2006
 
 
 


November, 2006
 
 
 


December, 2006
 
 
 


January, 2007
 
35.2
 


February, 2007
 
40.8
 


March, 2007
 
39.2
 


April, 2007
 
50.0
 


May, 2007
 
45.9
 


June, 2007
 
38.5
 


July, 2007
 
52.0
 


August, 2007
 
38.4
 


September, 2007
10.1
51.4
39.5


October, 2007
15.0
36.5
30.4


November, 2007
10.0
29.7
25.0


December, 2007
11.1
28.9
25.5

The right column shows the increase in unique patrons requesting holds. This column presents an interesting set of statistics because we see a striking increase in the number of patrons using this service.

So, the number of people requesting holds is going up, too. The number of holds per person (not shown) is up only slightly averaging fewer than 4.

Why these increases? I think we are seeing an “Evergreen Effect” which I would define as increases in library use brought on by saving the time of the reader. I don’t believe Ranganathan would be surprised by these data. Those readers are responding to a lower bar between them and materials they are seeking by using the PINES system and materials from its constituent libraries more. Evergreen was designed from the ground up with close consultation with librarians, Georgia Public Library Services staff, and library users. Modern notions about presentation of data were included from the beginning, not bolted on to an aging system. Hence, it is easy to use and intuitive.

These figures overstate the changes slightly. The number of systems at the start of PINES was 44 (including the State Library) and by December, 2007, there were 47. We did look at backing the data from the newer libraries out and that would decrease the raw numbers and percentages about 4%, we estimate.

But we ran into a problem and that was that people living in the service areas of libraries not served by PINES have been using PINES libraries in order to get access to the large, virtual library. There are currently 1.82 million titles in PINES libraries with 9.4 million copies. As I said: a large virtual library and one with many resources to share to library patrons with a PINES card. Any Georgia citizen can get one and we know that library users from non-PINES are availing themselves of these resources and 1.82 million titles is a powerful draw and, not inconceivably, a draw for some of the non-PINES libraries to join. In any case, if we took the effect of the newer libraries out, many of the users in the new systems were already users of PINES. There is an apples to oranges comparison but the currently unmeasurable effect is likely small—probably lower than 4% but we just can’t measure it. So, for December 2007 increase in circulations over December 2006 might be as low as 7%. The holds might also be about 4% lower, that is, perhaps 25% increase in total holds for December 2007 over December 2006. Teasing out a better number is going to be tricky.

Laurentian University, my employer, recently issued a press release announcing that we have selected Evergreen as our future library system. I think this is big news, because Evergreen has thus far been seen primarily as a public library system. We recognize that there are some major tasks to accomplish before we can migrate to Evergreen — acquisitions, serials, academic reserves, documentation — but we’ve observed and participated in the Evergreen development effort, were encouraged by the results, and therefore have chosen to contribute resources to the development effort to help evolve Evergreen towards what we need from an academic library system.

We believe that Evergreen represents the best possible path towards a library system that will meet our users’ current and future needs. Evergreen does not repeat the mistakes of integrated library systems of the past: its service oriented architecture and Perl, C, Java, and Python bindings make it an integrated, yet loosely coupled, library system that can also integrate with our existing campus applications.

The University of Windsor took the important first step of establishing a community for academic institutions by partnering with GPLS to work on the acquisitions module. By selecting Evergreen as our future library system and contributing to the development effort, Laurentian University is taking the next step in solidifying that community — and based on the interest that has already been expressed by other academic institutions (both directly to us, and publicly on the Evergreen mailing lists), we look forward to other institutions joining us in building a strong, collaborative, and innovative community.