High-level preliminary assessment from Evergreen Documentation Survey

Karen G. Schneider, Equinox Software, and Karen Collier, Kent County (Md.) Library

September, 2009

Recommendation

The needs assessment group of the Evergreen Documentation Interest Group (DIG) recommends that activities to produce single-source, XML-based project-wide Evergreen documentation commence immediately in these four areas: reports; installation, upgrading, and migrations; cataloging; and circulation. (Note: the DIG concurred at the September 9 meeting, and work has commenced.)

Discussion

The needs assessment functional workgroup of the Evergreen Documentation Group designed, tested, and then conducted a survey from August 12 through August 20, 2009. This was a wide-open survey, with no limits on who could respond, and was intended to be useful but not scientifically rigorous.

The survey had a predictably strong response in the first 24 hours after its announcement, typical of most online surveys, with 84 responses accrued within the first four days—a realistic and useful response rate. The following Monday, August 17, after another survey reminder, the survey experienced an unusually strong spike in response rates.

This second wave of responses, combined with other indicators, suggest that there may have been some “survey-loading” to ensure varied demographics were represented. This was actually not discouraged, and overall, these responses tend to cancel one another out. However, in some cases we took the “pre-wave” data (the 84 responses received before August 17) and give it further manipulation, then compared results. We also took a quick look at crosstabbed results, such as by status with Evergreen (running live, preparing to migrate, testing, actively evaluating, or considering).

Narrative Review of Responses

Responses: 273; 222 completed all applicable questions, for a completion rate of over 81%. Most respondents completed most of the questions (a more meaningful measure, since none of the questions were mandatory).

73% of respondents reported that they worked in a public library. Over 10% of respondents work in a library consortium. Respondents in academic and public libraries were asked to identify their library’s size. 45.7% said “medium,” though large libraries were almost 15% of the responses.

We encouraged a broadly representative response, which is what we got to the question, “What are your roles in your organization?” The top responses were from library circulation (48%), followed by administration (43%), cataloging and training (each 40%), but responses came from all over the spectrum — not only the 12 identified areas, but over 2 dozen other roles as well, from pages to outreach librarians to project managers.

Over 100 respondents were also in some technical role, such as system administration, technical support, or development. 91 respondents claimed to work in system administration (39 prior to 8/17).

Four out of five respondents were from organizations actively running Evergreen or preparing to migrate their first libraries. For those libraries running Evergreen, over 70% had been running Evergreen less than one year.

Priorities for Evergreen Documentation

Overall, 79% of respondents thought version 1.6 (versus earlier versions such as 1.4 or 1.2) should be the highest priority for centralized, Evergreen-wide documentation efforts. This did not significantly shift when compared by role.

Based on all responses, the top six priorities for first topics for Evergreen-wide documentation are:

Circulation 71.8% (153)
Cataloging 59.1% (123)
Reports 58.8% (120)
Installing, upgrading, migrating 50.5% (99)
Local administration 50.2% (102)
System administration 46.8% (94)

Based on the 84 responses received before August 13, the priorities are similar in most areas:

Installing, upgrading, migrating 69.8% (44)
Circulation 57.4% (39)
Local administration 58.5% (38)
System administration 63.1% (41)
Reports 52.3% (120)
Cataloging 51.5% (34)

The key point is that there were no drop-offs—all six topics remained the highest priorities, pointing to a clear mandate for practical, field-oriented documentation. (For example, there had been discussion in the community that online help in the OPAC may be a high priority, but this is not indicated as a high priority by the survey results.)

Furthermore, when results were crosstabbed with the respondent’s role, there were some predictable fluctuations. Migration was a high priority for close to 90% of those migrating to or running a test instance of Evergreen, while only a high priority of 50% of circulation staff.

Meanwhile, reports documentation was a high priority for 73.9% of respondents reporting that they were in libraries running Evergreen live in production, where presumably there is always a real-world need for running and manipulating reports, but for only 14.3% of respondents “actively evaluating Evergreen.”

Formats for documentation

Priorities were at 68.2% each for Web-based online documents and PDFs (over 90% for libraries running a test instance or actively evaluating Evergreen).

Overall, nearly half of all respondents (48%) indicated they would like to see context-sensitive help in the staff client, though this fluctuated widely by status with Evergreen, with a low of 42.9% from libraries running Evergreen to a high of 72.7% for libraries either actively evaluating or considering Evergreen.

On languages for documentation, 76.7% indicated a need for Spanish, while 20% asked for French, and Czech and Armenian received some votes as well.

The following question was based on the results of a (unanimous) vote at the Documentation Interest Group founding meeting in May, 2009: “We are considering implementing the ability for registered users to add comments to each section of the central online version of the documentation. How important is this feature?”

For the total survey, 21.1% (47) replied “Absolutely, please make this happen”; 64.6% (144) identified it as “A nice feature, when you can get around to it”; and 14.3% (32) indicated it was “not that important.”

The comment feature was rated even more highly by the early, pre-8/17 group; of them, 36.5% said “please make it happen,” and fewer than 5 percent thought it was “not that important.” This again fluctuated by status. 52% of the respondents from libraries running Evergreen rated this a “nice feature,” compared to 66.7% for libraries preparing to migrate and 81.8% for libraries actively evaluating Evergreen.

More Documentation Lurking in the Wild?

35 respondents indicated they had local documentation they would be willing to share with the Evergreen project (that had not been shared in the past), with responses in every category, ranging from web developers’ references to sysadmin and circulation. Their responses will be filtered against the contact information they provided in the survey.

Moving Forward

Finally, the best advice the DIG got was in one of the 39 responses in the final comments section: “Soldier on!”