Structured library data: holdings, libraries, and beyond

MARC: (still) kind of a big deal

If your holdings are not in OCLC, you're not linked to Google Books.

Our decentralized infrastructure has been centralized.

Linked data library system leadership

• Some library systems have supplemented or discarded MARC for a linked data model:
• Swedish Union Catalog
• German National Library
• Bibliothèque nationale de France

But is your library...

• Large (national scale)
• Able to sustain multi-year development efforts with concurrent systems
• Able to mandate significant changes to established practices
• ...?

Better living through Web standards

• Hypothesis: We can iterate towards linked data with existing systems
• Enhance our catalogues with web standards:
  • Persistent URIs
  • HTML5
  • RDFa (or microdata) expressing schema.org
  • Sitemaps listing all the URIs of interest

Reality check: 2012 Common Crawl

• Ronallo(*) found that American academic libraries published under 10,000 schema.org instances in total
• Possible reasons for this low adoption rate include:
  • Perceived return on investment is low in risk-averse, under-resourced institutions
  • Proprietary systems do not facilitate shared modifications of HTML templates
  • Sufficient access to underlying metadata may not be available in proprietary systems

Crawling: Evergreen early days

• Simplistic implementation of schema.org via microdata with the 2.2 and 2.3 releases:
  • title/author/keyword properties
  • Direct plain text values (no embedded types)
• 2.4 release broke entities out as Person and Organization types, separated birthDate and deathDate values
• 2.5 release switched to RDFa, exposed holdings as Offer types
• 2.6 release includes links from holdings to Library types

Open source implementations

• My efforts have:
  • Enhanced Evergreen, Koha, and VuFind to publish schema.org data
  • Enhanced Evergreen to expose Library metadata (location, contact information, hours of operation) linked from offers
• Independently, the Blacklight library system was enhanced to publish schema.org data
• Approximately 4,000 library systems will publish schema.org data as sites adopt the latest releases

SchemaBibEx: Citations everywhere

• Originally only the MedicalScholarlyArticle type had a citation property
• Alf Eaton, via SchemaBibEx, proposed making the property applicable to all CreativeWork types
• Status: accepted

SchemaBibEx: Library items as schema.org Offers

• GoodRelations vocabulary for surfacing product information, based on an agent-object-promise model, was incorporated into schema.org
• Library-specific holdings vocabulary seemed unlikely to gain traction from search engines
• Libraries therefore offer items for "lease" at zero cost
• Status: SchemaBibEx recommended practice

Mapping holdings to schema.org Offers

Holdings as Offers: example

<tr property="offers" typeof="Offer">
  <td>
    <a property="seller" typeof="Library" href="http://example.org/library">
      <span property="name">C.H. Booth Library - Newtown</span>
    </a>
    <link property="businessFunction" href="http://purl.org/goodrelations/v1#LeaseOut">
    <link property="itemOffered" href="#schemarecord">
  </td>
  <td><span property="sku">641.5 ROM c. 2</span></td>
  <td property="serialNumber">34014012862984</td>
  <td property="availableAtOrFrom">Adult Nonfiction</td>
  <td><link property="availability" href="http://schema.org/InStock">Available</td>
</tr>

SchemaBibEx: Periodicals

• schema.org had an Article type and an unstructured citation property
• Focus on online access overlooked relationships to publications and print attributes
• Drove a Periodical extension proposal that supported journal/magazine relationships:
  • Article (and all child types such as ScholarlyArticle)
    • :: isPartOf :: PublicationIssue
    • :: isPartOf :: PublicationVolume
    • :: isPartOf :: Periodical
    • :: isPartOf :: Book
• Enables structured inline citations, regardless of citation format!
• Status: Under consideration by schema.org

SchemaBibEx: Comics

SchemaBibEx: Comics

SchemaBibEx: Comics

• Collaborating with Henry Andrews (comics.org) and Peter Olson (Marvel)
• Specialization of Periodical
• Depends on outcome of contributor/role discussion for SportsEvent proposal
• Status: In progress

SchemaBibEx: overall status

• Exited active extension proposal development mode
• Documenting best practices and supporting adopters
• Will call ad-hoc meetings for new extension requirements
• Join our community and post questions to the mailing list!

Structured library information

• schema.org offers a Library organization type
• ILS often tracks library hours, contact information, address, branch relationships
• Evergreen, as of 2.6 release, will expose this information by default
• You should not have to maintain business pages in umpteen different social networks!

Structured library information example

<div vocab="http://schema.org/" typeof="Library">
    <h1 property="name">Example Branch 1</h1>
    <div><a href="http://example.com/BR1" property="url">Library web site</a></div>

    <h2>Opening hours</h2>
        <div property="openingHoursSpecification" typeof="OpeningHoursSpecification">
            <link property="dayOfWeek" href="http://purl.org/goodrelations/v1#Monday">
            Monday: <span property="opens">12:00 AM</span> - 
                    <span property="closes">12:00 AM</span>
       </div>
    
    <h2>Contact information</h2>
        <div>Email: <a href="mailto:br1@example.com" property="email">br1@example.com</a></div>
        <div>Telephone: <a href="tel:(555) 555-0271" property="telephone">(555) 555-0271</a></div>

        <div property="location address" typeof="PostalAddress">
            <h3 property="contactType">Mailing address</h3>
            <span property="streetAddress">BR1<br>123 Main St.</span><br>
            <span property="addressLocality">Anywhere</span><br>
            <span property="addressRegion">GA</span><br>
            <span property="addressCountry">US</span><br>
            <span property="postalCode">30303</span><br>
        </div>

    <h2>Branch relationship</h2>
    <div id="branch-info">Parent library: 
        <a property="branchOf" href="/eg/opac/library/SYS1">Example System 1</a>
    </div>

Removing the human link to library information

• I know some libraries aren't thrilled by the holdings links to library pages
• Open up Open-ILS/src/templates/opac/parts/library_name_link.tt2 and change:

      IF lib_url; '<a property="seller" typeof="Library" href="'; lib_url | url; '">'; END;
      '<span property="name">'; org_name | html; '</span>';
      IF lib_url; '</a>'; END;
  

• to:

      IF lib_url; '<span property="seller" typeof="Library">
      <link property="url" href="'; lib_url | url; '">'; END;
      '<span property="name">'; org_name | html; '</span>';
      IF lib_url; '</span>'; END;
  

Thought experiment

• So we're exposing our holdings to the web...
• in a common format understood by search engines...
• linked to the library that holds the items...
• with that library's physical address, hours of operation, web site and contact info...
• ...
• when can we stop doing batch uploads to OCLC and/or other union catalogues?

Sitemaps

• Standard documented at http://sitemaps.org
• Simple XML lists of URLs that should be crawled, with optional change information:

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://laurentian.concat.ca/eg/opac/record/100000?locg=105</loc>
    <lastmod>2009-05-03</lastmod>
  </url>
  <url>
    <loc>http://laurentian.concat.ca/eg/opac/record/100001?locg=105</loc>
    <lastmod>2009-05-03</lastmod>
  </url>
 ...
</urlset>

• Basic SQL for an Evergreen sitemap-generating script: Google Docs

Quick union catalogues

• Many libraries have resource-sharing agreements, but infrastructure requires crufty Z39.50 lookups or periodic MARC batch loads
• Promise: Common vocabulary and mapping practices should ease cross-system integration
• Use case: Google Custom Search Engine (CSE) supports facets based on structured data such as schema.org
• Result: It works!
  • Combined Evergreen + VuFind instances in a single search instance
  • Faceted by schema.org author value
  • Exposes Google's simplistic string-based implementation of structured data

Quick union catalogues: in progress

• Schema UnionCat: < 100 lines of Python to crawl sitemaps and extract complex RDF
• Currently a proof of concept
• Future enhancements (if there is interest):
  • Only crawl new and updated sitemap entries
  • Store and search extracted RDF
  • Check item availability on demand

Evergreen: next steps

• Extend the item type mappings (fixed fields / MVF / CRA) to more granular schema.org types
• Turn the rough sitemap SQL into an actual script
• Track the impact of this effort on relevance in "normal" search engines

Resources

• RDFa Lite 1.1 W3C recommendation
• schema.org - official web site
• schema.org hands-on code lab - teaches you step by step how to add schema.org to a web page using RDFa
• W3C Schema Bib Extend Community Group - best practices for schema.org bibliographic use cases, extension proposals in progress, mailing list for assistance
• goo.gl/SO3WwZ - these slides