Adding MARC shelfmarks to the bibliographic display

Our main Library Management System is down at the moment. It meant our main Aquabrowser based catalogue worked, but was unable to show item holdings/availability/shelfmarks. For users, this made it pretty useless, knowing we have a book is a little useless if they don’t know where it is. The fact that our classification system is modified from the standard Library of Congress classification meant we can’t just refer users to similar University catalogue’s for a shelfmark.

So as a work around I added the Shelfmarks held within the MARC record on to the standard vufind (sabre) display.

Basic item information is held in the MARC record, you can see an example below in the 852 field. This example has five copies.

Image

Steps to add this:

First I edited web/RecordDrivers/MarcRecord.php. 

/**
* Get item information from MARC 852 - useful if lms system is down for classmark
*
*/
protected function getItemHoldings()
{
return $this->getFieldArray('852', array('h','c','b'));
}

This calls another function in this file which does the heavy work of getting all the fields and sub-fields out of the record. You can see I’m using subfields h, c and b.

Next, I edited IndexRecord.php. Adding the following to the function ‘getCoreMetadata()’ 

$interface->assign('coreHoldings', $this->getItemHoldings());

If you’re following along at home you will find the top of the function has a number of similar lines and I just added it straight after these.

Finally I edited web/interface/themes/default/RecordDrivers/Index/core.tpl

I added this near the end:

<!– CJK show holdings info –>
{if !empty($coreHoldings)}
<tr valign=”top”>
<th>Shelfmark: </th>
<td>
{foreach from=$coreHoldings item=text1 key=text2 name=loopholdings}
{if $text1 neq $prevholding}
<strong>{$text1|escape}</strong> <br />
{/if}
{assign var=’prevholding’ value=$text1}
{/foreach}
</td>
</tr>
{/if}

This will display the shelfmark if we have one (or more) shelfmark, and only show each unique classmark once, i.e. if there are several items with the same shelfmark/collection/library then only one shelfmark is displayed.

If I was doing this properly the ‘Shelfmark: ‘ would be in the language file and not hard coded in.

End result:

Image

Link to the record: http://sabre.sussex.ac.uk/vufindsmu/Record/9780702029936

(note at the time of writing this is a temporary measure and will probably be removed once the issue is resolved)

 

Advertisements

Final Blog post

Background

The Brighton and Sussex Medical School is a joint venture between the University of Brighton and the University of Sussex (and the local NHS Trust). There was an identified need to provide a unified search interface for the two University Libraries, medical students had to search two or more catalogues to find where a book may be located, there resources are divided between the two Universities and also at the local NHS library. Furthermore a combined catalogue would be of use to other local researchers, both at the Universities and in the surrounding area.

Intended outcomes

  • A catalogue interface allowing users to search the records of both libraries, allowing medical students and others to quickly find and locate an item in either library.
  • A review of the two Open Source systems which we had shortlisted: VuFind and Blacklight.
  • User Testing of the chosen system

The Challenge

Medical Students had to search at least two systems, often more: two University library catalogues, e-journal/resource systems, NHS Library etc. While some Union catalogues exist, such as COPAC and Inform25, we were aware of few catalogues to cater to such needs, especially with limited resources.

Established Practice

As noted, Medical students and staff had to use a number of systems, performing the same search, to find an item. Any other user who was looking to see if an item was in either library would also need to search both separately.
The University of Sussex use Aquabrowser as their main Catalogue web interface. The system provided a lot of modern features, though did not provide My Account or reservation functionality. It also provides Prism2 (the web catalogue bundled with the Talis Alto LMS) as an alternative catalogue interface, and the only method to renew books and place reservations. Each night the Talis Alto LMS would export a completely dump of MARC files, to be indexed by Aquabrowser, as such as knew this functionality existed and worked.
The University of Brighton use Prism 2 as the main catalogue interface.

The LMS advantage

VuFind, the chosen Open Source system, provides a modern, quick, easy to use system that can search both University Libraries. It provides the clear benefit that users only have to search one catalogue to search across the two Libraries, it does not replace the existing catalogues, but provide an additional service to those who find it useful. As well as the bibliographic records it also provides holdings/availability/location information in an easy to ready format, no small feat when there are no standards for this.
Additionally the service – sabre – provides other benefits.
  • a backup catalogue should either main system become unavailable.
  • A good mobile interface – which neither main catalogue currently offers
  • options for different languages and browse options
  • export options.
It’s also helped us become familiar with working with our MARC21 records, and the options available to us when we require a new catalogue interface. Setting up additional VuFind interfaces becomes fairly trivial.

Key points for effective practice

  • User testing is incredibly useful and should be carried out for all new services
  • exporting all MARC records can take time to perfect, ensuring all are being successfully exported and imported. A recent blog post links to some tips regarding this.
  • VuFind has a good user/developer community, it is easy to adapt to meet local requirements and provides a lot of modern useful features that users like. Blacklight also looks like a very promising package.
  • VuFind has an exciting development roadmap, for example with support for EAD and other metadata types.
  • There are no standards for holdings/availability/location. While the international library community has made some efforts in this area, nothing has really come of it of late. This needs to change and an open standard, supported by all major suppliers is required. Meanwhile Libraries need to be prepared to allocate time and resources in to this functionality. Talis Alto had a relatively small user base, based mainly in the UK and Ireland and therefore lacks the potential community of VuFind users who use the LMS (at present ifind, developed at the University of Swansea and sabre, developed at the University of Sussex are the only two).
  • Likewise, no standards for allowing one system (i.e. the catalogue) to renew/reserve items on another (LMS) exist. VuFind does provide this functionality but it is dependant on a working driver for the LMS.
  • jangle provides a middleman between LMS systems and client applications (such as third party catalogues). It could avoid the need for every client system to provide a way to connect to every server, instead each just needs to be able to communicate with jangle (i.e. from Many-to-Many to One-to-Many). It would be good to see suppliers adopting this. Though jangle itself does require resources and time to setup.
  • VuFind could be used for many niche and specialist requirements, or even as a backup catalogue. Perhaps providing a search interface for a subset of records (just AV, or a particular library). If development time is tight then more complex features, such as ‘My Account’ can be hidden. A simple bibliographic search is very quick and easy to setup.
  • Be clear what can be offered, and what you plan to offer. If you are planning to provide a ‘quick win’ solution to a problem, be clear that not all of VuFind’s functionality will be available.

Conclusion and recommendations

  • Many of the points above can be seen as recommendations
  • User Testing can be fairly easy to perform and is incredibly useful
  • VuFind is flexible and relatively easy to work with, it can be used as a replacement catalogue, or as here used to cater for a specialist need.
  • The Library community urgently needs a standard – supported by vendors – to provide holdings/availability/location. Additionally a standard to support My Account / reservations and renewals.
  • Library suppliers need to understand that providing open, unrestricted APIs and data export is part of the service/system Libraries pay for, not a threat to ‘bolt-on’ products. If you system does not work in our eco-system then it is a bug.
The sabre service can be found at: http://sabre.sussex.ac.uk
Regards
Chris Keene

Processing MARC records

To keep the records in sabre up to date, we need to copy over the MARC records from the two systems; pre-process them a little and then import them in to vufind. The pre-processing script was created by the University of Swansea and adds, amongst other things, a subfield to record which library(ies) it belongs to, and also deduplicates records for the same item between the two catalogues.

During the pre-processing stage we were having some problems, some of the MARC files were not being fully processed by the perl script, resulting in missing records, due to unusual characters in certain records.

This blog post (on another blog) outlines how we found a work around for this.

We’ve also uploaded a few scripts on to Google Code: http://code.google.com/p/credaul/source/browse/

VuFind Orientation

This post will try to be a brief guide to getting a feel for administrating a VuFind installation.

It’s not a definitive account, just my own experience so far. I’m still learning.

Installation

VuFind uses Apache Solr to index records and provide such things as facets. Some have described Vufind as a lite frontend to Solr, but I feel this is a little unfair, a car is more than an engine, and VuFind does more than just return search results.

VuFind is PHP based, making use of various PEAR libraries and uses MySQL for various settings. Solr is written in Java and will essentially run a web server on a different port which VuFind will use to query it. VuFind uses Smarty for templates.

The VuFind installer takes care of all of this, but at times you may find it needs a little tweaking (when I used it a few months a go it wasn’t downloading/installing Smarty correctly.

So in all, VuFind requires a based LAMP stack (Linux , Apache, MySQL, PHP), PEAR, Java (Tomcat), Solr, and Smarty. Other OS can be used with a little extra work.

I’ve also created an install script which works on Fedora based systems (I tested it on the Amazon AWS Linux installation which is based on Fedora). The difference with my script is that it will allow you to install multiple instances of VuFind on to the same server with ease. It uses an instance name which is used for the installation directory, url, database name etc, it also uses a different port number for the Solr server for the instance. Even if you don’t use the script I recommend you look at it to get a feel of what it is doing.

The installation notes for the CReDAUL project can be found here.

Useful urls

Vufind comes with an administration site. http://vufind.yourdomain.org/vufind/Admin/Home

This provides a way to view some statistics, some configuration (including ability to edit), and internal Record view. the htpasswd file is located in services/Admin directory, you can add a user to gain entry to the admin site. Also change the password on the default admin user, and consider restricting to local IP addresses.

You can also view the solr index on the port number it is running on e.g. http://vufind.yourdomain.org:17080/solr/biblio/

Layout

The top level of the vufind directory (by default /usr/local/vufind/ ) is of use for (a) starting/stopping vufind (b) importing records.

I should say here, when we talk about starting/stopping VuFind (using vufind.sh) we are really talking about starting/stopping the Solr service (Vufind wont do anything without it).

You will spend 90% of your time administrating VuFind within the ‘web’ directory, but before will move on to that we’ll mention the ‘solr’ directory.

solr – you’ll never guess – contains the solr installation. Including relevant logs and configuration. A useful file is solr/jetty/etc/jetty.xml which contains the configuration, including such things as port number and the number of threads to run. The actual index is stored in solr/biblio/index – if you’ve got a problem with your records, just delete the directory and re-import your MARC files.

Just before we move on to the ‘web’ directory I’ll mention the import directory, if contains the config for the import script which may need some tweaking.

The web directory.

These are the main sub-directories of the web directory:

  • conf – the first file you will probably play with is the config.ini within this directory. It contains a lot of settings, from the core (what URL does it use), to quite specific settings and options. You’ll also find here the configuration file for the driver which is used to connected to your ILS (aka LMS), and a few more to configure facets (which ones are shown in which order) and searches.
  • Drivers – the driver file which connects to your ILS is found here. Depending on what system you use you may find it needs tweaking to meet your needs.
  • interface – you’ll find the smarty templates in interface/themes – more on this below
  • lang – language files. Useful to define local terms, in your library do you refer to a Lending Desk, Issue desk, Service Point or Circulation desk? Vufind makes it easy to set this, for us in the English speaking world in the en.ini file, in fact this is much neater than many commercial products.
  • RecordDrivers – two key files IndexRecord and MarcRecord.php. The latter is a child/extension of the former. IndexRecord contains methods for exposing record details (for use in smarty templates) for any sort of bib record. MarcRecord extends this with MARC specific information.
  • services – key php application files. I haven’t found the need to edit anything in here yet, but can help to understand how system is structured.

Smarty

I quite like how Smarty is used, it keeps the interface quite separate from the rest of the logic and code. Though I have heard others say that it is essentially a templating system (with it’s own code and syntax) built on top of a language that in many ways was designed to be just that (PHP).
Vufind has a number of out-the-box themes, by default it uses the one called, well, default. And also mobile if it detects the user is on a mobile device.
I wont describe everything here, you can often guess which file is used for what on screen, it may take a couple of guesses but it is normally quite quick to see if this contains the html code you are interested in changing once you have opened the file.
At the top level there is layout.tpl, and footer.tpl which contain system wide layout templates. There’s also a css and a images file (there’s also a images dir higher up for images which are not theme specific).
  • The homepage is in Web/home.tpl
  • Record and Search dirs have contained most of the files I’ve needed to edit
  • You can see many Smarty tags within the files to translate terms in to the local language and local terminology. for example {translate text=”Add to favorites”} which insert what text you have chosen for ‘adding to favorites’, e.g. ‘Add to My List’ etc.
  • The Record files use variables created in the IndexRecord/MarcRecord files (see above ‘RecordDrivers’). So you can refer to those files to see which variables will contain the information you want to show. Different information is exposed to different views, e.g. the export citation option has just the information required to create a citation. If you want to add an extra bit of information to a citation then you will need to probably edit the RecordDriver files to make sure they expose the data, and then edit the appropriate smarty template file to incorporate it.

Some examples of changes

  • For internet resources we use the MARC 856 field, subfield $u contains the actual link and subfield $y contains the text to use for the link. However vufind presumes the link text will be in subfield $z (which we don’t use) so out the box it would use the URL itself as the text for the link as we had no z. To fix this we edited RecordDrivers/MarcRecord.php so that the getURLs function uses the y subfield.
  • Our setup actually searches two libraries, those of the University of Sussex and University of Brighton. We pre-process the MARC records from each University to add a 969 field for each University which has the item in question. The 969 contains the University code (susx/brighton), and the bibid the library uses for the item (if both Libraries have the item then the record has two 969 fields). When we display a record we want the user to be able to easy link to the record in the main catalogue of the University in question. To do this we edited MarcRecord to create a function which can take this MARC field and turn it in to a link (using the bibid to create a link to the catalogue record in question). I think we also created a skelton function in IndexRecord (which the function in MarcRecord over rides). Finally we edited interface/themes/default/RecordDrivers/Index/core.tpl to add the field to the default record display. [to see the MARC tags referred to here and the example above see this record ]
  • We didn’t want the search results to show availability/shelfmark. With two library systems there was too much potential for mistakes to creep in, best left to the record view of the item. Plus it meant VuFind had to get information for every item on the search results page, potentially putting a load on the original ILS (LMS) server(s). This is actually something which uses AJAX. And this is something which uses the design where a MARC record extends what a ‘default’ record can do (i.e MarcRecord.php extends IndexRecord.php in RecordDrivers). VuFind decides that if something is a MARC record it has come from a Library system and so it can try and find holdings/availability. Essentially the IndexRecord function getSearchResult() will set $interface->assign(‘summAjaxStatus’, false); so it wont try and get availability via AJAX. Where as MarcRecord overrides the function, doing all but the same with everything else but sets that value to true. The smarty template for search results (interface/themes/default/RecordDrivers/Index/result.tpl) then uses the value of summAjaxStatus in a if statement, if true it will output some html/javascript code to perform thee AJAX calcode to call and display the results. So, to do what we wanted, we changed MarcRecord to set summAjaxStatus to false (so it should always be false). And then edited the smarty file so that what gets shown when it is false meets our needs.
  • We edited the help files in interface/themes/default/Help/
  • we changed the order of facets in conf/facets.ini
  • We edited interface/themes/default/Record/view-holdings.tpl so that it doesn’t show ‘item 1’, ‘item 2’ etc next to each copy of the item (we found the numbers weren’t really helpful and didn’t mean anything).
  • And much more….
These examples show that it is quite easy to customise vufind, once you know which files to poke around in.
I can say that the only other system which compares in this flexibility, in my experience, is Eprints. I have not come across a single commercial system (catalogue or otherwise) which we use as a library which has such a structured and logical design.

We have a name

We generally avoid having special names for services. Our Catalogue is ‘The Catalogue’, and so on. We avoid specialist terms (OPAC, a term I’ve always felt was never required in the first place!), or product names (Metalib).

However, when it comes to the service being created in the CReDAUL project this didn’t work. It wasn’t replacing an existing service but creating something new: a catalogue of records from both the Universities of Sussex and Brighton Libraries. Frankly we couldn’t go around referring to the ‘Joint University of Sussex and University of Brighton Libraries Catalogue thing‘ – we needed a name.

So we held a competition amongst staff to come up with a name. We had a lot of entries, nearly a full page of A4, we have very creative staff!

In the end we decided on…..

sabre : Sussex and Brighton Resource Explorer

In my experience the words that make up acronyms tend to be lost over the course of time (COPAC, UKOLN) the name becomes just that, a name. And it’s the name we will push, a long with a brief sentence as to explain what it is (currently: Searching across Sussex and Brighton University Libraries).

Personally, I really like the name, it is easy to pronounce and is short and simple. We also asked a member of staff to come up with a logo, and an hour later he come up with three. There were all amazing!

So the homepage to the new service – sabre – currently looks like this:

sabre homepage

sabre homepage

We’ll be using the new name and logo in publicity for the new service. I couldn’t be happier with the end result.

User Experience Testing

We’ve just done our User Experience Testing. If you struggle to read more than 140 characters: It was great. Learnt lots and given us confidence to make changes.

The Vertical Slide Offices

The Vertical Slide Offices (the room where the testing took place was similar)

As part of the project we had enlisted the help of  Dr Graham McAllister in the University’s School of Informatics, who not only specialises in User Experience, but also has a spin-out company – called Vertical Slice – based at the University’s Innovation Centre which carries out User Experience Testing mainly for Video Games.

The setup

The User Experience Testing consisted of a series of one on one sessions, each lasting around 40 minutes. Graham asked a number of questions (‘how would you find x’), but also more general questions: what do you think of the site? can you tell what it does? would you use it? He sat next to the user as they carried out the exercises. He’s approachable manner, and the fact he was not connected to the new service, helped in letting the users speak freely as to what they thought.

A number of students and staff from BSMS, the joint medical school (with the University of Brighton), who will hopefully be key users to the new service, agreed to come along. It was carried out throughout the course of one day (Wednesday the 11th May).

On the day

I was sitting in the Vertical Slice offices near to where the testing was taking place, and was able to watch the Video recordings of the screen/video of the user/audio throughout the day.

It would be unfair to say I was cynical of what value it would bring, that would be too strong. But I wasn’t fully sure how useful it could be. They would search for books etc, they would probably find them, some things we know would be harder than others.

Within two minutes of watching the first recording any doubts had been blown away. You could see the frown on their face as something wasn’t obvious and the eyes widen when they spotted what they were after. The mouse would dance between two parts of the screen as they quickly thought through whether to perform a search or try and browse.

An example: we had added a link back to the originating library’s catalogue (or both catalogues if they both had a copy). This, I thought, would be useful for doing anything you could not do on the Vufind interface itself. Yet each time someone was asked how they would reserve a copy, or some other similar task, a frown would appear, then a gap, then a ‘no I don’t know how I would do that’. After the first one I wanted to dive in and start making changes to improve it.

Another example was the filter restrictions that could be applied. For us, a key one is to restrict by institution. However once a filter is applied it continues to be applied to future searches. It does say on the right the filter is applied, and under the search box there was a ticked option to ‘Retain my current filters’ but these things do not matter. What matters is that time and again they were not getting back the results they expected which lead to confusion as it was not clear the filter from before was still being applied. We need to make the active filter much clearer and try and make it impossible to miss, using colour and screen location.

Findings

  • On the Homepage: It’s not clear what the service does when you first look at the homepage (though on the whole comments were positive to the clean and modern look).
  • The browse options on the homepage (shelfmark, language, item type) – while potential useful at times – distracted more than the use they provided.
    Further more, while these seemed like a good idea when I first encounted them – from a technical perspective making good use of the MARC metadata – in fact they can be very misleading as users quite rightly think that when you click on ‘Medicine’ or ‘ebooks’ you will see all items for that selection. In reality due to the lack of detail in older records and the different ways the two Universities have catalogued their collections these browse options rarely displayed all items of the selected choice.
    Another reason, users would click on Medicine and be presented with the first few items of the very large list of items classified in the Library of Congress Medicine (R) range. However these first few items were not particularly good examples, or the type of books you would expect to show up, which confused people.
  • The is perhaps part of a more general trend throughout; simplify where possible.  Users actually said ‘all this stuff is getting in the way‘, ‘why am I being shown this information?‘, ‘There might be an option to do X but I can’t see it as there’s so much stuff‘. This was a clear directive: hide whatever is not needed.
  • On the search results: need to make it clear what filters are currently applied, probably needs to be at the top, just above the results and hard to miss. Again, users were not always sure if a filter had been applied (‘did that do anything?).
  • Sort Date Ascending/Descending : while not hard for the average person to work out what these mean, they do make most people stop and think to make sure they have got it the right way around. This is bad. So use wording that doesn’t require people to think it through (‘oldest first’).
  • On Record pages: Make it really clear which Libraries have this item.
  • Clear ‘action’ options: Reserve this, etc.
  • Use Shelfmark, not Call number. Make sure American English is changed to British English, Catalog to Catalogue.
  • Remove Tags/Comments, unlikely to be used and, as noted above, hide the more important options and information.
  • While getting Holdings information from the two underlying systems is somewhat of an art, need to continue to refine how this information is displayed.
  • Generally: need more information as to what it does, and a dedicated feedback page, which has clear information as to how to contact the two Libraries regarding items and borrowing, and contact us (well, me) regarding comments and issues with the system.

A full list of changes we are going to make can be found here.


What next

Over the next couple of weeks we will be making changes to the user interface based on this feedback. We are using Git to keep track of changes to the system (code/config/language&wording/template) and a Google Spreadsheet to keep track of what has been changed, and what we still need to do.

jisclms

This project is partly being funded by the JISC as part of their Library Management Systems Programme (referred to as jisclms). The JISC provide services and fund projects related to developing University IT, Libraries, web services, e-learning and research.

The various projects (all based at UK Universities) which are being funded by jisclms recently met up in Glasgow for a two day meeting. This was a incredibly productive event, to discover what others are working on,  share experiences and , and discuss areas and issues which we currently face with library management systems. The latter included issues around OSS LMS (what are the issues, and how could they see the introduction in UK academic libraries), Shared Electronic Resource Management and Usability/User Experience.  What was clear was that the benefit of the projects as whole was greater than the sum of each project combined. Each project requires that University to reuse, open up, or export data in a new way. It forces openness and the net effect is that there will be multiple people around the UK who have recent experience in using LMS data and using it with new applications. This helps create a community who and share and promote such innovation, and help others who are interested in trying such things. It helps to set a ball rolling, or rather, gives a ball which was starting to move slowly a very big push.