Final Blog post

Background

The Brighton and Sussex Medical School is a joint venture between the University of Brighton and the University of Sussex (and the local NHS Trust). There was an identified need to provide a unified search interface for the two University Libraries, medical students had to search two or more catalogues to find where a book may be located, there resources are divided between the two Universities and also at the local NHS library. Furthermore a combined catalogue would be of use to other local researchers, both at the Universities and in the surrounding area.

Intended outcomes

• A catalogue interface allowing users to search the records of both libraries, allowing medical students and others to quickly find and locate an item in either library.
• A review of the two Open Source systems which we had shortlisted: VuFind and Blacklight.
• User Testing of the chosen system

The Challenge

Medical Students had to search at least two systems, often more: two University library catalogues, e-journal/resource systems, NHS Library etc. While some Union catalogues exist, such as COPAC and Inform25, we were aware of few catalogues to cater to such needs, especially with limited resources.

Established Practice

As noted, Medical students and staff had to use a number of systems, performing the same search, to find an item. Any other user who was looking to see if an item was in either library would also need to search both separately.

The University of Sussex use Aquabrowser as their main Catalogue web interface. The system provided a lot of modern features, though did not provide My Account or reservation functionality. It also provides Prism2 (the web catalogue bundled with the Talis Alto LMS) as an alternative catalogue interface, and the only method to renew books and place reservations. Each night the Talis Alto LMS would export a completely dump of MARC files, to be indexed by Aquabrowser, as such as knew this functionality existed and worked.

The University of Brighton use Prism 2 as the main catalogue interface.

The LMS advantage

VuFind, the chosen Open Source system, provides a modern, quick, easy to use system that can search both University Libraries. It provides the clear benefit that users only have to search one catalogue to search across the two Libraries, it does not replace the existing catalogues, but provide an additional service to those who find it useful. As well as the bibliographic records it also provides holdings/availability/location information in an easy to ready format, no small feat when there are no standards for this.

Additionally the service – sabre – provides other benefits.

• a backup catalogue should either main system become unavailable.
• A good mobile interface – which neither main catalogue currently offers
• options for different languages and browse options
• export options.

It’s also helped us become familiar with working with our MARC21 records, and the options available to us when we require a new catalogue interface. Setting up additional VuFind interfaces becomes fairly trivial.

Key points for effective practice

• User testing is incredibly useful and should be carried out for all new services
• exporting all MARC records can take time to perfect, ensuring all are being successfully exported and imported. A recent blog post links to some tips regarding this.
• VuFind has a good user/developer community, it is easy to adapt to meet local requirements and provides a lot of modern useful features that users like. Blacklight also looks like a very promising package.
• VuFind has an exciting development roadmap, for example with support for EAD and other metadata types.
• There are no standards for holdings/availability/location. While the international library community has made some efforts in this area, nothing has really come of it of late. This needs to change and an open standard, supported by all major suppliers is required. Meanwhile Libraries need to be prepared to allocate time and resources in to this functionality. Talis Alto had a relatively small user base, based mainly in the UK and Ireland and therefore lacks the potential community of VuFind users who use the LMS (at present ifind, developed at the University of Swansea and sabre, developed at the University of Sussex are the only two).
• Likewise, no standards for allowing one system (i.e. the catalogue) to renew/reserve items on another (LMS) exist. VuFind does provide this functionality but it is dependant on a working driver for the LMS.
• jangle provides a middleman between LMS systems and client applications (such as third party catalogues). It could avoid the need for every client system to provide a way to connect to every server, instead each just needs to be able to communicate with jangle (i.e. from Many-to-Many to One-to-Many). It would be good to see suppliers adopting this. Though jangle itself does require resources and time to setup.
• VuFind could be used for many niche and specialist requirements, or even as a backup catalogue. Perhaps providing a search interface for a subset of records (just AV, or a particular library). If development time is tight then more complex features, such as ‘My Account’ can be hidden. A simple bibliographic search is very quick and easy to setup.
• Be clear what can be offered, and what you plan to offer. If you are planning to provide a ‘quick win’ solution to a problem, be clear that not all of VuFind’s functionality will be available.

Conclusion and recommendations

• Many of the points above can be seen as recommendations
• User Testing can be fairly easy to perform and is incredibly useful
• VuFind is flexible and relatively easy to work with, it can be used as a replacement catalogue, or as here used to cater for a specialist need.
• The Library community urgently needs a standard – supported by vendors – to provide holdings/availability/location. Additionally a standard to support My Account / reservations and renewals.
• Library suppliers need to understand that providing open, unrestricted APIs and data export is part of the service/system Libraries pay for, not a threat to ‘bolt-on’ products. If you system does not work in our eco-system then it is a bug.

The sabre service can be found at: http://sabre.sussex.ac.uk

Regards

Chris Keene

Processing MARC records

To keep the records in sabre up to date, we need to copy over the MARC records from the two systems; pre-process them a little and then import them in to vufind. The pre-processing script was created by the University of Swansea and adds, amongst other things, a subfield to record which library(ies) it belongs to, and also deduplicates records for the same item between the two catalogues.

During the pre-processing stage we were having some problems, some of the MARC files were not being fully processed by the perl script, resulting in missing records, due to unusual characters in certain records.

This blog post (on another blog) outlines how we found a work around for this.

We’ve also uploaded a few scripts on to Google Code: http://code.google.com/p/credaul/source/browse/

VuFind Orientation

This post will try to be a brief guide to getting a feel for administrating a VuFind installation.

It’s not a definitive account, just my own experience so far. I’m still learning.

Installation

VuFind uses Apache Solr to index records and provide such things as facets. Some have described Vufind as a lite frontend to Solr, but I feel this is a little unfair, a car is more than an engine, and VuFind does more than just return search results.

VuFind is PHP based, making use of various PEAR libraries and uses MySQL for various settings. Solr is written in Java and will essentially run a web server on a different port which VuFind will use to query it. VuFind uses Smarty for templates.

The VuFind installer takes care of all of this, but at times you may find it needs a little tweaking (when I used it a few months a go it wasn’t downloading/installing Smarty correctly.

So in all, VuFind requires a based LAMP stack (Linux , Apache, MySQL, PHP), PEAR, Java (Tomcat), Solr, and Smarty. Other OS can be used with a little extra work.

I’ve also created an install script which works on Fedora based systems (I tested it on the Amazon AWS Linux installation which is based on Fedora). The difference with my script is that it will allow you to install multiple instances of VuFind on to the same server with ease. It uses an instance name which is used for the installation directory, url, database name etc, it also uses a different port number for the Solr server for the instance. Even if you don’t use the script I recommend you look at it to get a feel of what it is doing.

The installation notes for the CReDAUL project can be found here.

Useful urls

Vufind comes with an administration site. http://vufind.yourdomain.org/vufind/Admin/Home

This provides a way to view some statistics, some configuration (including ability to edit), and internal Record view. the htpasswd file is located in services/Admin directory, you can add a user to gain entry to the admin site. Also change the password on the default admin user, and consider restricting to local IP addresses.

You can also view the solr index on the port number it is running on e.g. http://vufind.yourdomain.org:17080/solr/biblio/

Layout

The top level of the vufind directory (by default /usr/local/vufind/ ) is of use for (a) starting/stopping vufind (b) importing records.

I should say here, when we talk about starting/stopping VuFind (using vufind.sh) we are really talking about starting/stopping the Solr service (Vufind wont do anything without it).

You will spend 90% of your time administrating VuFind within the ‘web’ directory, but before will move on to that we’ll mention the ‘solr’ directory.

solr – you’ll never guess – contains the solr installation. Including relevant logs and configuration. A useful file is solr/jetty/etc/jetty.xml which contains the configuration, including such things as port number and the number of threads to run. The actual index is stored in solr/biblio/index – if you’ve got a problem with your records, just delete the directory and re-import your MARC files.

Just before we move on to the ‘web’ directory I’ll mention the import directory, if contains the config for the import script which may need some tweaking.

The web directory.

These are the main sub-directories of the web directory:

• conf – the first file you will probably play with is the config.ini within this directory. It contains a lot of settings, from the core (what URL does it use), to quite specific settings and options. You’ll also find here the configuration file for the driver which is used to connected to your ILS (aka LMS), and a few more to configure facets (which ones are shown in which order) and searches.
• Drivers – the driver file which connects to your ILS is found here. Depending on what system you use you may find it needs tweaking to meet your needs.
• interface – you’ll find the smarty templates in interface/themes – more on this below
• lang – language files. Useful to define local terms, in your library do you refer to a Lending Desk, Issue desk, Service Point or Circulation desk? Vufind makes it easy to set this, for us in the English speaking world in the en.ini file, in fact this is much neater than many commercial products.
• RecordDrivers – two key files IndexRecord and MarcRecord.php. The latter is a child/extension of the former. IndexRecord contains methods for exposing record details (for use in smarty templates) for any sort of bib record. MarcRecord extends this with MARC specific information.
• services – key php application files. I haven’t found the need to edit anything in here yet, but can help to understand how system is structured.

Smarty

I quite like how Smarty is used, it keeps the interface quite separate from the rest of the logic and code. Though I have heard others say that it is essentially a templating system (with it’s own code and syntax) built on top of a language that in many ways was designed to be just that (PHP).

Vufind has a number of out-the-box themes, by default it uses the one called, well, default. And also mobile if it detects the user is on a mobile device.

I wont describe everything here, you can often guess which file is used for what on screen, it may take a couple of guesses but it is normally quite quick to see if this contains the html code you are interested in changing once you have opened the file.

At the top level there is layout.tpl, and footer.tpl which contain system wide layout templates. There’s also a css and a images file (there’s also a images dir higher up for images which are not theme specific).

• The homepage is in Web/home.tpl
• Record and Search dirs have contained most of the files I’ve needed to edit
• You can see many Smarty tags within the files to translate terms in to the local language and local terminology. for example {translate text=”Add to favorites”} which insert what text you have chosen for ‘adding to favorites’, e.g. ‘Add to My List’ etc.
• The Record files use variables created in the IndexRecord/MarcRecord files (see above ‘RecordDrivers’). So you can refer to those files to see which variables will contain the information you want to show. Different information is exposed to different views, e.g. the export citation option has just the information required to create a citation. If you want to add an extra bit of information to a citation then you will need to probably edit the RecordDriver files to make sure they expose the data, and then edit the appropriate smarty template file to incorporate it.

Some examples of changes

• For internet resources we use the MARC 856 field, subfield $u contains the actual link and subfield $y contains the text to use for the link. However vufind presumes the link text will be in subfield $z (which we don’t use) so out the box it would use the URL itself as the text for the link as we had no z. To fix this we edited RecordDrivers/MarcRecord.php so that the getURLs function uses the y subfield.
• Our setup actually searches two libraries, those of the University of Sussex and University of Brighton. We pre-process the MARC records from each University to add a 969 field for each University which has the item in question. The 969 contains the University code (susx/brighton), and the bibid the library uses for the item (if both Libraries have the item then the record has two 969 fields). When we display a record we want the user to be able to easy link to the record in the main catalogue of the University in question. To do this we edited MarcRecord to create a function which can take this MARC field and turn it in to a link (using the bibid to create a link to the catalogue record in question). I think we also created a skelton function in IndexRecord (which the function in MarcRecord over rides). Finally we edited interface/themes/default/RecordDrivers/Index/core.tpl to add the field to the default record display. [to see the MARC tags referred to here and the example above see this record ]
• We didn’t want the search results to show availability/shelfmark. With two library systems there was too much potential for mistakes to creep in, best left to the record view of the item. Plus it meant VuFind had to get information for every item on the search results page, potentially putting a load on the original ILS (LMS) server(s). This is actually something which uses AJAX. And this is something which uses the design where a MARC record extends what a ‘default’ record can do (i.e MarcRecord.php extends IndexRecord.php in RecordDrivers). VuFind decides that if something is a MARC record it has come from a Library system and so it can try and find holdings/availability. Essentially the IndexRecord function getSearchResult() will set $interface->assign(‘summAjaxStatus’, false); so it wont try and get availability via AJAX. Where as MarcRecord overrides the function, doing all but the same with everything else but sets that value to true. The smarty template for search results (interface/themes/default/RecordDrivers/Index/result.tpl) then uses the value of summAjaxStatus in a if statement, if true it will output some html/javascript code to perform thee AJAX calcode to call and display the results. So, to do what we wanted, we changed MarcRecord to set summAjaxStatus to false (so it should always be false). And then edited the smarty file so that what gets shown when it is false meets our needs.
• We edited the help files in interface/themes/default/Help/
• we changed the order of facets in conf/facets.ini
• We edited interface/themes/default/Record/view-holdings.tpl so that it doesn’t show ‘item 1′, ‘item 2′ etc next to each copy of the item (we found the numbers weren’t really helpful and didn’t mean anything).
• And much more….

These examples show that it is quite easy to customise vufind, once you know which files to poke around in.

I can say that the only other system which compares in this flexibility, in my experience, is Eprints. I have not come across a single commercial system (catalogue or otherwise) which we use as a library which has such a structured and logical design.

We have a name

We generally avoid having special names for services. Our Catalogue is ‘The Catalogue’, and so on. We avoid specialist terms (OPAC, a term I’ve always felt was never required in the first place!), or product names (Metalib).

However, when it comes to the service being created in the CReDAUL project this didn’t work. It wasn’t replacing an existing service but creating something new: a catalogue of records from both the Universities of Sussex and Brighton Libraries. Frankly we couldn’t go around referring to the ‘Joint University of Sussex and University of Brighton Libraries Catalogue thing‘ – we needed a name.

So we held a competition amongst staff to come up with a name. We had a lot of entries, nearly a full page of A4, we have very creative staff!

In the end we decided on…..

sabre : Sussex and Brighton Resource Explorer

In my experience the words that make up acronyms tend to be lost over the course of time (COPAC, UKOLN) the name becomes just that, a name. And it’s the name we will push, a long with a brief sentence as to explain what it is (currently: Searching across Sussex and Brighton University Libraries).

Personally, I really like the name, it is easy to pronounce and is short and simple. We also asked a member of staff to come up with a logo, and an hour later he come up with three. There were all amazing!

So the homepage to the new service – sabre – currently looks like this:

sabre homepage

We’ll be using the new name and logo in publicity for the new service. I couldn’t be happier with the end result.

User Experience Testing

We’ve just done our User Experience Testing. If you struggle to read more than 140 characters: It was great. Learnt lots and given us confidence to make changes.

The Vertical Slide Offices (the room where the testing took place was similar)

As part of the project we had enlisted the help of  Dr Graham McAllister in the University’s School of Informatics, who not only specialises in User Experience, but also has a spin-out company – called Vertical Slice – based at the University’s Innovation Centre which carries out User Experience Testing mainly for Video Games.

The setup

The User Experience Testing consisted of a series of one on one sessions, each lasting around 40 minutes. Graham asked a number of questions (‘how would you find x’), but also more general questions: what do you think of the site? can you tell what it does? would you use it? He sat next to the user as they carried out the exercises. He’s approachable manner, and the fact he was not connected to the new service, helped in letting the users speak freely as to what they thought.

A number of students and staff from BSMS, the joint medical school (with the University of Brighton), who will hopefully be key users to the new service, agreed to come along. It was carried out throughout the course of one day (Wednesday the 11th May).

On the day

I was sitting in the Vertical Slice offices near to where the testing was taking place, and was able to watch the Video recordings of the screen/video of the user/audio throughout the day.

It would be unfair to say I was cynical of what value it would bring, that would be too strong. But I wasn’t fully sure how useful it could be. They would search for books etc, they would probably find them, some things we know would be harder than others.

Within two minutes of watching the first recording any doubts had been blown away. You could see the frown on their face as something wasn’t obvious and the eyes widen when they spotted what they were after. The mouse would dance between two parts of the screen as they quickly thought through whether to perform a search or try and browse.

An example: we had added a link back to the originating library’s catalogue (or both catalogues if they both had a copy). This, I thought, would be useful for doing anything you could not do on the Vufind interface itself. Yet each time someone was asked how they would reserve a copy, or some other similar task, a frown would appear, then a gap, then a ‘no I don’t know how I would do that’. After the first one I wanted to dive in and start making changes to improve it.

Another example was the filter restrictions that could be applied. For us, a key one is to restrict by institution. However once a filter is applied it continues to be applied to future searches. It does say on the right the filter is applied, and under the search box there was a ticked option to ‘Retain my current filters’ but these things do not matter. What matters is that time and again they were not getting back the results they expected which lead to confusion as it was not clear the filter from before was still being applied. We need to make the active filter much clearer and try and make it impossible to miss, using colour and screen location.

Findings

• On the Homepage: It’s not clear what the service does when you first look at the homepage (though on the whole comments were positive to the clean and modern look).
• The browse options on the homepage (shelfmark, language, item type) – while potential useful at times – distracted more than the use they provided.
  Further more, while these seemed like a good idea when I first encounted them – from a technical perspective making good use of the MARC metadata – in fact they can be very misleading as users quite rightly think that when you click on ‘Medicine’ or ‘ebooks’ you will see all items for that selection. In reality due to the lack of detail in older records and the different ways the two Universities have catalogued their collections these browse options rarely displayed all items of the selected choice.
  Another reason, users would click on Medicine and be presented with the first few items of the very large list of items classified in the Library of Congress Medicine (R) range. However these first few items were not particularly good examples, or the type of books you would expect to show up, which confused people.
• The is perhaps part of a more general trend throughout; simplify where possible.  Users actually said ‘all this stuff is getting in the way‘, ‘why am I being shown this information?‘, ‘There might be an option to do X but I can’t see it as there’s so much stuff‘. This was a clear directive: hide whatever is not needed.
• On the search results: need to make it clear what filters are currently applied, probably needs to be at the top, just above the results and hard to miss. Again, users were not always sure if a filter had been applied (‘did that do anything?).
• Sort Date Ascending/Descending : while not hard for the average person to work out what these mean, they do make most people stop and think to make sure they have got it the right way around. This is bad. So use wording that doesn’t require people to think it through (‘oldest first’).
• On Record pages: Make it really clear which Libraries have this item.
• Clear ‘action’ options: Reserve this, etc.
• Use Shelfmark, not Call number. Make sure American English is changed to British English, Catalog to Catalogue.
• Remove Tags/Comments, unlikely to be used and, as noted above, hide the more important options and information.
• While getting Holdings information from the two underlying systems is somewhat of an art, need to continue to refine how this information is displayed.
• Generally: need more information as to what it does, and a dedicated feedback page, which has clear information as to how to contact the two Libraries regarding items and borrowing, and contact us (well, me) regarding comments and issues with the system.

A full list of changes we are going to make can be found here.

What next

Over the next couple of weeks we will be making changes to the user interface based on this feedback. We are using Git to keep track of changes to the system (code/config/language&wording/template) and a Google Spreadsheet to keep track of what has been changed, and what we still need to do.

jisclms

This project is partly being funded by the JISC as part of their Library Management Systems Programme (referred to as jisclms). The JISC provide services and fund projects related to developing University IT, Libraries, web services, e-learning and research.

The various projects (all based at UK Universities) which are being funded by jisclms recently met up in Glasgow for a two day meeting. This was a incredibly productive event, to discover what others are working on,  share experiences and , and discuss areas and issues which we currently face with library management systems. The latter included issues around OSS LMS (what are the issues, and how could they see the introduction in UK academic libraries), Shared Electronic Resource Management and Usability/User Experience.  What was clear was that the benefit of the projects as whole was greater than the sum of each project combined. Each project requires that University to reuse, open up, or export data in a new way. It forces openness and the net effect is that there will be multiple people around the UK who have recent experience in using LMS data and using it with new applications. This helps create a community who and share and promote such innovation, and help others who are interested in trying such things. It helps to set a ball rolling, or rather, gives a ball which was starting to move slowly a very big push.

• JISC page on the jisclms programme: http://www.jisc.ac.uk/whatwedo/programmes/inf11/jisclms.aspx
• Jisclms resource page http://code.google.com/p/jisclms/
• Image showing the different projects and the systems they work with. The key here is that libraries have tended to work with the community around their LMS, creating five or so somewhat isolated communities, but as you can see in this diagram, this hopefully helps to break down the barriers. We have have been part of the Talis (our LMS) community, but our now working closly with Swansea, and sharing notes with Kent (neither use Talis ) as we setup our VuFind installation. http://www.flickr.com/photos/creatinginthedark/4992287137/
• A blog providing an overview of developments: http://lmsmash.blogspot.com/
• A Google Reader bundle of all blogs (that I am aware of) related to jisclms projects http://www.google.com/reader/bundle/user%2F11598365848111438788%2Fbundle%2Fjisclms
• Archive of tweets using the #jisclms hastag http://twapperkeeper.com/hashtag/jisclms

Elevator Pitch

• ‘The name “elevator pitch” reflects the idea that it should be possible to deliver an elevator pitch in the time span of an elevator ride, or approximately thirty seconds to two minutes.‘ (wikipedia)
• Militant British-English speakers may prefer “lift pitch”
• Those who find a ‘pitch’ a little crass may prefer ‘concise-explanation-of-product-or-idea-while-in-lift’. Frankly, it looses some bite.
• Here’s ours…

The University of Sussex and The University of Brighton have various partnerships and joint initiatives, helped by their close proximity. The most prominent of these is the joint medical school: BSMS (Brighton and Sussex Medical School).

The CReDAUL project is creating a combined web catalogue of the University of Brighton and the University of Sussex Libraries. It will contain all the items of both libraries in one interface, so users can see at a glance if either library has a book or other item, and if so at which library and branch.

What’s in it for Sussex? Researchers and students can easily search University libraries to see if a book is available locally. Library staff trying to locate an item for a user will find it useful, e.g. to check for Interlibrary loan requests, increasing efficiency and service. Users have another interface to search the Sussex catalogue should our main catalogue not be available, you can restrict a search to just one Library. It helps us to work together with Brighton in developing something, and the end product may help future collaborations. The interface will are developing (called VuFind, a free off the shelve application used by other Universities) has good support for mobile devices and integrating with applications such as Zotero.

What’s in it for BSMS? At the moment they have to search two different catalogues as their books and resources are split between the two Universities. They will now just need to do one search and can quickly see which Library it is at and if it is available. This will improve the experience for the medical students and staff, who often have a very high work load.

What’s in it for the local community? members of the public may have reason to refer to an item in a local academic library. If they wish to access an academic book, and would like to check to see if it is in a local academic library, they can now do so using just one search from one interface.

What’s in it for other Universities? We are writing up our experience on this blog, and finding other ways to share what we have learnt. Using Open Source discovery layer applications is something of a hot topic at the moment. We have looked at two: Vufind and Blacklight, and are writing up our findings. We are also writing up our experience with VuFind and how we installed it. We’ll also write up how we found setting up a union catalogue of two libraries records and holdings.

PS some background to this post: http://lmsmash.blogspot.com/2010/09/conversations-for-very-short-lift.html

Step-by-step implementation of VuFind

This blog (along with the blogs linked from here) is a step-by-step guide to the implementation of VuFind starting with installation, on our VuFind server.

(TODO: Eventually move contents under level 1 headings to separate posts?)

Installation

The basics

We start with a minimal Debian 5.05 installation (from its 40MB business card image) on a VMWare virtual machine on a VMWare ESXi server. The virtual machine has 2 processors available to it (2.4GHz AMD Quad-Core Opteron), 4GB RAM and 75GB hard drive space on SAN. Having logged in as root, the best place to start is:

#> apt-get update
#> apt-get upgrade
#> apt-get dist-upgrade

Since we are dealing with a very minimal installation of Debian, we need to get a number of basic tools installed, such as the following:

#> apt-get install sudo
#> apt-get install less
#> apt-get install wget
#> apt-get install screen
#> apt-get install subversion

Although we have installed sudo and we have at least one user with sudo rights, we will carry on with the installation as root.

Getting the pre-requisites

Now we move on to installing VuFind pre-requisites. Most of what we do are according to the detailed instructions for VuFind installation. First, we install Apache, MySQL, PHP:

#> apt-get install apache2 mysql-server php5 php5-dev php-pear php5-ldap php5-mysql php5-xsl php5-pspell php5-sybase
#> pear upgrade pear

The first line in the above process will ask you to configure MySQL server root password. After this, we enable the URL rewriting module for Apache:

#> a2enmod rewrite
#> /etc/init.d/apache2 force-reload

We install Java in a slightly different way than instructed on VuFind’s website. We would have gone for the Sun/Oracle JDK, but since the Oracle take-over of Sun, it seems to be more complicated to download it from Oracle website than it was to do from Sun’s website, so we use OpenJDK from the java.net website https://jdk6.dev.java.net/

At the time of this writing, we download and install the JDK6 update 23.

#> mkdir -p /opt/java
#> cd /opt/java
#> wget http://www.java.net/download/jdk6/6u23/promoted/b01/binaries/jdk-6u23-ea-bin-b01-linux-i586-30_aug_2010.bin
#> chmod +x jdk-6u23-ea-bin-b01-linux-i586-30_aug_2010.bin
#> ./jdk-6u23-ea-bin-b01-linux-i586-30_aug_2010.bin

This will create a directory called /opt/java/jdk1.6.0_23. Now, we need to put it in PATH and also create symbolic links so that we can upgrade JDK later, quite easily by switching symbolic links. Note that JDK can also be installed using the alternatives framework (see /etc/alternatives, man update-alternatives). In the /opt/java directory:

#> ln -s jdk1.6.0_23 current

Then, we edit the /etc/profile file by adding the following lines before export PATH.

PATH="$PATH:/opt/java/current/bin"
JAVA_HOME="/opt/java/current
export JAVA_HOME

Then we reload /etc/profile and we will be able to access the JDK as follows:

#> source /etc/profile
#> echo $JAVA_HOME
/opt/java/current
#> echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/java/current/bin
#> java -version
java version "1.6.0_23-ea"
Java(TM) SE Runtime Environment (build 1.6.0_23-ea-b01)
OpenJDK Server VM (build 19.0-b06, mixed mode)

Obtaining and configuring VuFind for first run

We do not install the standard VuFind. We will use the one that contains multi-ILS capabilities, maintained by Luke O’Sullivan (Swansea). We check-out from SVN, the appropriate branch.

#> svn co https://vufind.svn.sourceforge.net/svnroot/vufind/branches/master /usr/local/vufind

Now, we have a VuFind installation at /usr/local/vufind. We shall now configure it. First, we have to initialise the installation by running /usr/local/vufind/install. This will setup MySQL database amongst other default configurations. NOTE that at the time of writing, there was a bug in installation script, as filed in the bug report http://vufind.org/jira/browse/VUFIND-315. The fix will be available in version 1.1 or revision r2976 of https://vufind.svn.sourceforge.net/svnroot/vufind/branches/master. Updates to versions can be obtained by running svn update in /usr/local/vufind. We also need to make the following scripts executable in /usr/local/vufind.

#> chmod +x vufind.sh
#> chmod +x import-marc.sh

The following two lines need to be added to /etc/profile before export JAVA_HOME and the file needs to be re-sourced.

VUFIND_HOME="/usr/local/vufind"

export VUFIND_HOME

Now, we need to tell Apache where VuFind lives. Also, we need to let Apache have write access to some directories. We can do that as follows.

#> ln -s /usr/local/vufind/httpd-vufind.conf /etc/apache2/conf.d/vufind
#> /etc/init.d/apache2 reload
#> chown www-data:www-data /usr/local/vufind/web/interface/compile
#> chown www-data:www-data /usr/local/vufind/web/interface/cache
#> chown www-data:www-data /usr/local/vufind/web/images/covers/*

The final step is to edit the /usr/local/vufind/web/conf/config.ini to match the requirements of the institution (e.g. support email, path to VuFind, URL on the web server, etc.) and then run /usr/local/vufind start. Running it at this stage will generate errors as multi-ILS configurations have not been done properly, which is what we will work on later, after we have gone through some other installations.

Installing the Talis-Jangle connector

In our specific installation of VuFind, we plan to have multi-ILS connections to Talis systems. For this reason, we must install a number of other things to make VuFind work with Talis. One of the first things to install is the Talis-Jangle connector http://code.google.com/p/talislms-jangle/. The instructions provided below are derived from http://code.google.com/p/talislms-jangle/wiki/InstallRailsConnector.

Install JRuby

We install JRuby in the way we installed Java. Download the latest distribution of JRuby (1.5.2 at the time of this writing) from http://jruby.org/download into a new directory /opt/jruby. Untar it and create a symbolic link such that /opt/jruby/current points to the untarred JRuby installation. Now, change the PATH line in /etc/profile right above the declaration for JAVA_HOME to PATH="$PATH:/opt/java/current/bin:/opt/jruby/current/bin" and run:
#> source /etc/profile
After this, running the following will show the version of JRuby you have installed, for example:

#> jruby -v
jruby 1.5.2 (ruby 1.8.7 patchlevel 249) (2010-08-20 1c5e29d) (OpenJDK Client VM 1.6.0_23-ea) [i386-java]


Update RubyGems and install various Gems

To update RubyGems, we run

#> jruby -S gem update --system

After that, we install the following:

#> jruby -S gem install rails -v 2.3.8
#> jruby -S gem install rsolr-direct
#> jruby -S gem install activerecord-jdbc-adapter -v 0.9.4
#> jruby -S gem install marc jrexml vpim composite_primary_keys cql-ruby glassfish

Having installed the various Gems, we now install the Sybase JDBC driver from http://jtds.sourceforge.net/, unzip and place the JTDS driver JAR file in /opt/jruby/current/lib.

Obtaining the Talis-Jangle connector

We checkout from SVN (http://talislms-jangle.googlecode.com/svn/trunk/) the Talis-Jangle connector into /opt/talislms-connector. The file /opt/talislms-connector/config/database.yml-dist needs to be renamed to /opt/talislms-connector/config/database.yml and edited to reflect the connection to Talis Sybase. The file /opt/talislms-connector/config/connector.yml may be edited to change the administrator password. Before we start Glassfish, we have to create a symbolic link to the Solr directory as follows:

#> cd /opt/talislms-connector/lib
#> ln -s apache-solr-1.4.0 solr

This helps us keep link to Solr and update it later in case a different version of Solr is used. However, a more up-to-date version of the Talis connector may not need this as it may automatically point to the apache-solr-1.4.0 directory. To start Glassfish in production mode:

#> cd /opt/talislms-connector
#> jruby -J-Druby.runtime=8 -S glassfish -e production &

This step starts up Glassfish, which takes quite a long time to cache the Sybase database. The log can be found at /opt/talislms-connector/log/production.log

Obtaining the Jangle feed generator

Having configured the Talis Jangle connector, we now need to configure the feed generator. We can download it by checking out the SVN repository http://jangle.googlecode.com/svn/trunk/PHPCore as follows:

#> cd /usr/local
#> svn co http://jangle.googlecode.com/svn/trunk/PHPCore jangle-feed

We must tell Apache about jangle-feed. To do so, create a file called jangle-feed-httpd.conf inside the jangle-feed directory with the following contents.

Alias /janglefeed /usr/local/jangle-feed
<Directory /usr/local/jangle-feed/>
Order allow,deny
Allow from 127
AllowOverride All
Options -Indexes
</Directory>

Then create a symbolic link and reload Apache:

#> ln -s /usr/local/jangle-feed/jangle-feed-httpd.conf /etc/apache2/conf.d/janglefeed
#> /etc/init.d/apache2 reload

We must also configure the Jangle feed generator by ensuring that the config file /usr/local/jangle-feed/config/config.yml looks as follows.

server:
   base_url: http://localhost/janglefeed/
services:
   talislms:
     connector_url: http://localhost:3000/connector/
     description: Alto

cache_location:
   directory: "./cache"
   function: functions
   page: pages

Note that there is nothing available at http://localhost:3000/connector/, but the config file uses that as a base URL. We must also install PHP Caches:

#> pear install Cache-1.5.5

and /usr/local/jangle-feed/.htaccess must be edited to contain the following.

RewriteEngine On
RewriteBase /janglefeed
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php [QSA,L]

Thus, the Jangle feed generator will now be available at http://localhost/janglefeed/talislms. So, for example http://localhost/janglefeed/talislms/items shows the Jangle feed.

This concludes the installation process for VuFind and the associated software. Before we move on to configuring VuFind, let us setup some administrative scripts.

Auto-starting VuFind

VuFind provides a startup script, which can be used by Linux to auto-start VuFind. However, we also need the Talis Jangle connector to be started up. Therefore, we write our own startup script as follows, and save it to /etc/init.d/vufind-all:

#!/bin/bash
#/etc/init.d/vufind-all

### BEGIN INIT INFO
# Provides:    vufind-all
# Required-Start:   $local_fs $network
# Required-Stop:   $local_fs $network
# Default-Start:   2 3 4 5
# Default-Stop:    0 1 6
# Short-Description: Start VuFind daemon at boot time
# Description:  Enable VuFind and associated services provided by daemon.
### END INIT INFO

#See details about LSB information at http://wiki.debian.org/LSBInitScripts

source /etc/profile

TALISLMS_CONNECTOR_PATH=/opt/talislms-connector
VUFIND_PATH=/usr/local/vufind

touch /var/lock/vufind-all

case "$1" in
  start)
    echo "Starting VuFind and other associated services..."
    cd $TALISLMS_CONNECTOR_PATH && jruby -J-Druby.runtime=8 -S glassfish -e production &
    cd $VUFIND_PATH && ./vufind.sh start
     ;;
  stop)
    echo "Stopping VuFind and other associated services..."
    GLASSFISH_PID=`ps aux | grep glassfish | grep jruby | awk '{print $2}'`
    cd $VUFIND_PATH && ./vufind.sh stop
    kill $GLASSFISH_PID
     ;;
  *)
    echo "Usage: /etc/init.d/vufind-all {start | stop}"
    exit 1;
     ;;
esac

exit 0;


To add this to Debian startup, run update-rc.d vufind-all defaults.

Having VuFind on the web server root

Sometimes it may be necessary to have VuFind on the root of the web server (e.g. http://hostname/ instead of http://hostname/vufind). We show the steps to configure this on our Apache2/Debian setup.

First, remove the symbolic link /etc/apache2/conf.d/vufind that was created earlier in this installation process. Following that, run #> a2dissite default, which will disable the default site on the root of the Apache2 web server. Now, create a file /etc/apache2/sites-available/vufind with the following contents.

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  ServerName full-host-name-of-webserver
  ServerAlias full-cname-of-webserver

  DocumentRoot /usr/local/vufind/web

  ErrorLog /var/log/apache2/vufind-error.log
  LogLevel warn
  CustomLog /var/log/apache2/vufind-access.log combined
</VirtualHost *:80>

Between the DocumentRoot and ErrorLog declarations, copy and paste the entire <Directory> section from /usr/local/vufind/httpd-vufind.conf and change the RewriteBase line to RewriteBase /. Do not copy the Alias line. Note that we have not copied the Location line, so this will disable the web-based admin interface for VuFind, which is not a bad thing to do.

In /usr/local/vufind/web/conf/config.ini, make sure the Site section is edited to contain the following:

[Site]
path = http://full-host-name-of-webserver
url = http://full-host-name-of-webserver


You can use a host name corresponding to DNS A or CNAME records for your web server. Having done this, enable the new site and reload the web server.

#> a2ensite vufind
#> /etc/init.d/apache2 force-reload


In our view, this is not the best way of configuring VuFind at the root of the web server, but it works for now.

Configuring VuFind

One of the first things to do regarding the multi-ILS integration is to edit the DriverIDs section of /usr/local/vufind/web/conf/config.ini file, for example adding two library IDs:

[DriverIDs]
lib1 = "Library 1"
lib2 = "Library 2"


This will imply that there will be directories corresponding to the driver IDs in the directory /usr/local/vufind/web/conf/systems/. Each such directory (e.g. /usr/local/vufind/web/conf/systems/lib1/) will contain a config.ini which will contain configuration information specific to that particular library only.

Using multi-ILS configuration will mean that the imported MARC records from various libraries are merged. This calls for creating new MARC fields that can be used as identifiers necessary for the multi-library search. We should edit /usr/local/vufind/import/marc_local.properties to uncomment the two lines as shown:

# Uncomment the following settings to insert appropriate values for your site:
#collection = "Catalog"
institution = 969a
systemids = 969b
#building = "Library A"


Importing data

[This is draft so far... it is being edited]
1. Use Paul Johnson’s (Swansea) MARC merge script to create the annotated MARC records and import them as usual into VuFind.
2. Requires installation of additional Perl modules, using CPAN. It is necessary to install make.
3. Requires JangleXXX.ini in /usr/local/vufind/web/conf/ directory which is a copy of Jangle.ini from the same directory with XXX specific information, where XXX is the library code being used, e.g. lib1. This file needs to be referenced as config = JangleXXX (without the .ini extension) from /usr/local/vufind/web/conf/systems/XXX/config.ini — rather long-winded configuration, in my opinion!

Connecting VuFind to Talis for holdings information

Integration with Talis

After having VuFind 1.0 installed at $VUFIND_HOME=/usr/local/vufind, I started to explore how to connect VuFind to our existing Talis system in order to obtain holdings information. Unlike support for many other library systems, VuFind does not come with a ready-made driver for Talis in $VUFIND_HOME/web/Drivers.

Multi-ILS Jangle driver

Swansea University implements VuFind for multi-ILS integration (including Talis), so I got in touch with Luke O’Sullivan there to learn about their experience. It turns out that he has written his own driver to connect to Talis, but the process is not straightforward. The driver (called Jangle driver) is actually a generic driver enabling multi-ILS integration, with some changes to VuFind itself. One of the ILS that it can connect to is Talis but not directly. It talks to a Jangle connector for Talis (and similarly Jangle connector for OpenBiblio, for example) through the Jangle feed generator (http://jangle.googlecode.com/svn/trunk/PHPCore). The Talis Jangle connector has been developed by Ross Singer at Talis and maintained by him at http://code.google.com/p/talislms-jangle/.

Talis-Jangle connector and the Jangle feed generator

I started off with getting the Talis Jangle connector to work. With assistance from Ross, the Jangle connector was setup at /opt/local/talislms-connector/ after having followed instructions from http://code.google.com/p/talislms-jangle/wiki/InstallRailsConnector. With a bit of adjustments, I got it to work finally. It connected successfully to our Talis server. The connector could either be run using the built-in WEBrick server or Glassfish. I ran it using Glassfish and monitored the log at /opt/local/talislms-connector/log/development.log.

In the next step, the Jangle feed generator was setup at /usr/local/jangleFeed/ and configured to connect to the Talis-Jangle connector. A symbolic link had to be created from /etc/apache2/conf.d/ to point to /usr/local/jangleFeed/jangleFeed-httpd.conf.

VuFind Jangle driver

Having setup the connector successfully, I went on to obtaining the Jangle driver from Luke. It was setup and configured to connect to the Jangle feed. After a bit of troubleshooting of a problem arising from a missing PHP library (i.e. Cache.php), we got it to work. At this point, however, there was only one ILS (i.e. ours) that we connected to. The point of this experiment was to connect VuFind to Talis using the Jangle driver.

Slowness fixed, somewhat

However, obtaining the holdings information was dreadfully slow. It turned out that for every result in the result-set of a query on VuFind, the Talis-Jangle connector fired several SQL requests to our Talis/Sybase server. After presenting Ross with the log file from the Talis-Jangle connector, he tweaked it. Now, it is faster although further improvements in speed would be better.

As of August 10, the slowness problem has reduced with at least a single ILS connectivity: running Glassfish in production mode resulted in drastic performance improvement from running it in development mode.

Now, multi-ILS option

The structure of VuFind does not directly allow for connections to multi-ILS. Also, with a multi-library environment, it is likely that we may be interested in connecting to multiple LDAP servers across more than one institutional networks. VuFind does not have a built-in facility for that. However, Luke has extended VuFind to support this. Therefore, besides writing the Jangle driver, Luke has modified the base VuFind installation to support multi-institutional connections. Luke is going to merge his branch of VuFind development with the main development trunk of VuFind (?).

To accommodate testing the multi-ILS features, I installed more than one instances of VuFind according to the instructions at http://vufind.org/wiki/installing_multiple_instances. One of the instances is what we already had running at /usr/local/vufind but now moved to /usr/local/vufind-susx and the other one at /usr/local/vufind-smu which is Luke’s version. Symbolic links from Apache config had to be updated accordingly. From this point forward, I shall call the former as VUFIND-SUSX and the latter as VUFIND-SMU. This required multiple MySQL databases. To avoid getting into trouble of moving MySQL databases, the database for VUFIND-SUSX was kept unchanged while a new database was created for VUFIND-SMU.

One of the things to keep in mind about multi-ILS connection is the way MARC records are imported into VuFind. Since they will come from multiple sources, they need to be merged. Otherwise, we shall not have multiple library locations displayed for any single record that exists in more than one library. I am not a MARC expert but what I understand is that certain fields in MARC (e.g. 001) may be used by the VuFind Jangle driver to query the Talis-Jangle connector for the correct record in the Talis system. Such a field has to be unique. However, it may not be possible to have two different 001 values for the same entry in two different libraries, so Luke works with MARC files which are pre-processed to add in field 969 (multiple-values allowed). The 969 field contains references with ILS prefixes and unique IDs. Therefore, a record existing in the Sussex library could have SUSX-123456 whereas it may also have BRI-123456 which indicates that it exists in the Brighton library (e.g. prefixed as BRI). Luke’s driver can then use these 969 values to initiate connections to two different ILS based on the ILS prefix.

In order to get it work…

To begin with, the file VUFIND-SMU/web/conf/config.ini is different from VUFIND-SUSX/web/conf/config.ini. The former has a specification for Controller and DriverIDs, which is how it keeps track of multiple drivers for multiple ILS connections. The directory VUFIND-SMU/web/conf/systems contains sub-directories with names corresponding to DriverID entries. For example, if we have:

[Controller]
type = default

[DriverIDs]
susx = "University of Sussex"


then we will also have VUFIND-SMU/web/conf/systems/susx/. Such a directory contains a config.ini, which provides configuration instruction for the individual driver. For example, we may have:

[driverID]
name = University of Sussex
driver = Jangle
config = Jangle
prefix = susx

Thus, with this configuration, we are using the Jangle driver to connect using ILS prefix “susx”. The above driverID specification in VUFIND-SMU/web/conf/systems/susx/config.ini expects a driver configuration file by the same name in VUFIND-SMU/web/conf/ which is configured to allow the driver to connect to the Jangle feed generator.

Connecting to the multi-ILS data

Having tested the example setup that emulates the multi-ILS connectivity, we are now in the process of obtaining MARC records from Brighton university that will have to be merged with those from the University of Sussex. The merger process will generate, in the MARC records, the 969 fields with different ILS prefixes based on which VUFIND-SMU will connect to the Talis systems in both the universities.

Acknowledgments

Thanks to Luke O’Sullivan (Swansea University) and Ross Singer (Talis) for their assistance with implementing VuFind.

The work plan

Since we have working versions of both VuFind and Blacklight installed, the plan is to investigate the following. As we go on, I will add further details on each of these topics in separate blog posts.

Holdings and multi-library search

The aim is to find out how we should implement holdings information, e.g. classmark (LCCN), on-loan, number of copies, etc. One of the primary objectives of our OPAC installation will be to enable multi-library search between the University of Sussex and the University of Brighton. A cursory glance at VuFind’s ILS drivers suggests that it is possible. Some universities (e.g. Swansea) have done it. The OPAC installation should integrate with our existing Talis system.

Read our experience with implementation of multi-ILS holdings information queries.

To-do: Multi-library search facility, which means having identifiers in MARC records that distinguish which library they come from. Such an identifier could be field 969 as explained in the multi-ILS holdings information queries.

System customisation

Having installed both systems, we will now want to look into how easy it is to:

1. Customise the branding and layout.
2. Change how bibliographic records and search results are displayed.
3. Customise search refinements and facets.
4. Provide bibliographic record enhancements, e.g. displaying book jacket covers, Amazon comments, etc.
5. Provide extended export formats, e.g. BibTeX.

Apache Solr and data storage

Since both VuFind and Blacklight use Apache Solr, it is worth exploring if both of them can use the same Solr index. If this is possible, it will be easier to have both systems on trial and then phase in the one users prefer. Initial investigation shows that linking to the same Solr index may not be possible, but this continues to be a subject of investigation.

User authentication

We would like to find out how both VuFind and Blacklight works with some form of user authentication, e.g. LDAP and any other SSO and that too across multiple libraries.

Automation

It would be useful to setup automated scripts that can be run as cron jobs to implement daily/hourly updates of records imported as MARC data. It is worth looking into how one can delete certain records, preferably in a batch. Slightly side-tracking, it is important to see if the Talis-exported MARC records are actually usable by either of the OPAC systems. The fact that some MARC records were imported with errors could point to some incompatibility issues.