Step-by-step implementation of VuFind

This blog (along with the blogs linked from here) is a step-by-step guide to the implementation of VuFind starting with installation, on our VuFind server.

(TODO: Eventually move contents under level 1 headings to separate posts?)

Installation

The basics

We start with a minimal Debian 5.05 installation (from its 40MB business card image) on a VMWare virtual machine on a VMWare ESXi server. The virtual machine has 2 processors available to it (2.4GHz AMD Quad-Core Opteron), 4GB RAM and 75GB hard drive space on SAN. Having logged in as root, the best place to start is:

#> apt-get update
#> apt-get upgrade
#> apt-get dist-upgrade

Since we are dealing with a very minimal installation of Debian, we need to get a number of basic tools installed, such as the following:

#> apt-get install sudo
#> apt-get install less
#> apt-get install wget
#> apt-get install screen
#> apt-get install subversion

Although we have installed sudo and we have at least one user with sudo rights, we will carry on with the installation as root.

Getting the pre-requisites

Now we move on to installing VuFind pre-requisites. Most of what we do are according to the detailed instructions for VuFind installation. First, we install Apache, MySQL, PHP:

#> apt-get install apache2 mysql-server php5 php5-dev php-pear php5-ldap php5-mysql php5-xsl php5-pspell php5-sybase
#> pear upgrade pear

The first line in the above process will ask you to configure MySQL server root password. After this, we enable the URL rewriting module for Apache:

#> a2enmod rewrite
#> /etc/init.d/apache2 force-reload

We install Java in a slightly different way than instructed on VuFind’s website. We would have gone for the Sun/Oracle JDK, but since the Oracle take-over of Sun, it seems to be more complicated to download it from Oracle website than it was to do from Sun’s website, so we use OpenJDK from the java.net website https://jdk6.dev.java.net/

At the time of this writing, we download and install the JDK6 update 23.

#> mkdir -p /opt/java
#> cd /opt/java
#> wget http://www.java.net/download/jdk6/6u23/promoted/b01/binaries/jdk-6u23-ea-bin-b01-linux-i586-30_aug_2010.bin
#> chmod +x jdk-6u23-ea-bin-b01-linux-i586-30_aug_2010.bin
#> ./jdk-6u23-ea-bin-b01-linux-i586-30_aug_2010.bin

This will create a directory called /opt/java/jdk1.6.0_23. Now, we need to put it in PATH and also create symbolic links so that we can upgrade JDK later, quite easily by switching symbolic links. Note that JDK can also be installed using the alternatives framework (see /etc/alternatives, man update-alternatives). In the /opt/java directory:

#> ln -s jdk1.6.0_23 current

Then, we edit the /etc/profile file by adding the following lines before export PATH.

PATH="$PATH:/opt/java/current/bin"
JAVA_HOME="/opt/java/current
export JAVA_HOME

Then we reload /etc/profile and we will be able to access the JDK as follows:

#> source /etc/profile
#> echo $JAVA_HOME
/opt/java/current
#> echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/java/current/bin
#> java -version
java version "1.6.0_23-ea"
Java(TM) SE Runtime Environment (build 1.6.0_23-ea-b01)
OpenJDK Server VM (build 19.0-b06, mixed mode)

Obtaining and configuring VuFind for first run

We do not install the standard VuFind. We will use the one that contains multi-ILS capabilities, maintained by Luke O’Sullivan (Swansea). We check-out from SVN, the appropriate branch.

#> svn co https://vufind.svn.sourceforge.net/svnroot/vufind/branches/master /usr/local/vufind

Now, we have a VuFind installation at /usr/local/vufind. We shall now configure it. First, we have to initialise the installation by running /usr/local/vufind/install. This will setup MySQL database amongst other default configurations. NOTE that at the time of writing, there was a bug in installation script, as filed in the bug report http://vufind.org/jira/browse/VUFIND-315. The fix will be available in version 1.1 or revision r2976 of https://vufind.svn.sourceforge.net/svnroot/vufind/branches/master. Updates to versions can be obtained by running svn update in /usr/local/vufind. We also need to make the following scripts executable in /usr/local/vufind.

#> chmod +x vufind.sh
#> chmod +x import-marc.sh

The following two lines need to be added to /etc/profile before export JAVA_HOME and the file needs to be re-sourced.

VUFIND_HOME="/usr/local/vufind"

export VUFIND_HOME

Now, we need to tell Apache where VuFind lives. Also, we need to let Apache have write access to some directories. We can do that as follows.

#> ln -s /usr/local/vufind/httpd-vufind.conf /etc/apache2/conf.d/vufind
#> /etc/init.d/apache2 reload
#> chown www-data:www-data /usr/local/vufind/web/interface/compile
#> chown www-data:www-data /usr/local/vufind/web/interface/cache
#> chown www-data:www-data /usr/local/vufind/web/images/covers/*

The final step is to edit the /usr/local/vufind/web/conf/config.ini to match the requirements of the institution (e.g. support email, path to VuFind, URL on the web server, etc.) and then run /usr/local/vufind start. Running it at this stage will generate errors as multi-ILS configurations have not been done properly, which is what we will work on later, after we have gone through some other installations.

Installing the Talis-Jangle connector

In our specific installation of VuFind, we plan to have multi-ILS connections to Talis systems. For this reason, we must install a number of other things to make VuFind work with Talis. One of the first things to install is the Talis-Jangle connector http://code.google.com/p/talislms-jangle/. The instructions provided below are derived from http://code.google.com/p/talislms-jangle/wiki/InstallRailsConnector.

Install JRuby

We install JRuby in the way we installed Java. Download the latest distribution of JRuby (1.5.2 at the time of this writing) from http://jruby.org/download into a new directory /opt/jruby. Untar it and create a symbolic link such that /opt/jruby/current points to the untarred JRuby installation. Now, change the PATH line in /etc/profile right above the declaration for JAVA_HOME to PATH="$PATH:/opt/java/current/bin:/opt/jruby/current/bin" and run:
#> source /etc/profile
After this, running the following will show the version of JRuby you have installed, for example:

#> jruby -v
jruby 1.5.2 (ruby 1.8.7 patchlevel 249) (2010-08-20 1c5e29d) (OpenJDK Client VM 1.6.0_23-ea) [i386-java]


Update RubyGems and install various Gems

To update RubyGems, we run

#> jruby -S gem update --system

After that, we install the following:

#> jruby -S gem install rails -v 2.3.8
#> jruby -S gem install rsolr-direct
#> jruby -S gem install activerecord-jdbc-adapter -v 0.9.4
#> jruby -S gem install marc jrexml vpim composite_primary_keys cql-ruby glassfish

Having installed the various Gems, we now install the Sybase JDBC driver from http://jtds.sourceforge.net/, unzip and place the JTDS driver JAR file in /opt/jruby/current/lib.

Obtaining the Talis-Jangle connector

We checkout from SVN (http://talislms-jangle.googlecode.com/svn/trunk/) the Talis-Jangle connector into /opt/talislms-connector. The file /opt/talislms-connector/config/database.yml-dist needs to be renamed to /opt/talislms-connector/config/database.yml and edited to reflect the connection to Talis Sybase. The file /opt/talislms-connector/config/connector.yml may be edited to change the administrator password. Before we start Glassfish, we have to create a symbolic link to the Solr directory as follows:

#> cd /opt/talislms-connector/lib
#> ln -s apache-solr-1.4.0 solr

This helps us keep link to Solr and update it later in case a different version of Solr is used. However, a more up-to-date version of the Talis connector may not need this as it may automatically point to the apache-solr-1.4.0 directory. To start Glassfish in production mode:

#> cd /opt/talislms-connector
#> jruby -J-Druby.runtime=8 -S glassfish -e production &

This step starts up Glassfish, which takes quite a long time to cache the Sybase database. The log can be found at /opt/talislms-connector/log/production.log

Obtaining the Jangle feed generator

Having configured the Talis Jangle connector, we now need to configure the feed generator. We can download it by checking out the SVN repository http://jangle.googlecode.com/svn/trunk/PHPCore as follows:

#> cd /usr/local
#> svn co http://jangle.googlecode.com/svn/trunk/PHPCore jangle-feed

We must tell Apache about jangle-feed. To do so, create a file called jangle-feed-httpd.conf inside the jangle-feed directory with the following contents.

Alias /janglefeed /usr/local/jangle-feed
<Directory /usr/local/jangle-feed/>
Order allow,deny
Allow from 127
AllowOverride All
Options -Indexes
</Directory>

Then create a symbolic link and reload Apache:

#> ln -s /usr/local/jangle-feed/jangle-feed-httpd.conf /etc/apache2/conf.d/janglefeed
#> /etc/init.d/apache2 reload

We must also configure the Jangle feed generator by ensuring that the config file /usr/local/jangle-feed/config/config.yml looks as follows.

server:
   base_url: http://localhost/janglefeed/
services:
   talislms:
     connector_url: http://localhost:3000/connector/
     description: Alto

cache_location:
   directory: "./cache"
   function: functions
   page: pages

Note that there is nothing available at http://localhost:3000/connector/, but the config file uses that as a base URL. We must also install PHP Caches:

#> pear install Cache-1.5.5

and /usr/local/jangle-feed/.htaccess must be edited to contain the following.

RewriteEngine On
RewriteBase /janglefeed
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php [QSA,L]

Thus, the Jangle feed generator will now be available at http://localhost/janglefeed/talislms. So, for example http://localhost/janglefeed/talislms/items shows the Jangle feed.

This concludes the installation process for VuFind and the associated software. Before we move on to configuring VuFind, let us setup some administrative scripts.

Auto-starting VuFind

VuFind provides a startup script, which can be used by Linux to auto-start VuFind. However, we also need the Talis Jangle connector to be started up. Therefore, we write our own startup script as follows, and save it to /etc/init.d/vufind-all:

#!/bin/bash
#/etc/init.d/vufind-all

### BEGIN INIT INFO
# Provides:    vufind-all
# Required-Start:   $local_fs $network
# Required-Stop:   $local_fs $network
# Default-Start:   2 3 4 5
# Default-Stop:    0 1 6
# Short-Description: Start VuFind daemon at boot time
# Description:  Enable VuFind and associated services provided by daemon.
### END INIT INFO

#See details about LSB information at http://wiki.debian.org/LSBInitScripts

source /etc/profile

TALISLMS_CONNECTOR_PATH=/opt/talislms-connector
VUFIND_PATH=/usr/local/vufind

touch /var/lock/vufind-all

case "$1" in
  start)
    echo "Starting VuFind and other associated services..."
    cd $TALISLMS_CONNECTOR_PATH && jruby -J-Druby.runtime=8 -S glassfish -e production &
    cd $VUFIND_PATH && ./vufind.sh start
     ;;
  stop)
    echo "Stopping VuFind and other associated services..."
    GLASSFISH_PID=`ps aux | grep glassfish | grep jruby | awk '{print $2}'`
    cd $VUFIND_PATH && ./vufind.sh stop
    kill $GLASSFISH_PID
     ;;
  *)
    echo "Usage: /etc/init.d/vufind-all {start | stop}"
    exit 1;
     ;;
esac

exit 0;


To add this to Debian startup, run update-rc.d vufind-all defaults.

Having VuFind on the web server root

Sometimes it may be necessary to have VuFind on the root of the web server (e.g. http://hostname/ instead of http://hostname/vufind). We show the steps to configure this on our Apache2/Debian setup.

First, remove the symbolic link /etc/apache2/conf.d/vufind that was created earlier in this installation process. Following that, run #> a2dissite default, which will disable the default site on the root of the Apache2 web server. Now, create a file /etc/apache2/sites-available/vufind with the following contents.

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  ServerName full-host-name-of-webserver
  ServerAlias full-cname-of-webserver

  DocumentRoot /usr/local/vufind/web

  ErrorLog /var/log/apache2/vufind-error.log
  LogLevel warn
  CustomLog /var/log/apache2/vufind-access.log combined
</VirtualHost *:80>

Between the DocumentRoot and ErrorLog declarations, copy and paste the entire <Directory> section from /usr/local/vufind/httpd-vufind.conf and change the RewriteBase line to RewriteBase /. Do not copy the Alias line. Note that we have not copied the Location line, so this will disable the web-based admin interface for VuFind, which is not a bad thing to do.

In /usr/local/vufind/web/conf/config.ini, make sure the Site section is edited to contain the following:

[Site]
path = http://full-host-name-of-webserver
url = http://full-host-name-of-webserver


You can use a host name corresponding to DNS A or CNAME records for your web server. Having done this, enable the new site and reload the web server.

#> a2ensite vufind
#> /etc/init.d/apache2 force-reload


In our view, this is not the best way of configuring VuFind at the root of the web server, but it works for now.

Configuring VuFind

One of the first things to do regarding the multi-ILS integration is to edit the DriverIDs section of /usr/local/vufind/web/conf/config.ini file, for example adding two library IDs:

[DriverIDs]
lib1 = "Library 1"
lib2 = "Library 2"


This will imply that there will be directories corresponding to the driver IDs in the directory /usr/local/vufind/web/conf/systems/. Each such directory (e.g. /usr/local/vufind/web/conf/systems/lib1/) will contain a config.ini which will contain configuration information specific to that particular library only.

Using multi-ILS configuration will mean that the imported MARC records from various libraries are merged. This calls for creating new MARC fields that can be used as identifiers necessary for the multi-library search. We should edit /usr/local/vufind/import/marc_local.properties to uncomment the two lines as shown:

# Uncomment the following settings to insert appropriate values for your site:
#collection = "Catalog"
institution = 969a
systemids = 969b
#building = "Library A"


Importing data

[This is draft so far… it is being edited]
1. Use Paul Johnson’s (Swansea) MARC merge script to create the annotated MARC records and import them as usual into VuFind.
2. Requires installation of additional Perl modules, using CPAN. It is necessary to install make.
3. Requires JangleXXX.ini in /usr/local/vufind/web/conf/ directory which is a copy of Jangle.ini from the same directory with XXX specific information, where XXX is the library code being used, e.g. lib1. This file needs to be referenced as config = JangleXXX (without the .ini extension) from /usr/local/vufind/web/conf/systems/XXX/config.ini — rather long-winded configuration, in my opinion!