We’ve just done our User Experience Testing. If you struggle to read more than 140 characters: It was great. Learnt lots and given us confidence to make changes.
The Vertical Slide Offices (the room where the testing took place was similar)
As part of the project we had enlisted the help of Dr Graham McAllister in the University’s School of Informatics, who not only specialises in User Experience, but also has a spin-out company – called Vertical Slice – based at the University’s Innovation Centre which carries out User Experience Testing mainly for Video Games.
The setup
The User Experience Testing consisted of a series of one on one sessions, each lasting around 40 minutes. Graham asked a number of questions (‘how would you find x’), but also more general questions: what do you think of the site? can you tell what it does? would you use it? He sat next to the user as they carried out the exercises. He’s approachable manner, and the fact he was not connected to the new service, helped in letting the users speak freely as to what they thought.
A number of students and staff from BSMS, the joint medical school (with the University of Brighton), who will hopefully be key users to the new service, agreed to come along. It was carried out throughout the course of one day (Wednesday the 11th May).
On the day
I was sitting in the Vertical Slice offices near to where the testing was taking place, and was able to watch the Video recordings of the screen/video of the user/audio throughout the day.
It would be unfair to say I was cynical of what value it would bring, that would be too strong. But I wasn’t fully sure how useful it could be. They would search for books etc, they would probably find them, some things we know would be harder than others.
Within two minutes of watching the first recording any doubts had been blown away. You could see the frown on their face as something wasn’t obvious and the eyes widen when they spotted what they were after. The mouse would dance between two parts of the screen as they quickly thought through whether to perform a search or try and browse.
An example: we had added a link back to the originating library’s catalogue (or both catalogues if they both had a copy). This, I thought, would be useful for doing anything you could not do on the Vufind interface itself. Yet each time someone was asked how they would reserve a copy, or some other similar task, a frown would appear, then a gap, then a ‘no I don’t know how I would do that’. After the first one I wanted to dive in and start making changes to improve it.
Another example was the filter restrictions that could be applied. For us, a key one is to restrict by institution. However once a filter is applied it continues to be applied to future searches. It does say on the right the filter is applied, and under the search box there was a ticked option to ‘Retain my current filters’ but these things do not matter. What matters is that time and again they were not getting back the results they expected which lead to confusion as it was not clear the filter from before was still being applied. We need to make the active filter much clearer and try and make it impossible to miss, using colour and screen location.
Findings
- On the Homepage: It’s not clear what the service does when you first look at the homepage (though on the whole comments were positive to the clean and modern look).
- The browse options on the homepage (shelfmark, language, item type) – while potential useful at times – distracted more than the use they provided.
Further more, while these seemed like a good idea when I first encounted them – from a technical perspective making good use of the MARC metadata – in fact they can be very misleading as users quite rightly think that when you click on ‘Medicine’ or ‘ebooks’ you will see all items for that selection. In reality due to the lack of detail in older records and the different ways the two Universities have catalogued their collections these browse options rarely displayed all items of the selected choice.
Another reason, users would click on Medicine and be presented with the first few items of the very large list of items classified in the Library of Congress Medicine (R) range. However these first few items were not particularly good examples, or the type of books you would expect to show up, which confused people. - The is perhaps part of a more general trend throughout; simplify where possible. Users actually said ‘all this stuff is getting in the way‘, ‘why am I being shown this information?‘, ‘There might be an option to do X but I can’t see it as there’s so much stuff‘. This was a clear directive: hide whatever is not needed.
- On the search results: need to make it clear what filters are currently applied, probably needs to be at the top, just above the results and hard to miss. Again, users were not always sure if a filter had been applied (‘did that do anything?).
- Sort Date Ascending/Descending : while not hard for the average person to work out what these mean, they do make most people stop and think to make sure they have got it the right way around. This is bad. So use wording that doesn’t require people to think it through (‘oldest first’).
- On Record pages: Make it really clear which Libraries have this item.
- Clear ‘action’ options: Reserve this, etc.
- Use Shelfmark, not Call number. Make sure American English is changed to British English, Catalog to Catalogue.
- Remove Tags/Comments, unlikely to be used and, as noted above, hide the more important options and information.
- While getting Holdings information from the two underlying systems is somewhat of an art, need to continue to refine how this information is displayed.
- Generally: need more information as to what it does, and a dedicated feedback page, which has clear information as to how to contact the two Libraries regarding items and borrowing, and contact us (well, me) regarding comments and issues with the system.
A full list of changes we are going to make can be found here.
What next
Over the next couple of weeks we will be making changes to the user interface based on this feedback. We are using Git to keep track of changes to the system (code/config/language&wording/template) and a Google Spreadsheet to keep track of what has been changed, and what we still need to do.
CReDAUL 