Link Choice Usability Test
on UC Berkeley Library Prototype Home Page
Report of Findings, November 2003This test was administered between October 28 and November 7, 2003, using a paper questionnaire and a printed mockup of the prototype home page. It was not intended as a comprehensive usability test, but rather as a way of getting data on specific link-naming issues that had arisen in the Web Advisory Group (WAG). The test was conducted using our approved human subjects protocol, so this report may be shared outside the UC Berkeley Library.
Summary of results
In most cases, users made appropriate choices from the array of links presented. However,
the “Find/by type” link (Question 1) was not well understood; this area of the home page
needs further consideration. There are indications (Questions 1 and 3) that some users do not understand the scope and limitations of our library catalogs. Other results suggest possible content to be added to the “Help finding information” and “Frequently asked questions” pages. Two terms we have debated – “Article Databases” and “Electronic Resources” – both tested well, the latter with some qualifications. Specific analysis and recommendations follow the table of results for each question below.Respondents
Questionnaires were given to several RRC staff members (none familiar with the prototype or otherwise involved in the web design process) and to users in the Doe, Moffitt, and Biosciences Library reference areas, the latter with the help of Ilan Eyman. There were 18 respondents:
4 staff, 9 undergraduates, 3 graduate students, a Visiting Scholar, and a CSU Northridge faculty member.Major fields of interest included Anthropology (2 respondents), Development Studies/Economics, Ecosystem Science, English, History, Integrative Biology (3), Law, Natural Resources Manage-ment, Plant Biology, Rhetoric, and Statistics.
Excluding the library staff members, respondents reported coming to the library in person (each month):
Less than once 1
1-5 times 3
6-10 times 3
More than 10 times 6
This is my first time 1Excluding the library staff members, respondents reported using the library’s web site (each month):
Less than once 2
1-5 times 4
6-10 times 0
More than 10 times 7
Haven’t used it 1Thus, respondents tended to be frequent users of the physical library, with more variation in terms of how often they use the library web site, tending toward heavy use. It should be noted that “library web site” is a term with several possible meanings. The library-wide web site is not used as the default public home page in any of the locations where tests were conducted. Some respondents may have intended their answers to mean only the main site, or only the site used in their location.
Data – Part 1
The questions were based loosely on WAG’s initial list of hypothetical user tasks. Questions 1-5 provide data on link choice in the context of the page, when the user can see all his/her options. These questions invited test participants to refer to a printed mockup of the prototype home page, indicating which link they would try first for each task, and which link they would try if the first one didn’t provide the information they sought.
Participants were also asked what they would expect to find under their first choice link. Without the ability to press for details or ask follow-up questions as one can in a live interview, these parts of the questionnaire did not yield meaningful information. Most respondents simply echoed the link name or described the “target” item named in the question. These results are not included in this report.
Boldface in the tables below indicates the choices that were intended by the page designers. In some cases, other choices (including the generic Search and Sitemap) might work also. Many of the user choices indicated here would not be successful.
1. You need to find historical manuscripts, and you know these aren't listed in the library catalog.
First choice
# responses
Second choice
# responses
Help finding information
6
Help finding information
3
Find/By Subject
3
Libraries and collections A-Z
2
Electronic Resources
1
Electronic Resources
2
Guide to Library Catalogs
1
Frequently asked questions
2
Other library catalogs
1
Find/By Type
1
Find
1
Find/Books
1
Find/By Type
1
Help
1
Search
1
Research guides
1
“Google”
1
Pathfinder
1
[Left blank]
2
“Yahoo”
1
[Left blank]
3
These results show “Find/By Type,” as presented in the screen mockup, was not well understood. In conversations after taking the test, two respondents told us they believed everything in the “Find” menu related to (and would lead to) the library catalog. This reflects the common perception that the catalog “has everything,” and it suggests that our “Find/By Subject” option may need to be clarified also. When the full usability test is done, this area should be a focus of attention.
Overall, the spread of answers on this question underscores the challenge of making information access understandable. Since the “Help finding information” link was the most frequent first and second choice, that page should be designed to provide some initial guidance for users looking for various kinds of material, as well as describing our reference services.
2. You’re a new student with a physical disability and want to know if there are any special services available.
First choice
# responses
Second choice
# responses
Services for users
14
Frequently asked questions
4
Contact us
1
Help finding information
3
Visitor information
1
Library privileges
2
“Library Map – Visitor Info”
1
Visitor information
2
[Left blank]
1
Contact us
2
More … [under About the Libraries]
1
Search
1
“I would log off and speak to someone.”
1
[Left blank]
2
“Services for users” appears to be a big success here. This result may be skewed somewhat because the word “services” occurs in the question. It would be wise to re-test this link name in a different context, perhaps with faculty. The “Frequently asked questions” (FAQ) page should include information on services to the disabled; the current production FAQ page does not cover this topic.
3. You want to look up a book in the Berkeley Public Library.
First choice
# responses
Second choice
# responses
Other Library Catalogs
11
Other Library Catalogs
2
Pathfinder
2
Libraries and collections A-Z
2
Guide to Library Catalogs
1
Search
2
Books [under Find]
1
Help finding information
2
Libraries and collections A-Z
1
Guide to Library Catalogs
1
Borrowing from other libraries
1
Books [under Find]
1
“I wouldn’t go to the [UCB] website. I’d look for the Public library website.”
1
Pathfinder
1
GLADIS
1
Borrowing from other libraries
1
Frequently asked questions
1
Contact us
1
[Left blank]
3
This is another case in which our intended link, “Other Library Catalogs,” was quite successful. Perhaps a link to “other libraries” could be added to the bottom of the “Libraries and collections A-Z” menu. This result also shows that some users do not understand the scope of Pathfinder/GLADIS.
4. You want to send a question to our reference librarians.
First choice
# responses
Second choice
# responses
Contact us
14
Help finding information
3
Help finding information
3
Frequently asked questions
2
“lib.berkeley.edu”
1
More … [under About the Libraries]
2
Research guides
2
Contact us
1
Search
1
Sitemap
1
Borrowing materials
1
“I would call”
1
“a human”
1
“I’d just go to a library and talk to someone.”
1
[Left blank]
2
Once again, our intended links scored well. An FAQ item on “How do I ask a question?” seems indicated.
5. You want to apply for a job in the library.
First choice
# responses
Second choice
# responses
Frequently asked questions
6
More … [under About the Libraries]
6
More … [under About the Libraries]
5
Contact us
3
Contact us
4
Frequently asked questions
3
Giving to the Library
2
Help finding information
1
“lib.berkeley.edu”
1
Sitemap
1
UC Berkeley Home
1
“berkeley.edu”
1
[Left blank]
2
Our intended links worked well here. The results strongly suggest adding an FAQ item on library employment (the current production FAQ page does not cover this topic). It’s hard to know what to make of the two respondents who chose “Giving to the Library”; neither one is a staff member.
Data – Part 2
Questions 6 and 7 provide data on link choice outside the context of a particular page. These presented a set of essentially synonymous link names for each task, asking participants to mark which one seemed most natural to them, and to offer any alternative suggestions that came to mind, without referring to the printed screen mockup or to the existing web site.
6. Your professor has asked you to find journal articles on a topic. Which of these would you be most likely to try first?
Link name
# responses
Article Databases
12
Indexes and Abstracts
2
[Neither choice marked]
4
Alternatives suggested
“Use the word Journal?”
“Journal”
“Journal Articles” [suggested by 2 respondents]
“Journal Article Databases”
“[Article Databases] is more true for students who don’t already know to go to ‘Indexes & Abstracts’ as it stands now. I’m sure you realize this but if you were to change it, many existing students might be thrown off.”
“[Indexes and Abstracts] but I’ve been conditioned.”
“’Articles’ heading, then subheadings by genre: journal, news, etc.”
“Article Databases” scored convincingly well, being chosen by 67% of respondents. Respondents’ comments about adding the word “Journal” make sense in the context of this question, but we are left with the issue of non-journal article databases. Perhaps we should use the word anyway, since it so commonly appears in library assignments and faculty utterances. The final suggestion above is, of course, just what we are planning to do on the “Find articles” page.
7. The library offers hundreds of information sources in electronic form, available online through its web site. These include atlases, bibliographies, encyclopedias, dictionaries, full-text journal articles, and electronic books. Which is the best name for a link that leads to all of these?
Link name
# responses
Electronic Resources
14
Research Tools
2
Reference Sources
2
Alternatives suggested
“[Electronic Resources] is very broad, and would have to be further defined.”
“[Research Tools is] too general (I would avoid this and assume that it leads to an unhelpful general advice page).”
“Electronic Texts / Electronic Library / Library Online etc.”
“’All Electronic Resources’ [meaning] complete electronic resources (I’d emphasize its importance more).”
“Electronic text”
The popularity of “Electronic Resources” here may be due in part to contamination from respondents looking at the mockup home page (even though the instructions said not to), or just remembering it. Still, 78% choosing this term is substantial, and the written user comments suggest that some of them devoted some thought to this choice. This seems to be our best option but bears further testing.
General comments by respondents
At the end of the questionnaire, participants were invited to write additional comments or suggestions about the web site. There were only four responses, perhaps because the test did not involve actual interaction with the site.
“It’s not visually well designed, thus the user might have to spend time looking for the links they want to go for.”
“Great website! Very easy to use.” [This user was in Doe Library.]
“Once a student knows what they are studying they only visit one or two libraries – making the websites of each individual library user friendly is needed.”
“The BIOS website is useful & I find it easy to use.”
Evaluation of the link choice technique
As far as we know, this is the first time a link choice test of this kind has been done on the library web site. After working through the process, we believe this method is an efficient way to spot check certain kinds of usability problems, particularly those having to do with terminology (link names). As such, it is a complement to full-fledged user observation testing, not a substitute.
This test combined “in context” testing using a paper mockup of the home page, and synonym testing in which users chose among alternative names for the same link. Either method could certainly be used alone. One especially attractive possibility is the administration of a link choice test as an online survey.
We also learned some useful lessons in how to design this kind of test. Asking users to write their interpretations of link meanings, in a task context but without the give-and-take of a live interview, was not fruitful. A simple, non-task-based “define these terms” test might be more productive. Asking for second choices yielded interesting results, but we do not believe it is a necessary part of a link choice test.
Submitted to WAG 11/12/2003
![[ HELP/FAQ ]](http://www.lib.berkeley.edu/Images/help.gif){kind=small}
![[ CATALOGS ]](http://www.lib.berkeley.edu/Images/catalogs.gif){kind=small}
![[ COMMENTS ]](http://www.lib.berkeley.edu/Images/comments.gif){kind=small}
![[ HOME ]](http://www.lib.berkeley.edu/Images/home.gif){kind=small}