This online survey was conducted for the Web Advisory Group (WAG) to determine what terminology to use for certain functions in UC Berkeley’s Pathfinder library catalog.  After consultation with the university’s Survey Research Center and an initial paper pre-test, the survey ran November 3-16, 2004.  The survey form was made available only through a link on Pathfinder’s entry page, with no special publicity.  It was accessed 370 times and submitted 254 times, a response rate of 68.7%.  Except as noted below,  the margin of error is +/- 6%.

Key findings

• Most respondents find the current term “keyword(s)” appropriate in search names.
   
• A significant majority would prefer to have the left-anchored string searches now designated as “phrase” renamed as “<field> begins with …”.
  
  
• Both "Journal Title" and "Title of Journal" searches were understood correctly (as covering journal titles) by almost all respondents.  About one quarter of respondents and slightly over one third of undergraduates incorrectly assumed these searches would cover article titles, and a smaller number thought they would include full text. Differences between the two alternative names were within the margin of error for this question.
  
  
• No synonym for the current term “telnet” attracted a majority of respondents.  “Command-line” and “telnet” were each favored by about one third, the former being chosen slightly more often.
  
  

Respondents provided a wealth of additional comments, compiled in a separate document and not posted with this report.  As always, some indicated a need for more awareness of current Pathfinder features (e.g., full feature search, browse).  Requests that should be considered for future programming or a next-generation system included the current GLADIS-only functions (inventory, renewals, course reserves searches) and a general, multi-field keyword search.

Respondents

The UCB-affiliated respondents included 47 undergraduates, 119 graduate students, 22 faculty and 25 staff members.  Generally the undergraduate responses showed the same patterns as the overall sample; cases where they were significantly different are indicated below.  Other respondents included alumni, visiting scholars, people from other institutions, and members of the general public. 

Respondents indicated a wide range of subject majors and research interests, most being in the humanities and social sciences.   

As could be expected in a voluntary survey, respondents tended to be frequent visitors to the library and frequent users of Pathfinder.

Because the survey was online, users could be in the library or in remote locations (we did not ask where they were).

Data

1. One of the search options in Pathfinder looks for titles that contain all the words you type, no matter where they occur in the title. Example: searching bell tolls would find For Whom the Bell Tolls. Which is the best name for this kind of search?

Response	Count	Percent
Title	16	6.3%
Title Keyword(s)	158	62.2%
Title Words	72	28.3%
Other	7	2.8%
No response	1	0.4%

"Other" responses:

Words in Title or Title contains Words in title title contains the words title phrase is very useful in many inst Title Words (any place, any order) title (words taken as a phrase) or title words in title Words in the title Title Phrase juvenile death penalty Note: The last response shows the astonishing persistence of users in entering searches into any available input box.

---

2. Another search option looks for titles that start with the words you type, in the order you type them. Example: searching for whom* would find For Whom the Bell Tolls. Which is the best name for this kind of search?

Response	Count	Percent
Title (exact)	23	9.1%
Title begins with ...	180	70.9%
Title phrase	40	15.7%
Other	9	3.5%
No response	2	0.8%

"Other" responses:

Title or Exact Title Portion of title or title portion title exact phrase title keyword Title (specific order) Title (begininning phrase) title keywords (exact order) part of title First words of the title

---

The next question was changed in mid-survey to test an alternate name for this search option. Results for both are given below. With both name variants, misunderstanding of what this search would cover was more prevalent among undergraduates; these results are broken out in separate tables.  Highlighted responses are those indicating the best understanding (or least misunderstanding).

3. We’re considering adding a “Journal Title” search option. Which of the following do you think this search would cover?
(Check all that apply)

All respondents (n=152) Margin of error  +/- 8% Response Count Percent Titles of articles published in journals 42 27.6% Titles of the journals themselves 145 95.4% Full text of the articles 15 9.9%

Undergraduates only (n=22) Margin of error  +/- 21% Response Count Percent Titles of articles published in journals 9 40.9% Titles of the journals themselves 20 90.9% Full text of the articles 3 13.6%

3. We’re considering adding a “Title of Journal” search option. Which of the following do you think this search would cover?
(Check all that apply)

All respondents (n=102) Margin of error  +/- 10% Response Count Percent Titles of articles published in journals 24 23.5% Titles of the journals themselves 94 92.2% Full text of the articles 9 8.8%

Undergraduates only (n=25) Margin of error  +/- 20% Response Count Percent Titles of articles published in journals 8 32.0% Titles of the journals themselves 25 100.0% Full text of the articles 5 20.0%

---

4. GLADIS does not use the web for its displays. GLADIS appears as words on a blank screen, as shown here. When you type your search and press ENTER, your search is executed. Which is the best name for this kind of interface?

Response	Count	Percent
command-line	86	33.9%
telnet	68	26.8%
terminal-style	20	7.9%
text-based	25	9.8%
text-only	23	9.1%
Other	16	6.3%
No response	16	6.3%

"Other" responses:

useless This system is too hard to use. don't know non-web crap! It's an old technology -- get rid of it crap DOS kind of screen ? ? I do not know what to call it?? No opinion doesn't matter non-web user friendly, with no unexplained words terminal style, command-line interface old telnet command-line question-answer old fashioned

---

5. Would the name "text-based" or "text-only" mean that GLADIS contains the full text of materials?

Response	Count	Percent
Yes	38	15.0%
No	202	79.5%
No response	14	5.5%

85.1% of the undergraduates in this sample answered correctly that these names would not mean GLADIS contains full text. This compares favorably with 79.5% of all respondents.

---

6. Any other comments about Pathfinder or GLADIS?

Responses to this question are contained in a separate document, not posted on the web.

---

6. You are:

Response	Count	Percent
UC Berkeley undergraduate	47	18.5%
UC Berkeley graduate student	119	46.9%
UC Berkeley faculty	22	8.7%
UC Berkeley staff	25	9.8%
Other	37	14.6%
No response	4	1.6%

Most of the “Other” responses were alumni (15) and members of the general public (13).

---

7. What is your major or field of interest?

This table is a rough breakdown of the 225 classifiable responses in a free-form comment box.  The preponderance of respondents in the humanities and social sciences probably reflects patterns of library and catalog use, rather than any bias in the survey.

Response	Count	Percent
humanities	72	32.0%
social sciences, education, business	96	42.7%
biological sciences	18	8.0%
physical sciences, engineering	28	12.4%
International/area studies	11	4.9%

---

8. Approximately how often do you come to the library?

Response	Count	Percent
Less than once a month	17	6.7%
1-5 times a month	74	29.1%
6-10 times a month	47	18.5%
More than 10 times a month	106	41.7%
No response	10	3.9%

---

9. Approximately how often do you use Pathfinder?

Response	Count	Percent
Less than once a month	7	2.8%
1-5 times a month	34	13.4%
6-10 times a month	47	18.5%
More than 10 times a month	159	62.6%
This is my first time	2	0.8%
No response	5	2.0%

---

Evaluation of the online survey technique

This survey generated a statistically significant number of responses from the user population we wanted to sample, with relatively little effort.  The WebSurveyor online survey tool, licensed by CDL, turned out to be an efficient way to compose the survey, publish it, collect responses, and analyze the results.  Being able to do all this through WebSurveyor’s web interface was a great convenience.

Lessons learned:  A seemingly simple survey can generate large amounts of information.  Advance effort in thinking through the questions paid off here.  However, changing question 3 in mid-survey (in order to test an alternative search name), created a separate set of results.  The two sets had to be combined manually, a time-consuming process.  This was worthwhile in this case but generally should be avoided. Providing a free-form comment box for respondents to indicate their subject majors/interests yielded over 225 responses that had to be classified and counted manually.  It would be more cost-effective to present users with a multiple choice of broad categories and let them classify themselves.

Submitted to WAG 11/22/2004
Revised 1/5/2005