The SuperJournal Project
Home | Search | Demo | News | Feedback | Members
Summary of SuperJournal Findings: Readers
Draft, 26 April 1999
This is a draft version of the SuperJournal findings prepared for the SuperJournal
Conference on 21 April 1999. This is a "bullet point" summary of key findings by
topic. As we finalize the findings, links will be made from the various headings to
individual documents on each topic with fuller findings, supporting data tables, and
discussion of relevant issues.
1. Core User Requirements
- The most important requirements for electronic journal services are a critical mass of
journals, access, and timeliness. Users want fast and easy access to a wide range of
quality journals that are up to date. If these factors are not in place, it is unlikely
that readers will use an electronic journal service. Of these, Science users rated
timeliness most important, and Social Science users rated the range of journals most
- The next important requirement is the core functionality provided: the ability to
browse, search, and print. Searching is probably more important to Social Scientists, who
are not as well served as Scientists with bibliographic databases in their disciplines.
- The next requirement is for a backfile, ideally of 5-10 years, and this is probably most
important in the Social Sciences. As important as having access to the backfile, is the
knowledge that the journal content (both current and back issues) will remain available
into the future, and will not disappear (eg when a subscription ceases). Depth and
permanence of content in combination with breadth will enhance the overall critical mass
and encourage users to make more extensive use of the service (though depth and permanence
on their own are unlikely to convert non-users into users).
- The last important requirement is that of gateways or "one-stop shopping",
ways for users to discover what relevant information is available and then get to it
quickly. Users do not want one monolithic service that works in only one way, but choice
among multiple services, and organization of content that facilitates discovery within
their disciplines. Clustering journals by subject, local library gateways, and interfaces
between large bibliographic databases and full text journals will all contribute to
seamless discovery and access.
- From the user's point of view, the key benefits of electronic journals are
convenience, in particular that of desktop access (from wherever their desktop happens to
be), keeping up to date with their disciplines, saving time, and managing the journal
literature more easily and efficiently. Another benefit they hope for is access to a wider
range of journals that their library currently offers, decreasing the time and frustration
of interlibrary loans.
2. Users and Behaviour
Use of Printed Journals
- Typically Science users work in a competitive environment, and spend more time on a
regular basis keeping up with what's being published in their research area. The
Social Scientists seem more task-driven, and use journals and the library in response to
specific tasks, eg write an essay, research for a publication, prepare a lecture, etc.
- Scientists generally use online bibliographic databases to identify relevant articles,
where Social Scientists use a broader range of methods for discovery, including following
references in other articles and browsing, as well as using databases.
- When Scientists find a relevant article they generally photocopy it to read later. Many
Social Scientists do the same, but more of them read and take notes than the scientists,
and some consciously try to avoid copying for environmental reasons.
- Both Scientists and Social Scientists choose a place to read journal articles which they
describe as convenient, quiet, and conducive to reading. For Scientists this is most often
their office. For Social Scientists, it may be the office, library, or home.
- Scientists and Social Scientists value many aspects of the printed journal, eg it's
easy to "flip" through, easy to read, and has quality presentation. They value
photocopies of articles because they are theirs to own and annotate, store and refer back
to, and they are portable. Scientists also mentioned the importance of high quality
graphics, particularly colour images.
The Users of SuperJournal
- Postgraduates were the most numerous users of SuperJournal. Postgraduates accounted for
36% of registered users spread evenly across the journal clusters.
- Academic and research staff accounted for 18% and 22% of users. They displayed the
highest rates of repeat use (45% and 47%), perhaps due to continuity associated with their
research and permanent status.
- Undergraduates accounted for 20% of users, and their numbers were growing quickly in the
last few months of SuperJournal. They displayed the lowest rate of repeat use (25%),
perhaps because their needs varied from term to term with course work.
Patterns of Use for SuperJournal
- Reasons for using SuperJournal varied per discipline. Social Sciences users said they
used it most often to identify some recent articles in their area of research. Science
users said they used it to keep up to date with specific journals and to find particular
journal articles. Their use tended to be more directed, and that of Social Scientists more
- Repeat users use SuperJournal on average every other month.
- Science users used the journal cluster that best matched their disciplinary interests,
but users in the Social Sciences often used both the CCS and PS clusters, particularly
where their research was in interdisciplinary areas.
- Users said that typically they browsed the tables of contents and then read abstracts.
When they found a relevant article, they would print it out, with about half studying it
onscreen before printing, and half printing directly. This is entirely consistent with how
they use printed journals.
Types of Repeat Users
Cluster analysis was done on the usage logfiles, and repeat users were found to cluster
into five groups based on frequency, breadth, and depth of use.
- Enthusiastic Users: This is a small group users, typically in the Social Sciences, who
used SuperJournal very frequently, used a large number of journals, and in most sessions
viewed full text articles.
- Vanilla Users: This is a large group of mainstream users spread evenly across the
journal clusters. They used a moderate number of journals with moderate frequency, and
viewed full text articles in just under two thirds of their sessions.
- Unfulfilled Users: This is a large group of users, typically users of the MGP cluster.
They used only one or two journals, used the system infrequently and got to article level
in 15% of sessions. They appeared to be using the service to check on one or two journals
and were happy that, on most occasions, they did not to find anything they had to read.
- Gap-Fillers: These are mostly Science users. They are like the Infrequent Journal Users
in that they only accessed a few journals, but did so much more frequently, and viewed
full text more frequently (half of their sessions).
- Demand Specific Users: Typically these users are in the Sciences, but many are in the
Social Sciences. They used a small number of journals and used them infrequently, but they
got to full text in a high percentage of sessions. They appeared to have specific article
references and were using SuperJournal to get a copy.
Types of Non-Users
- Many people registered to use SuperJournal and used it once, or for a short period of
time (a month or less).
- The reasons these non-repeat users had for using SuperJournal were similar to the
reasons of repeat users, ie to keep up to date with specific journals, to identify recent
articles of interest, and to get specific articles.
Cluster analysis was done on the usage logfiles, and non-repeat users were found to
cluster into three groups based on frequency, breadth, and depth of use.
- Tourists: The non-repeat users included a large group of tourists who visited the
application but never viewed a particular journal. Many of them seemed to come from
unrelated disciplines and apparently had no interest in the journals offered via
SuperJournal. They may have looked at SuperJournal from curiosity, or because the
librarian suggested it.
- Lost users: The other two groups did explore the service more thoroughly but did not
return after the first month. The lost users behaved like enthusiastic users in the month
of their use, but they did not return subsequently. Possibly they had a short-term
interest, such as a student project, and then had no further interest in the journals.
- Exploratory users: Typically they checked a few journals and articles but did not come
back. Like the tourists, they may have concluded there was insufficient of relevance for
Changes in Work Practice
- Electronic journals are resulting in fewer visits to the library. Two years ago most
Scientists and Social Scientists said they used the library weekly, some monthly, and a
small number daily. In a similar survey at the end of the project, half of all respondents
said they are using the library somewhat less because of access to electronic
journals, and a quarter of the Scientists say they are using it much less than
- Users do not perceive electronic journals to be a "replacement" for the
library as an institution. They value the library as a place to visit and browse through
journals, the location of the journal archive, and where they can find helpful staff.
However, they do perceive electronic journals as a replacement for the process of
getting copies of journal articles quickly and easily without physically visiting the
library, an important service.
- Readers also view electronic journals as a way to extend the range of journals they have
access to. In the survey two years ago, half of the Science readers and 70% of the Social
Science readers said their access to journals significantly limited the breadth and depth
of their reading, giving lack of journals in the library as the most frequent reason. In a
similar survey at the end of the project, over 90% of the Social Science readers said they
used journals in SuperJournal they hadn't read before, and 70% journals that
weren't available in the library.
- Many SuperJournal users reported a great sense of comfort that they could now stay up to
date with their discipline in an easy and manageable way, particularly in the Social
- A small area where user expectations has changed is user authentication. Two years ago
users viewed usernames and passwords as a ubiquitous but necessary evil. Now they are
aware of systems like ATHENS and feel they should no longer be subjected to the
inconvenience of logging in.
- In some areas, users would like to see little change. They value the journal and
expressed concern about changes that might damage the quality of articles, both in content
(peer review) or format, or endanger the permanent archive function of journals.
3. Use of Content
- Users liked the concept of journal clusters, with a range of journals from different
publishers all on the same general topic. The concept was easy to understand and it was a
manageable way to gain access to the journals.
- A follow-up survey of repeat users indicated they were fairly well divided between
wanting small clusters of 20-50 journals, and larger more comprehensive clusters of
- Some chose broad topics for their ideal cluster (eg Biomedical Sciences, Social
Sciences) and some chose narrower topics (eg Molecular Biology, Cultural Studies), but
overall the breadth of the topic did not determine the number of journals they wanted. The
most common topics requested by Science users were Biological Sciences, Biomedical
Sciences, and Molecular Biology. The most common topics requested by Social Science users
were Social Sciences and Political Science.
- Users were also asked how many journals relevant to them personally the cluster should
contain. Typically responses were in the range 5-10, regardless of the size of their ideal
- The clusters of journals offered by SuperJournal were on relevant topics, but in general
did not contain enough journals, or enough relevant titles within each cluster. Interviews
with repeat users indicate that 50% of the Social Science users found the SuperJournal
clusters contained over 20% of the journals they needed, where only 10% of the Science
user could say the same.
- For the cluster concept to work, it's essential that it should contain sufficient
relevant journals for the intended audience. This was the most frequent reason given by
non-repeat users for not returning to use SuperJournal. Even for repeat users, the most
disliked feature of the service was that it did not contain enough of the right journals.
- Users consider the backfile to be an important feature of an electronic journal service,
and most would like a backfile of 5-10 years. Responses to the user survey varied from 1
to 50 years, with Science users averaging 7 years, and Social Science users averaging 11
- The user's discipline is one factor in how important the backfile is, but also
important is the task at hand and the particular journals.
- The presence of backfile is likely to influence how a user perceives an electronic
journal service and how s/he will use it, but is less likely to be a deciding factor in whether
or not to use it.
Use of the Journals
- Users in the Social Sciences made wider use of the journals than users in the Sciences.
Averaged over all journal users, the majority (82%) of Social Science users used 1-6
journals, where the majority of Science users used 1-3 journals.
- Frequent users use more journals than infrequent users. For the high repeat user groups,
CCS users used an average of 10 journals, PS users used 8, and MGP users used 4, and
Materials Chemistry users used 4.
- A follow-up survey of users indicates that the majority used journals in SuperJournal
they had never read before (58%) and journals they couldn't get in the library (65%).
Particularly in the Social Sciences, availability leads to use. The survey indicates that
over 90% used journals they had not read before.
- Use of journal issues in the current year was greater in the Sciences than in the Social
Use of Articles
- The proportion of articles viewed was greater in the Social Sciences than the Sciences.
For the CCS cluster 75% of the articles were viewed at least once, for PS 64%, and MGP
39%. For MC only 13% of the articles were viewed at least once, but there was a
substantially greater number of articles, and the cluster was available for the shortest
- The proportion of articles viewed varied from journal to journal, and but all the
clusters contained journals where all or virtually all of the articles were viewed at
- We found little evidence of highly popular articles being viewed by many users.
Typically articles were viewed 1-3 times, with a higher proportion of articles viewed once
in the Sciences than Social Sciences.
Use of Abstracts
- Users value abstracts. Most users (72%) said they would like publishers to publish
abstracts in both print and electronic versions of their journals. Five of the Social
Science journals included in SuperJournal did not contain abstracts, and another four did
not have abstracts for all articles.
- The Science users viewed abstracts less frequently than the Social Science users.
Science users more often browsed at Table of Contents level, or went direct from the TOC
to the article itself. Typically users in the Sciences were using SuperJournal to get
specific articles and to keep up to date with specific journals.
- The Social Science users viewed abstracts more frequently. Typically users in the Social
Sciences were using SuperJournal to keep up with what was being published generally, or to
find some recent articles in their subject area, and checking the reading the abstract was
a logical step. Also, site variations in use of abstracts suggest that the user's
facilities may be a factor. If the full article will be slow to load in PDF format,
it's useful to check the abstract for relevance first.
- Focus groups at the start of the project indicated that readers had moderate interest in
"multimedia", ie including additional content in the electronic version of an
- Some saw this as a way to overcome the page or cost restrictions of print, eg to add
more photos, or to add expanded sections on experimental methods.
- Some saw this as a way to verify the author's work, eg by appending the
author's source data. However, as authors themselves, they had reservations about
releasing their own source data, as it might decrease the "mileage" they could
get from it, and it might make it vulnerable to exploitation by others.
- Others saw multimedia as a way to include visual examples of what was described in the
text, and most often cited the potential for including film clips (Social Sciences) and 3D
- A questionnaire was sent to authors of articles in the SuperJournal journals, and
questions on multimedia were included. The majority (61%) used multimedia in connection
with their work. Social Scientists indicated that video, supporting data sets, and film
were most relevant, and Scientists indicated that colour images and 3D graphics were most
relevant. However, only 36% said that the ability to include multimedia content in their
journal articles would be an advantage.
4. The Service
Patterns of uptake
- During a period of 23 months of availability, 2,867 users registered to use SuperJournal
at the 13 test sites. The patterns of uptake varied per site and per cluster, but overall
it took 10 months for SuperJournal to achieve a population of repeat users for each
cluster, which it sustained for the remainder of the project.
- The CCS cluster was launched first and took 12 months to achieve a stable population of
repeat users (22%), MGP was launched next and took 6 months (46%), PS took 8 months (25%),
and MC took 5 months (25%).
- Academic staff were the "early adopters", typically the first to use
SuperJournal, followed by postgraduates and researchers, with undergraduates becoming
Awareness and Promotion
- Most users found out about SuperJournal via the library. The librarians used various
techniques to make users aware of SuperJournal. Typically these included techniques for
general awareness (eg posters and user guides in the library, articles in newsletters),
techniques to indicate what journals were available (eg library Web site, OPAC, notices on
shelves), and promotion targeted a specific individuals who might be interested (eg
letters and emails). It's difficult to say that one technique is "better"
than another, as most were used at each site, each site is individual, and factors other
than promotion and awareness are involved in uptake.
- Surveys at the end of the project suggest that listing electronic journals on the
library Web site is the most successful technique overall. 24.5% of repeat users and 37.1%
of non-repeat users found out about SuperJournal in this way.
- Targeted promotion is also effective. Letters/emails from the librarian was the most
common way that repeat users found out about SuperJournal (29.4%), suggesting the
librarians know their audience and can reach it. Not surprisingly only 14.3% of non-repeat
users found out about SuperJournal by emails/letters.
- The techniques for general awareness proved moderately successful, with about 10% of
users finding out about SuperJournal from posters, and another 10% from brochures.
However, virtually none learned of SuperJournal from the library catalogue (perhaps
because it's book oriented) or from training courses (perhaps because much else is
- Different techniques proved successful at different sites. The library system is
fragmented at Cambridge, with many Departmental libraries. The librarian at Cambridge
focused promotion and awareness on the Departmental librarians more than on end users.
Departmental librarians registered, and spread the word to their users who then
- Some librarians made information about username/password available on
"private" pages of their Web site, and others gave a contact name to call/visit.
Success in this area is difficult to assess, but some of the librarians felt they had
greater success when barriers to knowing the username/password was removed.
- Interestingly, library demonstrations featuring electronic journals generally, were
quite successful in increasing registrations (eg at Ulster, NIMR, and DMU). These may have
increased awareness of electronic journals generally among the user population that was
not aware of them previously.
Barriers to use
- The most important factor in whether a user will use an electronic journal service is
the number of relevant journals. A survey of non-users indicated that this was the main
reason they didn't use SuperJournal.
- The second greatest barrier focuses on access, eg ability to access, slow access,
problems with passwords, or other technical problems. Non-users who gave access as a
reason for not using SuperJournal typically encountered a problem and didn't use it
- A number of other reasons for non-use were also given, some more related to lifestyle
and culture than features of an electronic journal service. For example, some non-users
said they didn't have time to use SuperJournal, forgot to use it, used other
services, or preferred to visit the library.
Time of Use
- Users value the ability to use electronic journals at any time of the day or week, but
in practice tended to use SuperJournal during "working hours".
- During working hours (9 AM 6 PM), the hours of greatest use were 2-5 PM, with 2-3
PM the hour of greatest use. The hour of least use was 9-10 AM.
- During the working week (Monday Friday), usage was fairly well distributed over
the days, with Wednesday the day of greatest use and Friday the day of least use.
- A significant amount of usage (21%) took place outside working hours, most during the
evening, some at weekends, and the least in the early mornings.
- Sessions did not vary much in length throughout the day and week (average 11 minutes),
but sessions in the early morning were distinctly longer (19 minutes).
Location of Use
- Users value the concept of "desktop access" and the ability to access
electronic journals from any location. The convenience of getting journals without leaving
your desk (wherever that might be) was considered to be the most important benefit of
- Most use of SuperJournal was on-campus use. Though in many cases we could not determine
the user's location, we estimate that on-campus use outweighed off-campus use by
- The largest amount of on-campus access took place in departments, followed by cached
areas (eg computer clusters), and the library. The largest amount of off-campus use took
place in colleges and homes.
- Users have mixed views about using electronic journals from home. Some would like to use
from home, as it's quiet. Some prefer not to "take work home". Students
seem most interested, and staff less so.
- Off-campus use varied per site. This does not seem related to the universities'
facilities for offsite use. The sites with the most and least offsite use all have dial-in
- The use of SuperJournal followed the pattern of university life. There was relatively
little use during vacations and a lot of use during term time.
- Decreased use during vacations resulted more from loss of users (natural turnover) than
decrease in frequency of use among individual users. The degree of seasonal change was
therefore more dramatic for students than for staff.
- Growth in the number of users and number of sessions was fastest in May, October, and
November, with October the month of fastest growth. July, August, and December were the
months in which users and sessions dropped, with greatest decrease in August.
- Seasonal variation was greatest among undergraduates, followed by postgraduates. Use by
academic and research staff decreased slightly during summer vacations, but was less
marked and for a shorter time.
- A survey of repeat users indicates that most (over 90%) would like user support by
email. Most felt that support within "working hours" (9 AM to 6 PM) was
sufficient, and expected a response within 24 hours.
- The most common user support question asked by SuperJournal users was to be reminded of
their Personal ID for logging in. This accounted for over 90% of emails received by
- The next most common user support question related to error messages that the user did
not understand, for example when the Manchester server was down or when it could not
accept any more users at peak times.
- The next most common user support issue focused on Adobe Acrobat Reader: how to get it,
install it, and get it to work properly.
- The feedback feature allowed users to send feedback messages to SuperJournal. About half
related to specific problems with using SuperJournal (eg searching), and about half were
general comments about the service (good and bad), suggestions for enhancements, and
questions about journal coverage (eg when will the journal I want appear).
- SuperJournal required all users to register, so that usage data could be logged, and to
ensure that the service was restricted to authorized users at the test sites. Each site
chose a username and password, gave them to potential users, they registered them, and
subsequently logged in to SuperJournal using their email address and a personal ID they
chose during registration. Users could later change their email address and personal ID if
they wanted to.
- Overall, the system worked well and few problems were encountered. As noted above, the
most common problem from the user's point of view was remembering their personal ID
once they had registered.
- Some users were confused by having a username/password that was only used once for
registration, and didn't understand that the email/personal ID should be used for
- The follow-up studies with users and non-user indicate that user authentication is an
important issues, and users would much prefer to be "recognized" by the system
than to have to remember multiple IDs or passwords for different systems.
Core vs "Nice to Have"
- Users perceive browsing, searching, and printing as the core features of electronic
- Features that allow the user to customize the service in some way, eg alerting, saving
references, building a virtual filebox of articles, and annotating articles, are
considered "nice to have".
- Users consider links from bibliographic references at the end of the article to
abstracts or the article itself to be "nice to have". They put a higher priority
on getting to a relevant article quickly and easily than on getting from the article to
somewhere else. However, they did express particular interest in links from the article to
the author's Web site and/or the author's previous articles.
- Browsing was the most common method of using SuperJournal. Browsing was used by nearly
all users in most sessions. The SuperJournal application was designed to make browsing
easy. If it had been designed differently (eg with a search engine on the opening screen),
the results might have been different.
- Users valued the easy-to-use feature of browsing by cluster.
- The ability to search for articles was valued by most SuperJournal users, in particular
those in the Social Sciences. Users valued searching titles and abstracts more than full
text searching of articles.
- Actual use of the search engines was low. They were used by one third of users in a
small proportion of the user sessions. Typically users rely on the large bibliographic
databases for searching, especially in the Sciences.
- Social Sciences users used the search engines more than Science users, students more
than staff, and frequent users more than infrequent user.
- Of the three search engines available to users, Isite was most frequently used, followed
by NetAnswer, and RetrievalWare was not used very much. (These results are consistent with
how long each search engine was available and how easy it is to use.)
- Users tended to do simple searches. Isite users typically specified one or two search
terms, and NetAnswer users one term.
Other special features
- There seems to be little interest in discussion forums or communicating with other
users. The Whiteboard feature was intended to serve as a discussion forum for users, to
comment on articles and discuss issues related to their disciplines. The feature was not
used much, but users of the MGP cluster used it to post questions to SuperJournal about
journal coverage and timeliness. Those who did use to communicate with other users in
their disciplines rarely got a response.
Timeliness and immediacy
- Timeliness is most important in the disciplines that move quickly, for example in the
life sciences. Science users ranked timeliness as the most important factor for electronic
journals, where Social Science users ranked timeliness second (with range of journals
Timing of electronic publication
- Users would like for electronic publication to coincide with print, or continuous
publication of electronic, followed up with publication of the printed issue.
- Users value the journal issue and the signposting associated with publication on a
particular date. Notification that the issue has been published helps them to organize
their time and prompts them to visit the journal.
- Users generally don't like to read articles onscreen. Most prefer to scan the
article onscreen, print it off if it's relevant, and read the article in hardcopy
- Users value the following when viewing articles onscreen: the article loads quickly, is
easy to scan, and has good presentation. While they like HTML for Web applications, they
commented that HTML doesn't indicate page numbers, and that presentation of tables
and equations is often poor quality.
- Users who were asked said they preferred viewing articles in HTML format over PDF, but
usage statistics indicate that articles available in both formats were viewed more
frequently in PDF format.
- Users value HTML because it loads quickly and is easy to scan onscreen, but prefer the
quality presentation of the PDF format overall.
- Most users print out articles. They want a copy of the article to "own" and
refer back to. Many noted that annotating the article as they read helped them to
- Users value the following when printing: high quality presentation, speed, similarity to
the printed page, the printout should contain all the information in the article (HTML may
lack page numbers). For users in the Life Sciences, high quality graphics are also very
- Users who were asked said they preferred printouts as PDF, with virtually none wanting
HTML printouts. Users in the Life Sciences were particularly keen on PDF, as a PDF
printout is better than a photocopy, especially for high-resolution graphics.
- Problems with Adobe Acrobat Reader were the most frequent technical problem experienced
by users, and some said this was a reason for not using SuperJournal. Typically users had
problems getting/installing Acrobat, or had low specification PCs that made viewing or
printing articles very slow.
- Journals in the Life Sciences contain high resolution graphics, often in colour, and
users need to be able to print them.
- Some users said they would prefer to have a "virtual filebox" of their
electronic articles to replace their file cabinet of printouts and photocopies, and think
this would reduce the need for printing.
This web site is maintained by firstname.lastname@example.org
Last modified: April 26, 1999