The SuperJournal Project

Summary of SuperJournal Findings: Readers

Draft, 26 April 1999

This is a draft version of the SuperJournal findings prepared for the SuperJournal Conference on 21 April 1999. This is a "bullet point" summary of key findings by topic. As we finalize the findings, links will be made from the various headings to individual documents on each topic with fuller findings, supporting data tables, and discussion of relevant issues.

1. Core User Requirements

The most important requirements for electronic journal services are a critical mass of journals, access, and timeliness. Users want fast and easy access to a wide range of quality journals that are up to date. If these factors are not in place, it is unlikely that readers will use an electronic journal service. Of these, Science users rated timeliness most important, and Social Science users rated the range of journals most important.
The next important requirement is the core functionality provided: the ability to browse, search, and print. Searching is probably more important to Social Scientists, who are not as well served as Scientists with bibliographic databases in their disciplines.
The next requirement is for a backfile, ideally of 5-10 years, and this is probably most important in the Social Sciences. As important as having access to the backfile, is the knowledge that the journal content (both current and back issues) will remain available into the future, and will not disappear (eg when a subscription ceases). Depth and permanence of content in combination with breadth will enhance the overall critical mass and encourage users to make more extensive use of the service (though depth and permanence on their own are unlikely to convert non-users into users).
The last important requirement is that of gateways or "one-stop shopping", ways for users to discover what relevant information is available and then get to it quickly. Users do not want one monolithic service that works in only one way, but choice among multiple services, and organization of content that facilitates discovery within their disciplines. Clustering journals by subject, local library gateways, and interfaces between large bibliographic databases and full text journals will all contribute to seamless discovery and access.
From the user's point of view, the key benefits of electronic journals are convenience, in particular that of desktop access (from wherever their desktop happens to be), keeping up to date with their disciplines, saving time, and managing the journal literature more easily and efficiently. Another benefit they hope for is access to a wider range of journals that their library currently offers, decreasing the time and frustration of interlibrary loans.

2. Users and Behaviour

Use of Printed Journals

Typically Science users work in a competitive environment, and spend more time on a regular basis keeping up with what's being published in their research area. The Social Scientists seem more task-driven, and use journals and the library in response to specific tasks, eg write an essay, research for a publication, prepare a lecture, etc.
Scientists generally use online bibliographic databases to identify relevant articles, where Social Scientists use a broader range of methods for discovery, including following references in other articles and browsing, as well as using databases.
When Scientists find a relevant article they generally photocopy it to read later. Many Social Scientists do the same, but more of them read and take notes than the scientists, and some consciously try to avoid copying for environmental reasons.
Both Scientists and Social Scientists choose a place to read journal articles which they describe as convenient, quiet, and conducive to reading. For Scientists this is most often their office. For Social Scientists, it may be the office, library, or home.
Scientists and Social Scientists value many aspects of the printed journal, eg it's easy to "flip" through, easy to read, and has quality presentation. They value photocopies of articles because they are theirs to own and annotate, store and refer back to, and they are portable. Scientists also mentioned the importance of high quality graphics, particularly colour images.

The Users of SuperJournal

Postgraduates were the most numerous users of SuperJournal. Postgraduates accounted for 36% of registered users spread evenly across the journal clusters.
Academic and research staff accounted for 18% and 22% of users. They displayed the highest rates of repeat use (45% and 47%), perhaps due to continuity associated with their research and permanent status.
Undergraduates accounted for 20% of users, and their numbers were growing quickly in the last few months of SuperJournal. They displayed the lowest rate of repeat use (25%), perhaps because their needs varied from term to term with course work.

Patterns of Use for SuperJournal

Reasons for using SuperJournal varied per discipline. Social Sciences users said they used it most often to identify some recent articles in their area of research. Science users said they used it to keep up to date with specific journals and to find particular journal articles. Their use tended to be more directed, and that of Social Scientists more exploratory.
Repeat users use SuperJournal on average every other month.
Science users used the journal cluster that best matched their disciplinary interests, but users in the Social Sciences often used both the CCS and PS clusters, particularly where their research was in interdisciplinary areas.
Users said that typically they browsed the tables of contents and then read abstracts. When they found a relevant article, they would print it out, with about half studying it onscreen before printing, and half printing directly. This is entirely consistent with how they use printed journals.

Types of Repeat Users

Cluster analysis was done on the usage logfiles, and repeat users were found to cluster into five groups based on frequency, breadth, and depth of use.

Enthusiastic Users: This is a small group users, typically in the Social Sciences, who used SuperJournal very frequently, used a large number of journals, and in most sessions viewed full text articles.
Vanilla Users: This is a large group of mainstream users spread evenly across the journal clusters. They used a moderate number of journals with moderate frequency, and viewed full text articles in just under two thirds of their sessions.
Unfulfilled Users: This is a large group of users, typically users of the MGP cluster. They used only one or two journals, used the system infrequently and got to article level in 15% of sessions. They appeared to be using the service to check on one or two journals and were happy that, on most occasions, they did not to find anything they had to read.
Gap-Fillers: These are mostly Science users. They are like the Infrequent Journal Users in that they only accessed a few journals, but did so much more frequently, and viewed full text more frequently (half of their sessions).
Demand Specific Users: Typically these users are in the Sciences, but many are in the Social Sciences. They used a small number of journals and used them infrequently, but they got to full text in a high percentage of sessions. They appeared to have specific article references and were using SuperJournal to get a copy.

Types of Non-Users

Many people registered to use SuperJournal and used it once, or for a short period of time (a month or less).
The reasons these non-repeat users had for using SuperJournal were similar to the reasons of repeat users, ie to keep up to date with specific journals, to identify recent articles of interest, and to get specific articles.

Cluster analysis was done on the usage logfiles, and non-repeat users were found to cluster into three groups based on frequency, breadth, and depth of use.

Tourists: The non-repeat users included a large group of tourists who visited the application but never viewed a particular journal. Many of them seemed to come from unrelated disciplines and apparently had no interest in the journals offered via SuperJournal. They may have looked at SuperJournal from curiosity, or because the librarian suggested it.
Lost users: The other two groups did explore the service more thoroughly but did not return after the first month. The lost users behaved like enthusiastic users in the month of their use, but they did not return subsequently. Possibly they had a short-term interest, such as a student project, and then had no further interest in the journals.
Exploratory users: Typically they checked a few journals and articles but did not come back. Like the tourists, they may have concluded there was insufficient of relevance for their purposes.

Changes in Work Practice

Electronic journals are resulting in fewer visits to the library. Two years ago most Scientists and Social Scientists said they used the library weekly, some monthly, and a small number daily. In a similar survey at the end of the project, half of all respondents said they are using the library somewhat less because of access to electronic journals, and a quarter of the Scientists say they are using it much less than before.
Users do not perceive electronic journals to be a "replacement" for the library as an institution. They value the library as a place to visit and browse through journals, the location of the journal archive, and where they can find helpful staff. However, they do perceive electronic journals as a replacement for the process of getting copies of journal articles quickly and easily without physically visiting the library, an important service.
Readers also view electronic journals as a way to extend the range of journals they have access to. In the survey two years ago, half of the Science readers and 70% of the Social Science readers said their access to journals significantly limited the breadth and depth of their reading, giving lack of journals in the library as the most frequent reason. In a similar survey at the end of the project, over 90% of the Social Science readers said they used journals in SuperJournal they hadn't read before, and 70% journals that weren't available in the library.
Many SuperJournal users reported a great sense of comfort that they could now stay up to date with their discipline in an easy and manageable way, particularly in the Social Sciences.
A small area where user expectations has changed is user authentication. Two years ago users viewed usernames and passwords as a ubiquitous but necessary evil. Now they are aware of systems like ATHENS and feel they should no longer be subjected to the inconvenience of logging in.
In some areas, users would like to see little change. They value the journal and expressed concern about changes that might damage the quality of articles, both in content (peer review) or format, or endanger the permanent archive function of journals.

3. Use of Content

Cluster Concept

Users liked the concept of journal clusters, with a range of journals from different publishers all on the same general topic. The concept was easy to understand and it was a manageable way to gain access to the journals.
A follow-up survey of repeat users indicated they were fairly well divided between wanting small clusters of 20-50 journals, and larger more comprehensive clusters of 100-500 journals.
Some chose broad topics for their ideal cluster (eg Biomedical Sciences, Social Sciences) and some chose narrower topics (eg Molecular Biology, Cultural Studies), but overall the breadth of the topic did not determine the number of journals they wanted. The most common topics requested by Science users were Biological Sciences, Biomedical Sciences, and Molecular Biology. The most common topics requested by Social Science users were Social Sciences and Political Science.
Users were also asked how many journals relevant to them personally the cluster should contain. Typically responses were in the range 5-10, regardless of the size of their ideal cluster.
The clusters of journals offered by SuperJournal were on relevant topics, but in general did not contain enough journals, or enough relevant titles within each cluster. Interviews with repeat users indicate that 50% of the Social Science users found the SuperJournal clusters contained over 20% of the journals they needed, where only 10% of the Science user could say the same.
For the cluster concept to work, it's essential that it should contain sufficient relevant journals for the intended audience. This was the most frequent reason given by non-repeat users for not returning to use SuperJournal. Even for repeat users, the most disliked feature of the service was that it did not contain enough of the right journals.

Backfile

Users consider the backfile to be an important feature of an electronic journal service, and most would like a backfile of 5-10 years. Responses to the user survey varied from 1 to 50 years, with Science users averaging 7 years, and Social Science users averaging 11 years.
The user's discipline is one factor in how important the backfile is, but also important is the task at hand and the particular journals.
The presence of backfile is likely to influence how a user perceives an electronic journal service and how s/he will use it, but is less likely to be a deciding factor in whether or not to use it.

Use of the Journals

Users in the Social Sciences made wider use of the journals than users in the Sciences. Averaged over all journal users, the majority (82%) of Social Science users used 1-6 journals, where the majority of Science users used 1-3 journals.
Frequent users use more journals than infrequent users. For the high repeat user groups, CCS users used an average of 10 journals, PS users used 8, and MGP users used 4, and Materials Chemistry users used 4.
A follow-up survey of users indicates that the majority used journals in SuperJournal they had never read before (58%) and journals they couldn't get in the library (65%). Particularly in the Social Sciences, availability leads to use. The survey indicates that over 90% used journals they had not read before.
Use of journal issues in the current year was greater in the Sciences than in the Social Sciences.

Use of Articles

The proportion of articles viewed was greater in the Social Sciences than the Sciences. For the CCS cluster 75% of the articles were viewed at least once, for PS 64%, and MGP 39%. For MC only 13% of the articles were viewed at least once, but there was a substantially greater number of articles, and the cluster was available for the shortest time.
The proportion of articles viewed varied from journal to journal, and but all the clusters contained journals where all or virtually all of the articles were viewed at least once.
We found little evidence of highly popular articles being viewed by many users. Typically articles were viewed 1-3 times, with a higher proportion of articles viewed once in the Sciences than Social Sciences.

Use of Abstracts

Users value abstracts. Most users (72%) said they would like publishers to publish abstracts in both print and electronic versions of their journals. Five of the Social Science journals included in SuperJournal did not contain abstracts, and another four did not have abstracts for all articles.
The Science users viewed abstracts less frequently than the Social Science users. Science users more often browsed at Table of Contents level, or went direct from the TOC to the article itself. Typically users in the Sciences were using SuperJournal to get specific articles and to keep up to date with specific journals.
The Social Science users viewed abstracts more frequently. Typically users in the Social Sciences were using SuperJournal to keep up with what was being published generally, or to find some recent articles in their subject area, and checking the reading the abstract was a logical step. Also, site variations in use of abstracts suggest that the user's facilities may be a factor. If the full article will be slow to load in PDF format, it's useful to check the abstract for relevance first.

Multimedia Content

Focus groups at the start of the project indicated that readers had moderate interest in "multimedia", ie including additional content in the electronic version of an article.
Some saw this as a way to overcome the page or cost restrictions of print, eg to add more photos, or to add expanded sections on experimental methods.
Some saw this as a way to verify the author's work, eg by appending the author's source data. However, as authors themselves, they had reservations about releasing their own source data, as it might decrease the "mileage" they could get from it, and it might make it vulnerable to exploitation by others.
Others saw multimedia as a way to include visual examples of what was described in the text, and most often cited the potential for including film clips (Social Sciences) and 3D graphics (Sciences).
A questionnaire was sent to authors of articles in the SuperJournal journals, and questions on multimedia were included. The majority (61%) used multimedia in connection with their work. Social Scientists indicated that video, supporting data sets, and film were most relevant, and Scientists indicated that colour images and 3D graphics were most relevant. However, only 36% said that the ability to include multimedia content in their journal articles would be an advantage.

4. The Service

Patterns of uptake

During a period of 23 months of availability, 2,867 users registered to use SuperJournal at the 13 test sites. The patterns of uptake varied per site and per cluster, but overall it took 10 months for SuperJournal to achieve a population of repeat users for each cluster, which it sustained for the remainder of the project.
The CCS cluster was launched first and took 12 months to achieve a stable population of repeat users (22%), MGP was launched next and took 6 months (46%), PS took 8 months (25%), and MC took 5 months (25%).
Academic staff were the "early adopters", typically the first to use SuperJournal, followed by postgraduates and researchers, with undergraduates becoming users last.

Awareness and Promotion

Most users found out about SuperJournal via the library. The librarians used various techniques to make users aware of SuperJournal. Typically these included techniques for general awareness (eg posters and user guides in the library, articles in newsletters), techniques to indicate what journals were available (eg library Web site, OPAC, notices on shelves), and promotion targeted a specific individuals who might be interested (eg letters and emails). It's difficult to say that one technique is "better" than another, as most were used at each site, each site is individual, and factors other than promotion and awareness are involved in uptake.
Surveys at the end of the project suggest that listing electronic journals on the library Web site is the most successful technique overall. 24.5% of repeat users and 37.1% of non-repeat users found out about SuperJournal in this way.
Targeted promotion is also effective. Letters/emails from the librarian was the most common way that repeat users found out about SuperJournal (29.4%), suggesting the librarians know their audience and can reach it. Not surprisingly only 14.3% of non-repeat users found out about SuperJournal by emails/letters.
The techniques for general awareness proved moderately successful, with about 10% of users finding out about SuperJournal from posters, and another 10% from brochures. However, virtually none learned of SuperJournal from the library catalogue (perhaps because it's book oriented) or from training courses (perhaps because much else is covered).
Different techniques proved successful at different sites. The library system is fragmented at Cambridge, with many Departmental libraries. The librarian at Cambridge focused promotion and awareness on the Departmental librarians more than on end users. Departmental librarians registered, and spread the word to their users who then registered.
Some librarians made information about username/password available on "private" pages of their Web site, and others gave a contact name to call/visit. Success in this area is difficult to assess, but some of the librarians felt they had greater success when barriers to knowing the username/password was removed.
Interestingly, library demonstrations featuring electronic journals generally, were quite successful in increasing registrations (eg at Ulster, NIMR, and DMU). These may have increased awareness of electronic journals generally among the user population that was not aware of them previously.

Barriers to use

The most important factor in whether a user will use an electronic journal service is the number of relevant journals. A survey of non-users indicated that this was the main reason they didn't use SuperJournal.
The second greatest barrier focuses on access, eg ability to access, slow access, problems with passwords, or other technical problems. Non-users who gave access as a reason for not using SuperJournal typically encountered a problem and didn't use it again.
A number of other reasons for non-use were also given, some more related to lifestyle and culture than features of an electronic journal service. For example, some non-users said they didn't have time to use SuperJournal, forgot to use it, used other services, or preferred to visit the library.

Time of Use

Users value the ability to use electronic journals at any time of the day or week, but in practice tended to use SuperJournal during "working hours".
During working hours (9 AM – 6 PM), the hours of greatest use were 2-5 PM, with 2-3 PM the hour of greatest use. The hour of least use was 9-10 AM.
During the working week (Monday – Friday), usage was fairly well distributed over the days, with Wednesday the day of greatest use and Friday the day of least use.
A significant amount of usage (21%) took place outside working hours, most during the evening, some at weekends, and the least in the early mornings.
Sessions did not vary much in length throughout the day and week (average 11 minutes), but sessions in the early morning were distinctly longer (19 minutes).

Location of Use

Users value the concept of "desktop access" and the ability to access electronic journals from any location. The convenience of getting journals without leaving your desk (wherever that might be) was considered to be the most important benefit of electronic journals.
Most use of SuperJournal was on-campus use. Though in many cases we could not determine the user's location, we estimate that on-campus use outweighed off-campus use by 10:1.
The largest amount of on-campus access took place in departments, followed by cached areas (eg computer clusters), and the library. The largest amount of off-campus use took place in colleges and homes.
Users have mixed views about using electronic journals from home. Some would like to use from home, as it's quiet. Some prefer not to "take work home". Students seem most interested, and staff less so.
Off-campus use varied per site. This does not seem related to the universities' facilities for offsite use. The sites with the most and least offsite use all have dial-in services.

Seasonal Use

The use of SuperJournal followed the pattern of university life. There was relatively little use during vacations and a lot of use during term time.
Decreased use during vacations resulted more from loss of users (natural turnover) than decrease in frequency of use among individual users. The degree of seasonal change was therefore more dramatic for students than for staff.
Growth in the number of users and number of sessions was fastest in May, October, and November, with October the month of fastest growth. July, August, and December were the months in which users and sessions dropped, with greatest decrease in August.
Seasonal variation was greatest among undergraduates, followed by postgraduates. Use by academic and research staff decreased slightly during summer vacations, but was less marked and for a shorter time.

User support

A survey of repeat users indicates that most (over 90%) would like user support by email. Most felt that support within "working hours" (9 AM to 6 PM) was sufficient, and expected a response within 24 hours.
The most common user support question asked by SuperJournal users was to be reminded of their Personal ID for logging in. This accounted for over 90% of emails received by Manchester Computing.
The next most common user support question related to error messages that the user did not understand, for example when the Manchester server was down or when it could not accept any more users at peak times.
The next most common user support issue focused on Adobe Acrobat Reader: how to get it, install it, and get it to work properly.
The feedback feature allowed users to send feedback messages to SuperJournal. About half related to specific problems with using SuperJournal (eg searching), and about half were general comments about the service (good and bad), suggestions for enhancements, and questions about journal coverage (eg when will the journal I want appear).

User Authentication

SuperJournal required all users to register, so that usage data could be logged, and to ensure that the service was restricted to authorized users at the test sites. Each site chose a username and password, gave them to potential users, they registered them, and subsequently logged in to SuperJournal using their email address and a personal ID they chose during registration. Users could later change their email address and personal ID if they wanted to.
Overall, the system worked well and few problems were encountered. As noted above, the most common problem from the user's point of view was remembering their personal ID once they had registered.
Some users were confused by having a username/password that was only used once for registration, and didn't understand that the email/personal ID should be used for logging in.
The follow-up studies with users and non-user indicate that user authentication is an important issues, and users would much prefer to be "recognized" by the system than to have to remember multiple IDs or passwords for different systems.

5. Features

Core vs "Nice to Have"

Users perceive browsing, searching, and printing as the core features of electronic journal services.
Features that allow the user to customize the service in some way, eg alerting, saving references, building a virtual filebox of articles, and annotating articles, are considered "nice to have".

Users consider links from bibliographic references at the end of the article to abstracts or the article itself to be "nice to have". They put a higher priority on getting to a relevant article quickly and easily than on getting from the article to somewhere else. However, they did express particular interest in links from the article to the author's Web site and/or the author's previous articles.

Browsing

Browsing was the most common method of using SuperJournal. Browsing was used by nearly all users in most sessions. The SuperJournal application was designed to make browsing easy. If it had been designed differently (eg with a search engine on the opening screen), the results might have been different.
Users valued the easy-to-use feature of browsing by cluster.

Searching

The ability to search for articles was valued by most SuperJournal users, in particular those in the Social Sciences. Users valued searching titles and abstracts more than full text searching of articles.
Actual use of the search engines was low. They were used by one third of users in a small proportion of the user sessions. Typically users rely on the large bibliographic databases for searching, especially in the Sciences.
Social Sciences users used the search engines more than Science users, students more than staff, and frequent users more than infrequent user.
Of the three search engines available to users, Isite was most frequently used, followed by NetAnswer, and RetrievalWare was not used very much. (These results are consistent with how long each search engine was available and how easy it is to use.)
Users tended to do simple searches. Isite users typically specified one or two search terms, and NetAnswer users one term.

Other special features

There seems to be little interest in discussion forums or communicating with other users. The Whiteboard feature was intended to serve as a discussion forum for users, to comment on articles and discuss issues related to their disciplines. The feature was not used much, but users of the MGP cluster used it to post questions to SuperJournal about journal coverage and timeliness. Those who did use to communicate with other users in their disciplines rarely got a response.

6. Timing

Timeliness and immediacy

Timeliness is most important in the disciplines that move quickly, for example in the life sciences. Science users ranked timeliness as the most important factor for electronic journals, where Social Science users ranked timeliness second (with range of journals first).

Timing of electronic publication

Users would like for electronic publication to coincide with print, or continuous publication of electronic, followed up with publication of the printed issue.
Users value the journal issue and the signposting associated with publication on a particular date. Notification that the issue has been published helps them to organize their time and prompts them to visit the journal.

7. Presentation

Viewing Onscreen

Users generally don't like to read articles onscreen. Most prefer to scan the article onscreen, print it off if it's relevant, and read the article in hardcopy form.
Users value the following when viewing articles onscreen: the article loads quickly, is easy to scan, and has good presentation. While they like HTML for Web applications, they commented that HTML doesn't indicate page numbers, and that presentation of tables and equations is often poor quality.
Users who were asked said they preferred viewing articles in HTML format over PDF, but usage statistics indicate that articles available in both formats were viewed more frequently in PDF format.
Users value HTML because it loads quickly and is easy to scan onscreen, but prefer the quality presentation of the PDF format overall.

Printing

Most users print out articles. They want a copy of the article to "own" and refer back to. Many noted that annotating the article as they read helped them to understand it.
Users value the following when printing: high quality presentation, speed, similarity to the printed page, the printout should contain all the information in the article (HTML may lack page numbers). For users in the Life Sciences, high quality graphics are also very important.
Users who were asked said they preferred printouts as PDF, with virtually none wanting HTML printouts. Users in the Life Sciences were particularly keen on PDF, as a PDF printout is better than a photocopy, especially for high-resolution graphics.
Problems with Adobe Acrobat Reader were the most frequent technical problem experienced by users, and some said this was a reason for not using SuperJournal. Typically users had problems getting/installing Acrobat, or had low specification PCs that made viewing or printing articles very slow.
Journals in the Life Sciences contain high resolution graphics, often in colour, and users need to be able to print them.
Some users said they would prefer to have a "virtual filebox" of their electronic articles to replace their file cabinet of printouts and photocopies, and think this would reduce the need for printing.

This web site is maintained by epub@manchester.ac.uk
Last modified: April 26, 1999