Home | Search | Demo | News | Feedback | Members Only
The purpose of the SuperJournal project is to identify the features and functionality of electronic journals that have value to the readers and authors of scientific journals in Universities thereby assisting in the design/creation of successful electronic journals in the future. There are many factors which might contribute to this value, for example, in the delivery `I can access it from my desk'; in the functionality `I can search through the full-text and then view the protein structures in 3-D' or in the content `All the journals I want to look at in my discipline are here'.
On the obverse side there are many known barriers to the use of electronic journals. Some are the reverse of the benefits; e.g. not enough material relevant to me available, not enough `added value' features to distinguish it from paper, etc. Some are additional issues. e.g. difficult to read full text on a screen, takes too long to download, etc.
In the SuperJournal project we shall aim to deliver as full a range of valued properties as possible in each journal cluster and to minimise known barriers. We will systematically vary some features of electronic journal delivery to test what is required, for example, in different subject domains. A number of user sites have been selected in order to see if the activities that the reader/author has to do is affected by the way that the sites choose to provide the information. At the functionality level, the project will provide a range of search engines and other functionality, and do so in both science and the social sciences to see if there is a difference.
At the end of the project we aim to be able to tell others what key features should be provided in an electronic journal and how they should be delivered. It is unlikely that, at any stage, we will be able to deliver all that is of value or remove all the barriers to use. The aim of the project is therefore to offer a sufficient critical mass in each domain to make apparent to authors and readers what is possible and to explore with them what would characterise the most valued service. In this way we aim to create the knowledge which interested parties need in order to plan their future contributions to electronic journals; for publishers to identify the USPs (unique selling points) for their future journals and for libraries to establish the rôle of electronic journals in the delivery of scientific material within their institutions.
Within the SuperJournal project there will be a number of phases when different clusters of journals will be delivered to different user groups within Universities and different groups of authors will be involved in the development of electronic journal papers. The successive delivery of different journal clusters provides an opportunity to change the journal content and means of delivery to explore what factors facilitate usage.
Within this framework the evaluation strategy may be characterised as longitudinal and interpretive action research:-
The evaluation process will serve to provide progressively fuller answers to the following eight questions which relate to the positive and negative consequences of electronic journals for four key stakeholders.
The Authors Authors need to perceive value in delivering electronic versions of their papers. Therefore:
The Readers Readers who must see additional value in electronic journals if they are to use them instead of/as well as paper based versions.
The Publishers As the stakeholders who have traditionally provided the link between authors and readers, the publishers must see commercial benefit in what authors and readers value in the provision of electronic journals.
The Libraries As the traditional local mediator of journals provision to readers, the Libraries are a significant stakeholder in the widespread adoption of electronic journals.
Each of these questions relates to the facilitators and inhibitors of electronic journal usage and a fundamental feature of the evaluation strategy is that it will be driven by a model of the factors that encourage/discourage usage of electronic journals. An outline framework for the facilitator/inhibitor model is given in table 1. It contains illustrative factors but is not a comprehensive listing.
Potential Value Added Features in Electronic Journals
Barriers to Usage
|Individual Journals||Multi-media content
Content better than print
Value added indexing
|Non-availability of full text
New Software to learn for Multimedia elements
No back file
Scope, Critical Mass
Functionality for search
Linking Between Journals
Understanding scope of content
|Local to University/ Reader||Physical availability
An Initial Framework for a Facilitator/Inhibitor Model with Illustrative Content
The model will be developed and used as follows:-
As a result of reviewing existing literature on the usage of electronic journals (SuperJournal report on The Usage of Electronic Journals) and of focus group discussions with publishers, a provisional list of value added features and functionality has been drawn up. It is being used to prepare the delivery system for the first cluster of journals.
While the evaluation strategy seeks to establish the facilitators and inhibitors for direct users of electronic publications, i.e. authors and readers, the knowledge that is generated will be of relevance to the range of other stakeholders who are part of the delivery chain for electronic publications, i.e. publishers, libraries, software vendors, developers, editors, etc. The data generated from journal cluster delivery will help these stakeholders develop a fuller understanding of the issues as viewed from their perspective. The aim of the project will be to systematically present this data to stakeholders and help them review the implications through the following process:
In this section the sequence of evaluation activities throughout the project will be described. The process is primarily a formative evaluation in that each stage of evaluation influences the next delivery but it is ultimately summative, providing a statement of our acquired knowledge on what is necessary to deliver electronic journals of value to different communities of authors and readers.
SuperJournal will develop by successively offering clusters of journals of interest to different communities of authors and readers. The evaluation process will have a cluster orientation; initially attempting to model the value added features to be provided in each cluster and to collect baseline data about the community of authors and readers who will be able to access the electronic journals. After delivery usage data about each cluster will be collected and followed up and a report prepared on the facilitators/inhibitors experienced by that community and the value added features of particular relevance to them. This process will be repeated for each cluster.
The participating libraries will be visited and their librarians interviewed. The aim of these visits is to learn more about the libraries' collections, organisation, management, resources and current involvement in electronic library activities. A particular interest will be the match between the libraries' holdings, in terms of the SJ journal clusters, and the degree of reader interest in those titles. At the end of this exercise the SJ team should be able to state which libraries have the best likelihood of providing access to readers of the SJ titles.
It is hoped that at least three libraries will have communities of users interested in the each journal cluster to be made available.
When the initial journal cluster libraries have been identified, likely readers will be contacted. These will be members of academic staff and post-graduate students. The objectives of this initial exercise will be to:
Readers will be identified by way of subject specialist library staff, departmental library representatives, inter-library loan request records and self report questionnaires attached to the current issues of the SJ titles.
While E-mail questionnaires will be used to gain the above information emphasis will be placed on direct contact (given the low response rates that are widely reported for E-mail based methods). Smaller numbers of targeted users will be invited to join focus groups.
Author participation in the development of electronic journals is critical to their success. Prior to the release of each journal cluster, UK university based authors will be identified from backcopies of each of the print-based journals. Questionnaire studies will be undertaken to establish the attitudes of these authors towards authoring electronic journal papers. Authors at institutions which are particularly well represented in terms of publications will be invited to attend locally held focus groups in order to confirm the results of the questionnaires.
The base line assessments of libraries will comprise interviews with the librarians undertaken during the initial visits to the participating libraries. See section 3.2.1.
A postal questionnaire will be administered to the journal publishers which will record publishers expectations concerning the impact of multi-media and the submission of electronic manuscripts by authors on the publishing process.
Prototype SJ interfaces will be evaluated initially using expert critiques and user walk-throughs by staff with a wide experience of interface design issues.
Interfaces that are made available to library users will be evaluated via self report questionnaires, structured interviews and group evaluations.
It is assumed that the participating libraries will take responsibility for implementing SJ in accordance with their usual procedure. The SJ team will provide support as required and encourage maximum publicity with respect to potential readers. This may involve sending out specific publicity material via E-mail and internal mail and invitations to on-line demonstrations. The evaluation team will not have a major role in the promotion of the system given their role as evaluators but will record the form taken by the implementation process in each institution. A key aspect of the implementation will be the confirmation of potential users.
The overall evaluation programme for each journal cluster is summarised in table 2. The timescales all derive from the dates when clusters of journals are made available. Clusters which are delivered early in the project can be finally evaluated after 18 months. Clusters which are delivered in the final 18 months will be reviewed 3 months from the end of the project.
At the end of the project the evaluation will have yielded considered assessments of the provision of five journal clusters plus interpretive commentaries by relevant stakeholders. We would have a view of the value added factors appropriate in subject domains and the barriers relevant in each case. This will provide the basis for a final summative evaluation report which will present an overall model of the factors influencing electronic journals with a contingency framework which explains the variations found for different subject domains, institutions, etc. The report will also include an analysis of the future of electronic journals from the perspectives of different stakeholders, i.e. readers, authors, publishers, librarians, etc.