PURE: relevant content on this site

Why should universities care about identifiers?

Talat Chaudhri — Fri, 17 Aug 2012 15:19:53 +0000

Why do identifiers matter for research?

Imagine that you are a senior manager in an institution within the UK Higher Education sector with responsibilities for research: you have read some basic details about unique researcher identifiers and perhaps institutional identifiers. However, it may not be immediately apparent just how important these issues are, which may seem on the face of it to be a relatively superficial and/or trivial organisational matter. Clearly, any such strategic decision-maker will long have been aware of the demands of the Research Excellence Framework (REF) and its predecessor the Research Assessment Exercise (RAE), in which successful reporting of the best research outputs of university departments is crucial to the on-going funding of the institution. This is particularly central to the work of research-led universities, which is an increasingly competitive sector: even universities that formerly focussed more on teaching than research are increasingly aware of the need to drive up standards of quality research in order to secure additional funding.

The reality of unique identification in research

However, as anyone who has actually engaged with the business of research reporting to any degree will tell you, it is far from a superficial or trivial matter to carry out such an exercise without thinking very carefully about how researchers are identified; moreover, identifying the research groups, departments, projects and institutions that they may have variously belonged to at different times, all of which may have been re-organised on many occasions, is a considerable challenge raising considerable technical as well as organisational issues.

Perhaps the biggest problem of all derives from the scale of research reporting. On such a massive scale, it has to be done in a systematic way across higher education institutions in order to be useful. Any lack of a systematic approach in collecting the information on the institutional level will inevitably result in higher costs in processing the information later into a useful form, for example by governmental organisations such as HESA and the Research Councils (RCUK) relevant to each area of academic study. This may be carried out for a variety of reasons, amongst them for example:

The need to produce statistics at a national and at an institutional level in order to gauge how successful different parts of the research community are performing in comparison to each other and to similar institutions internationally, which may be a determinant of how funding is allocated.
The production of good, widely accessible information about the work of academic researchers and research groups for the purposes of future research, both in identifying research as a basis for future work and for guiding individuals and groups in terms of who they might work with in future, who their competitors may be, and in creating wider bibliographic information for a whole range of related purposes related to future publications.
Open Access, an increasing requirement imposed by funders where research is publicly funded.
Accountability in the use of public funds for research.

It is precisely the lack of a national approach to providing consistent metadata about individuals and groups connected with research that raises costs, creates inefficiencies and frustrates the development of new software functionality that makes the jobs of research managers more difficult and ultimately reduces the funds available to research and their best use within the sector. It is therefore the business of senior managers of academic research to care about identifiers.

Researcher identifiers: a crucial first step

Before any wider metadata about research may be considered, the most fundamental issue is identifying individuals who carry out research. Before this happens consistently on a national level, there is little point addressing the subsequent issue of identifying groups and institutions engaged in research consistently. It is also important to consider any national approach in terms of interoperability with other international approaches wherever possible: while, on the one hand, funders and statistics agencies can only hope to mandate national identifier schemes, at the same time it is clear that research collaboration is cross-institutional and international in scope, in some cases including researchers from numerous countries in one project or even in the production of one individual paper, data set or other research activity. This is the approach that has been taken by the JISC, together with RCUK, HESA and other partners in setting up the Research Identifiers Task and Finish Group, which is due to report in October 2012.

One emerging candidate with cross-sector and international support is the ORCID researcher identifier scheme, whose rapid development in 2011-12 is scheduled to culminate in a public launch in October 2012. There are, of course, existing, widely-used but relatively simple identifiers such as the HESA researcher identifier, and identifiers provided through commercial providers' web interfaces, but thus far these have not provided dependable unique identification. All such identifiers could be linked to a system like ORCID that is designed on interoperable principles and is not dependant on any particular software platform or web interface. An alternative approach is taken by the ISNI number: whereas ORCID seeks to offer individual researchers and institutions the ability to manage their data on a distributed model, ISNI represents a centrally moderated, bibliographic approach led by national libraries and other similar institutions with national and strategic responsibilities. It remains to be seen whether these different approaches are in competition or whether they will offer different but complementary functionality within the sector, and much may be dependent on how software vendors implement them.

Current Research Information Systems (CRIS)

It is not simply a matter of tracking publications and other related ouputs, for example in institutional repositories. This part of the equation is by now relatively well established in the UK HE sector, although it continues to develop: the issues surrounding Open Access, for example, have not been fully resolved. This, however, is just at the level of the final outputs of research and does not provide anything like sufficient insight into the processes of research, the projects and groups carrying out, the staff involved or the costs. Traditionally, this information has been gathered in a very long-winded process that is individual to each institution's particular workflows and processes (although there are obviously great similarities of approach between them), often a partly paper-based exercise that has been migrated to an extremely varied range of systems and databases, few of which are interoperable or complete. Many departments may be involved in the process apart from the institution's research office and the department in which the researchers are based, but perhaps the most significant would be the finance office, the human resources department and the library, to name just the key players. It will be necessary to keep some information confidential, e.g. personal staff information, salaries and so forth, to share some information internally and with research funders, and to publish other information, e.g. in a research repository that forms the institution's "shop window" of public outputs, library databases and so forth. The term Research Information Management (RIM) has emerged to cover all of these information gathering and information processing activities.

In order to do this systematically, more sophisticated research information management software has been developed, often known as Current Research Information Systems (CRIS). The market in the HE sector is currently led, in terms of the number of institutions adopting the software, by PURE, produced by ATIRA; other major players are Symplectic Elements, and CONVERIS, produced by AVEDAS. More recent entrants to this market are Thomson Reuters' Research in View. There are currently no open source products, although a JISC-funded modular approach by the Research Management and Administration Service (RMAS) project may have an increasing impact in this area, depending on subsequent adoption by HE institutions. It is not an overstatement to say that HE institutions are currently in a rush towards early adoption of these CRIS systems, motivated by the need to use research data to compete with each other for funding opportunities.

Next steps: organisational identifiers

In the next 2-3 years, it is likely that the matter of unique researcher identification will be resolved through the emergence of a dominant standard that has sufficient take-up and leverage in the UK and international HE sector to faciliate the work of research institutions and funders. Following this, there will be organisational structures associated with research that will require unique identification, often on a multi-layed basis: for example, a project may be at several institutions, perhaps internationally, and their staff may be in various departments or similar units whose names have changed or have been merged or de-merged at various times, all of which will require careful date and time stamping to make the information reliable for the period that it covers. There will be issues related to copyright, commercialisation and spin-off companies that make the precise provenance of research critical to the future success of academic research and development. Standards for organisational indentifiers are therefore the next important issue on the horizon. Like researcher identification standards, research managers and senior managers with strategic responsibility for research will need to keep abreast of this rapidly developing area.

The collaborative research environment: publications management, CRIS systems and repositories

Talat Chaudhri — Thu, 02 Apr 2009 16:41:00 +0000

Some months ago, I intended to write a post inspired by this post by Chris Rusbridge on CRIS systems. One particular motivation for this was reference he made to some comments by Stuart Lewis, particularly relating to the Symplectic Publications Management System. This was the subject of an impressive demo in Aberystwyth by Daniel Hook of Symplectic and Imperial College, London, while I was in my previous post as Repository Advisor. We were very interested by CRIS systems as back-end systems for managing research outputs at source. I believe that Stuart had been in touch with Niamh Brennan, who has since been kind enough to give me some general details about the CRIS system that has been produced in-house at Trinity College, Dublin. I should note that I have not seen it in operation.

One reason that I never published the original post is that most commentators in the repository community probably agree that a CRIS is innately a Good Thing in any event, so it needs no detailed endorsement from me. (A further concern is that I do not wish to be seen to be making specific comments on developments at Aberystwyth, which is a matter for them, so I shall confine my remarks to the systems at issue here.) In short, as Stuart’s comments cited by Chris in the above blog make clear, the CRIS does not replace the repository but “sits behind it” and provides content. There are of course a great variety of ways in which this could be achieved in practice, depending on the particular system in question. I will address the definition of a CRIS further below. However, the Symplectic system deserves more detailed commentary here, for reasons that will become clear.

Symplectic Publications Management System

There seems no reason to doubt Symplectic’s obvious competence in publications management, having witnessed how effective this piece of software had already been, albeit having only been tested in a limited number of institutions at the time. It was, to my mind, impressive that Daniel Hook was both frank and receptive about the merits and development potential of his system. Indeed he had no real need for the hard sell because the software had been built "by academics for academics", so it consequently fulfilled most of the basic requirements already and many of the other features that he was questioned about were already in the pipeline.

However, this post is by no means intended as a wholehearted eulogy directed at the Symplectic system. It was developed in the context of scientific disciplines, and had not at the time been tested in arts and humanities disciplines, although to be fair this point was made openly by Daniel Hook himself. I do not see why it should fail in principle in non-science disciplines, although the take-up may be less enthusiastic in the same way that one finds in repositories. The real problem is the limited coverage of these disciplines by union databases such as Web of Science, though this is not the fault of Symplectic. Their efforts may in time provide one reason to improve those databases, which underlie the core functionality of the system.

Briefly, the system replaces the university's reporting system. The academic logs on and merely has to choose whether to agree that a new paper in Web of Science belongs to him or her. They are able to declare that it is identical to another version, perhaps in another database, or else they can separate two papers of the same name, e.g. a conference paper and a published paper. They can optionally correct the metadata. Evidently, for papers not found in Web of Science, academics can enter the details of the papers manually, although this process then loses all of its automated advantage over author self-deposit in a repository. For this reason, the system has the greatest advantage in science subjects. The administrator can see whether academics are responding to its suggestions and thus, in disciplines with good database coverage, can gain a good idea of how comprehensively they are archiving content. Finally the system is entirely interoperable with DSpace, so that items may be deposited through either interface (I am less sure about EPrints and Fedora). Permissions can be set and delegated on a fairly granular level, unlike in repositories, so that academics can have control over who can deposit items on their behalf, such as co-authors.

The system provides useful tools to help simplify and automate research reporting that is already mandatory for academics, rather than introducing new obligations and unnecessarily duplicating the deposit/ingest phase. By contrast, repositories require this as an additional process, after research reporting. Of course, certain repositories are used for research reporting, which requires an effective or explicit institutional mandate, but in their present forms they are badly suited to describing the finer details of funding grants and projects. This is handled rather better by Symplectic, but obviously requires manual metadata input.

It may come as no surprise, given my involvement with SWAP, that I should level a criticism at the Symplectic system on the basis of its simple versioning model. Clearly one might wish to be able to say more than whether two papers are identical or not. It was clear that no complex relationships could be described at the time of the demo. I hope that this is addressed in future iterations of the software.

CRIS systems

A further issue is whether the Symplectic system qualifies as a CRIS at all, as pointed out to me by Niamh Brennan. Apparently it does not. There is no indication that it supports the CERIF standard maintained by EuroCRIS, and moreover it does not provide the means to support the creation and versioning of research outputs and related project information from their very inception, which is one of the purposes of a CRIS system. Instead, it only deals with papers after publication, in the same way as a repository.

Whether or not academics, particularly those whose working practices are long-established, would willingly switch over to using a CRIS in this way is perhaps a matter for doubt. It might be possible to compare, for example, mandated and voluntary use of CRIS systems, but at present I have no evidence for the relative feasibility or success of either approach. I hope to be able to see other CRIS systems in action, and that an Open Source platform will become available before long. I suspect that they have considerable potential in managing research and making it freely available. However, it may be naïve to simply assume that they will provide a complete solution for research management. At present, the only commercial software seems to be PURE, given the apparent demise of UniCRIS there appear to be three commercial competitors: CONVERIS, PURE and UniCRIS [updated 22/09/2009].

The functionality that these systems highlight that traditional repository platforms lack – with the proviso of course that no two in-house CRIS systems necessarily share the same functionality – is the ability to offer a collaborative environment for research management and research reporting. In the case of a CRIS, this is supposed to extend to the creation of research from its inception. As a repository manager, I recall being asked by several academics why they were unable to manage their own metadata in our repository, considering that responsibility for their own research is a core function of their employment. This is an entirely fair criticism, to which I will return.

It is difficult to see why a CRIS should not share all of the main characteristics of a repository. Both manage the ingest of bitstreams and the creation of metadata. Both repositories and the Symplectic system can be used to expose these materials to the web or alternatively set permissions to view them, and to generate publications lists for authors or research groups. Ultimately these are a set of very similar systems with slightly different but related, complementary purposes.

Implications for repositories

This discussion highlights the conceptual similarity of the current repository platforms to publication platforms, despite the usual claim that they are not publishing research but merely archiving it after publication. In legal and practical terms at least, they are self-evidently re-publishing it by making it publicly available in a further location. (One might speculate that this is the reason for the animosity of certain publishers towards repositories.) After all, to publish ultimately means to make public. But in academic usage, publishing also includes peer review. In my view, it might be a good time to start drawing a clear distinction between these functions.

I have heard Paul Walk refer many times to the “silo effect”, which seems to be at the core of the problem with the present repository platforms. Unlike other websites, the repository is not an interactive site. Having seen the statistics for repository access using Google Analytics, I can report that only a small minority of visitors access the site directly or by referral from university web pages. Virtually all papers are found using Google (only rarely other search engines) and those visitors do not remain on the site after they have found the content. They are largely oblivious to the existence of the repository. Very few users were referred by Intute or OAIster. All of these problems are to some extent unavoidable and well-known, but the repository should at least be more interactive than it is from the point of view of the depositors.

I should note that the large part of my experience with repositories relates to DSpace, though I am also familiar with EPrints. The My DSpace (user area) function offers the user no functionality other than being able to change their personal details and submit content, and even the latter usually requires manual authorisation by the administrator. EPrints is generally similar. This immediately results in confusion, erroneous error reports and so on from sometimes irate new users, but it is quite logical for the repository manager, who needs to know who is submitting content and generally wants to intervene to give basic initial copyright advice in order to save problems later with incorrect versions being supplied.

There is clearly a role for automating the setting of permissions for deposit on the basis of staff records. Though it presents problems, it can be done routinely for other university software systems (including apparently the Symplectic system). I would be interested to hear whether this has been done successfully in the majority of EPrints and DSpace repositories.

But the issue runs deeper: the only tool that the repository manager has to safeguard copyright liabilities is the ability to monitor submissions. It would be most unwise to set editorial permissions for all authors over their own metadata because there is no function to view recent changes, only recent deposits. Such changes might easily be in breach of copyright, despite the author’s best intentions. (In DSpace up to version 1.5, permissions can only be set by group, not for individual authors.) There is no easy versioning system, such as the sort used in wikis, and any changes to bitstreams can normally only be reversed by recourse to restoring from general back-ups.

It is evident to me that the present situation is not scalable. When the majority or even the plurality of research outputs are available, there will be far too much in each repository for its staff to deal with. A more scalable, collaborative solution is needed before that situation is reached, involving all parties concerned with the production of research in maintaining it on the web.

General recommendations

To summarise, the consequence is that items are “frozen” as soon as the repository manager completes checking for copyright checking and basic metadata compliance, after which any changes must be requested by email. The following features would need to be in place in order to allow repository managers to implement a more collaborative worklow:

(1) historical versioning control needs to be in place for editors to have the ability to easily roll back records and choose
which may be viewed by the public.

(2) in addition to the initial checking step in the workflow, all recent changes to the repository need to be visible to the repository manager. These could be vetted, i.e. they could remain pending until allowed by the repository manager, if desired.

(3) users need to be better identified as authors, even where another individual, e.g. co-author or administrator, has deposited on their behalf.

(4) the permissions system needs to allow granular control over what an individual may change within their own deposits, whether bitstreams or metadata, and within groups in the repository.

(5) bitstreams and links to versions elsewhere on the web should be treated equally in the interface, since the user is concerned with getting the resource, not where it is.

(6) as in the Symplectic system, users should be allowed to delegate those rights to individuals within their workgroup.

(7) as in Symplectic, users should have control over which publications appear in their publications list, and in what order.

It would also be desirable for repositories to use a method similar to that used by Symplectic to compare records to new items appearing in Web of Science. In that way, authors would have an ongoing dialogue with the repository, and an impetus to use it as a collaborative tool. In effect, there should be no conceptual difference between a publications management system and a repository – and, frankly, the former term is instantly meaningful to the user, while the latter means nothing to them at all. Such public relations disasters have led to the present stagnation in repositories. I would suggest that in any working system, the CRIS, the publications management system and the repository ideally need to be modular parts of the same software.

The repository world currently suffers from a dictatorial, top-down management structure that is in considerable part imposed by the design of the current repository platforms. At present, repositories do not meet the standards of collaborative software that we come to expect from Web 2.0 services. They also seem to be in direct conflict with the traditional responsibility of the academic and/or department for proper research reporting within the institution. The Symplectic Publications Management System and the concept of CRIS systems offer a more collaborative model that can help avoid repeating the same tasks. In addition, they can help record complex grant information, some of it confidential or commercial, that may not be for public release on the Web. Moreover, they offer an interactive workflow with real time-saving benefits for academics. In contrast, self-archiving in repositories merely repeats a task that is mandatory elsewhere, and does not represent joined-up thinking.

Incorporating the already mandatory process of research reporting into such systems would effectively by-pass the need for institutional mandates for self-archiving by incorporating it into the existing workflow using new software tools. Progress with such mandates has been crushingly slow, despite the success of the few that have been achieved. However, since all institutions are organised differently, it is simply not the case that only one single method for acquiring freely accessible content, however effective when implemented, is the only possible way to success. Collaborative research management is clearly a good idea for this end, as well as for saving time and effort on the part of academics, irrespective of the merits of institutional mandates. It would also be a more scalable, sustainable basis for repositories to work that the present situation. On that basis, I suggest that it may well be a Good Thing.