Technical Foundations: relevant content on this site

Technical Standards for the JISC IE (part 2)

Paul Walk — Tue, 27 Apr 2010 09:05:00 +0000

Which standards are relevant to the JISC IE today? The JISC IE Technical Standards document has not been updated for some years. If this sort of document is considered to be useful still, then it needs to be brought up to date. The rest of this post will consider the standards indicated in the original document and give suggestions for what might be added, changed or deprecated from this list. Comments are very welcome on this. For the sake of clarity, 'original document' means the Technical Standards document in its current revision which was last edited 16/05/2006. The standards listed in the original document were structured into functional grouping as below. We might not use exactly these groupings or names in future, but they seem to me to be sufficiently handy to use here: Web standards and file formats Standards listed:

HTTP 1.1,WAI,HTML/XHTML,CSS,DOM,URI,IMS Content Packaging Specification,METS

One of the easier groups perhaps, with standards such as HTTP 1.1 being so ubiquitous that it would be idle to argue against their importance to the JISC IE. The clear statement invoking 'must' and 'should' around the different levels of WAI compliance would seem to continue to be appropriate. The area which might need revision is the recommendation to use IMS Content Packaging or METS for packaging content into re-deployable objects. It could be argued that this sort of packaging, once seen as a clear path towards encouraging the re-use of content, is not perhaps such a priority in an environment which has begun to favour access to atomic content artefacts with a view to remixing at the point of need. Questions:

To what extent has aggregation superseded packaging?
Should standards such as Atom be referenced here?
Is it sensible to create a new section combining packaging and aggregation as related approaches

Distributed searching Standards listed:

Z39.50,SRW/SRU,Bath Profile,UK LOM Core,IMS Digital Repositories Specification,Dublin Core

Distributed searching, as an approach, has been somewhat deprecated in general terms in favour of the harvest and index approach used by search engines such as Google. However, in the more library-oriented parts of the JISC IE protocols such as Z39.50 are still widely used today. Attempts over the years to replace this with more 'modern' technologies have met with limited success although SRW and SRU have had some take-up. Similarly SOAP which underpins SRW is, in some quarters, being deprecated in favour of a more RESTful approach such as that offered by SRU. Questions:

should we be advocating the continued development of distributed searching?
is the recommendation: "all JISC IE content providers should support either a distributed search interface or a metadata harvesting interface." still good?

Metadata Harvesting News and alerting Context-sensitive linking Transactional services Authentication and authorisation Metadata usage guidelines NB. Standards = protocols, standards, specifications, application profiles etc. The IE might identify core standards, e.g.:

HTTP
XML
RDF
UTF-8

Important, generic standards:

RSS/Atom
Dublin Core

Important, domain-specific standards:

Bibliographic standards
DOI

Standards being developed within the IE, e.g.:

SWORD
SWAP etc.

Technical Standards for the JISC IE (part 1)

Paul Walk — Tue, 27 Apr 2010 08:46:00 +0000

One of the key conclusions emerging from our ongoing consultation with some of those who have been involved with the JISC Information Environment (JISC IE) since its early days is that the emphasis on interoperability through open standards was one of the key drivers which gave the programme direction and momentum. Giving focus to this emphasis on open standards was a web document, JISC Information Environment Technical Standards, which introduced itself thus:

This document provides a list of the key standards and protocols that make up the JISC IE technical architecture. This document is intended primarily for developers, in order to provide them with a single point of reference to the main technologies that they should be using when working in the context of the JISC IE.

These standards are intended to apply to all JISC IE service components listed in the JISC IE Glossary (portals, brokers, aggregators, content providers, subject gateways, authentication/authorisation services, service registries, user-preferences services, OpenURL resolvers, institutional profile services, metadata schema registries, terminology services or other shared infrastructure services).

It has been suggested by more than one of those with whom we have consulted that this document was the most important of the several documents developed by UKOLN (primarily by Andy Powell) to technically inform what was then designated the JISC Information Environment Architecture. It gave those developing services in the JISC IE a touchstone, allowing them to validate that their work was in accordance with one of its over-arching principles. During our consultations, we have heard more than once that this document was more important than the perhaps more widely recognisable Technical Architecture diagram.

The document borrows the Internet Engineering Task Force (IETF) convention of using the words must, should and may (in a bold typeface) in a particular way to convey a more precise indication of the strength of recommendation or requirement being articulated. Nevertheless it has, for some, been unclear whether or not this document was intended to mandate or to advise on the use of technical standards in the JISC IE. Although the IETF convention was not applied to the document’s introduction, it seems reasonable to take the line that when the author said:

This document is intended primarily for developers, in order to provide them with a single point of reference to the main technologies that they should be using when working in the context of the JISC IE.

we might reasonably interpret that use of the word ‘should’ in the IETF sense, to mean:

should – indicates that there may exist valid reasons not to treat this point of guidance as an absolute requirement, but the full implications should be understood and the case carefully weighed before it is disregarded

So, it is reasonable to argue that the JISC IE was, at a technical level, based on an identification of appropriate technical standards. We will assume that the provision & maintenance of such a document is still useful which means that, in looking forward to the future, two questions present themselves:

Should the document be ‘prescriptive’ or ‘descriptive’?
Which standards are relevant to the JISC IE today?

For this post, we’ll address the first of these questions – a second post, which will appear in a few days, will address the other.

Prescriptive or descriptive?

Should our ongoing identification and documentation of the next iteration of what we are now calling the JISC IE Technical Foundations take a prescriptive (‘must’) or descriptive (‘may’) approach to its treatment of technical standards?

It has been suggested that the original, somewhat prescriptive approach had the effect of embedding the belief of the importance of shared, open standards for interoperability into the culture of those developing services for the JISC IE. But it has also been suggested that this cultural acceptance has now been achieved and that developers can be trusted to assume the need for interoperability and, consequently, be given freedom to innovate where appropriate.

Is the importance of open standards now so widely accepted that we can assume that developers will make sensible choices, balancing the need for interoperability with the desire to innovate?

Provision, fusion, presentation and shared infrastructure

Paul Walk — Tue, 01 Dec 2009 10:56:00 +0000

The JISC IE introduced a characterisation of service types which categorised them into one of provision, fusion, presentation or shared infrastructure. It should be stressed that this characterisation was not meant to be strict so much as it was intended to be a device to aid high level thinking around the problem space with which the IE is concerned. The fact that the iconic diagram (above) has become so closely associated with the IE is a testament to the appeal of this approach. The architecture which this categorisation scheme implies is informed by the contemporary interest in the Service Oriented Architecture (SOA) as an approach to sector-wide service provision and usage, and underpins the expectation of an environment evolving out of strategic investment, machine-level interoperability through open standards, and a separation of concerns.

Largely unanticipated only a few years ago, the effect of the Web 2.0 phenomenon on how users interact with services and on how services inter-operate has been profound. The diagram above implies a flow of information towards the user: the reality today is that users expect to interact with services with richer interfaces than was the case 3-4 years ago. The idea of a distinct layer of presentation services is something which is surely challenged today, as is the positioning of provision services, somewhat remote from the user, with layers of intervening services.

Having said that, while the diagram with its divisions of provision, fusion or presentation services might appear now to be an over-simplification of how services in the IE could fit together, it was in truth never intended to be a blueprint or even an architecture.The introduction to the IE of some of the fundamentals of modern systems design, such as the separation of concerns and modularity of services encouraged by the SOA has been valuable. However, it could be (and has been) argued that a Resource-Oriented-Architecture (ROA) is a better fit for the IE. We will examine the relevance of ROA to the JISC IE in detail in a subsequent post. In the meantime, we should take the opportunity to review the impact and continuing relevance of this approach.

Questions:

Is the characterisation of provision, fusion or presentation still useful? If not, is there a better categorisation we might adopt, or is this whole approach no longer useful?
How has the emergence of simple point-to-point services on the Web affected this picture?
Rather than a focus on services, there would seem to be an emerging emphasis on users and data. How should this inform the evolution of the JISC IE’s technical foundations?

Further Reading on this topic:

Mapping the JISC IE service landscape, Andy Powell, Ariadne, Issue 36

The JISC IE: Some lessons from Web 2.0 , Paul Walk, presentation to the JISC IE Working Group

Lost in the JISC Information Environment, Tony Ross, Ariadne, Issue 56

Lost in the JISC Information Environment? a post by Andy Powell on the eFoundations blog

All models are wrong, but some are useful, a post (+ associated comments) on Paul Walk’s blog

Reviewing the technical foundations of the JISC Information Environment

Paul Walk — Tue, 01 Dec 2009 10:54:00 +0000

The JISC Information Environment (IE) is one of of JISC’s well established strategic ‘themes’. At a technical level, the IE is framed by some important documents: a ‘technical architecture’ and by a set of technical standards, both previously developed at UKOLN.

Since this technical framework was created there have been significant changes in the wider technical environment such as the growth in Web 2.0 applications, cloud computing and the use of third party services. The time has come for a comprehensive technical review of these technical foundations which were established more than three years ago. Both the content of these foundation documents, and the approach of using such documents to frame something as broad and complex as the IE, need to be reviewed.

A great deal of technical development has taken place under the umbrella of IE Programmes in the last three years. Some of this has been pure R&D, while some has led to deployed and supported services. It is time to review the extent to which the technical architecture and standards of the IE have remained relevant to these development.

Funded by the JISC, UKOLN is undertaking a technical review of the IE. Towards this end, we will be carrying out a process of consultation. This consultation will take the following form:

A series of initial interviews with identified stakeholders in order to establish a broad sense of those issues in the IE which are relevant to a technical review. These interviews will be in the nature of an ‘intelligence gathering’ exercise – they will be neither transcribed nor quoted from.
A series of posts on this blog, introducing issues, themes and questions as part of the ongoing review. These will driven by the issues being raised in the initial interviews (in addition to others already identified) and will be posted with a view to generating discussion (either via direct comments on this blog, or elsewhere) from the community as a whole.
As issues are raised, posted here and discussed, materials will be synthesised form these to inform a workshop to be held in February 2010. This workshop will be offered by invitation, and will be focussed around major themes or issues emerging from the consultation to that point.
This blog will be continued indefinitely, as a component in a planned process of ‘continuous review’, intended to ensure that the IE remains current and relevant.
The review will be formerly written up and reported in 2010.

In order to integrate into this discussion any comments made elsewhere, please use the tag:

jiscietech

If you would like to comment on this review, or on the technical foundations of the IE in general, please feel free to do so – we are happy to receive any and all relevant comments.

We will be posting further, more specific discussion pieces on this blog, so if you are interested in being part of the discussion then please subscribe to the RSS feeds:

RSS feed of posts
RSS feed of comments

Some starter questions:

We intend to review the following technical documents:
Are these documents useful? Is it worth developing these further?
Is the IE’s technical focus on interoperability through open standards still relevant? Is it enough?
Does the IE require an architecture?
How has Web 2.0 changed the wider context for the IE?
There has been significant technological change and development since the IE Technical Architecture was published (~2005). What implications do you think this has for any attempt to update the technical standards list and the IE architecture?