In between the mega macro level of the universe of scholarly communication and the micro level of an individual scholar's interests and projects lie a number of mediating layers of culture and context, and in these the European perspective differs somewhat from that in North America.  European governmental subsidization of higher education has a number of exceptionally positive effects on society and education, not least among them the leveling of the financial playing field for access to university study.  It also, however, creates a system of administration and oversight that transcends not just the individual department or disciplinary unit but all of the institutions in the country.  The need to distribute resources “fairly” creates the need to measure and compare outputs, not just between the sciences and the humanities, but between large and small universities, rural and urban centers, and research versus teaching-focused institutions.  There is open recognition that institutional missions differ, that disciplinary norms of quality and impact differ, and that the presence or absence of quantifiable and comparable output measures of a scholarly communications ecosystem varies widely between the disciplines. Yet at the end of the day a certain exact number of Euros or Pounds must be allocated to each player in the system, and while there is no good way of making this determination, there are lots and lots of bad ones.

Although Eugene Garfield, the father of such approaches to scientific impact assessment as citation analysis and journal impact factors, was US-based, it is not surprising that his ideas and inventions have been embraced by European government departments of education and research councils alike.   Tools such as the “H-Index” provide an elegantly simple way of saying precisely where an individual scholar sits in the hierarchy of production, placing a simple numeric value on her contribution to her field, which can easily be compared with the numeric values of her colleagues and competitors.  The problem is that outside of a few relatively restricted areas, these simple indices are not representative of any accepted definition of either the quality or the impact of the work in question.  The fact that much of the bibliometrics system is built upon citation analysis, and that citation analysis is neither supported to the same level by such database tools as Thompson Reuters ISI or Elsevier’s Scopus for all areas, nor is as relevant for all areas as for, e.g., particle physics, means that our lovely simple system of numeric indices ranking scholarly work is as dodgy a basis for funding allocation across a national system as is the political connection of one’s local minister of state or, for that matter, a dart board.

The arts and humanities have largely stayed on the margins of this debate.  In fact, when the United Kingdom decided to scrap its hugely influential and much maligned Research Assessment Exercise (RAE) in favor of a metrics-based system, a special expert group had to be convened to determine how the arts and humanities would be treated.  This group came up with a very long list of possible forms of research output that could be measured (although the report did not always define or specify how one might capture these measurements) and determined that a basket of these measures, including a “peer review lite” approach, could be chosen closer in time to the first round of the new assessment exercise in 2013.  This report was largely an admission of failure in the face of a task for which the system had no easy or accessible tools.

This is not to say that there have not been attempts to develop frameworks to enable a more systematic quality assessment of research outputs in the arts and humanities.  The European Science Foundation funded the development of a project called the European Reference Index for the Humanities.  The ERIH was intended to provide a relatively uncomplicated impact mark for the journal in which an article was published on a discipline-by-discipline basis.  In spite of the project leaders’ insistence that the A, B, and C rankings given to journals were intended to indicate their degree of internationalization and profile, not quality (whatever that is), the use of the simple letter grading became a source of great controversy and discontent with the project.  The real threat this index posed to smaller publishers and national journals, which were very appropriate places for certain kinds of research to appear, resulted in the project being largely abandoned after its initial development funding ran out.

But even if ERIH had been embraced by the community, what then?  It didn’t include the publishers, so had no capacity to aid in the evaluation of monographs, chapters, editions, or articles in edited collections.  It was a misleading, unauthoritative tool, and although the scholarly community reviled the project, there were still enough cases seen of research councils and institutions attempting to use it that these bodies’ desire for some easy way of making even an initial judgment of quality or impact was clearly perceived. 

This extended anecdote is intended to illustrate an environment that holds both opportunities and threats for the development of scholarly communications in Europe.  Institutional autonomy is low, and the pressure to contribute to the system in certain particular ways is very real.  “Is it RAE-able” was the question that always used to be asked about potential project outputs in the UK, and it is doubtful that the question will change much under the new system; only the terminology used to describe acceptability to the higher powers will shift.  On the other side, however, the existence of national systems with gatekeepers who are unsure how to evaluate scholarship most effectively also means that these gatekeepers may also be willing to work with their community to define reasonable metrics and approaches to assessment.  If the traditional analog scholarly communications formats are to be transcended, then success will be guaranteed only if this transcendence can be translated into terms comparable with those applied to the known scholarly output forms.  There have been calls for years for the creation of “kite markers” and other ways of recognizing the scholarly contribution made by non-traditional output forms.  Certainly in Europe, one has the sense that if the scholarly community took the lead in establishing norms and measures for best practice where these norms don't already exist, then their lead would be both appreciated and followed.

If the opportunities are so clear, then why are the responses so unsatisfactory, as with the case of ERIH?  It is easy to point the finger at the faults in the tried and true system of peer review as the source of difficulties.  Yes, it is a system that is difficult to manage and expensive to maintain, but the glory of Garfield’s citation analysis is that it manages to capture peer review happening, as it were, in the wild, at that point when another scholar references a piece of work as influential for his own, thereby making explicit a judgment call about the importance of another scholar’s work.  Surely the digital humanities community can find a similar juncture, more subtle than hit counts but more easily captured than full-blown reviews, because the community of scholars should be able to agree upon what constitutes a quality output, whatever its format.

Unfortunately, it appears that we cannot do this even in the analog realm. A recent workshop held in conjunction with the 2010 Modern Language Association Conference in assessing digital humanities scholarship created in non-traditional formats (thematic research collections, e-lit, e-scholarly editions, etc) suggested the opposite. Over thirty academics, from Department Chairs to graduate students, attended. The workshop was developed around a series of case studies. The first case study was an e-edition of the work of a little-known poet. While the edition was replete with contextual information, scholarly apparatus, and technical background, the first comment by a workshop participant was that this was not scholarship, it was service. The creation of a scholarly edition was a service activity, not a scholarly one regardless of the medium of presentation. The workshop facilitators immediately realized that the battle lines were far from fixed and that having a work in digital form only served to reinforce certain existing prejudices rather than allow for a widened scholarly horizon.

Given such challenges, is it possible we could focus, writ large, on developing new modes of peer review for digital production, as occurs within the NINES community? If so, can we convince institutions and funding agencies to accept the results of these modes?  The answers to both of these questions surely must be yes, but the ever-broadening focus of European digital humanities scholars has not yet come to rest on these most political and procedural issues within the ecosystem of scholarly communications and production.  The problem may be deeper entrenched than we expect, but doesn't that also mean that the opportunities for real innovation are greater?

Many of the reasons for the disagreements among us are cultural, but others may have to do with the nature of the resources themselves, in particular when we shift our focus back purely to the digital realm. For it is here that we find ourselves yet again handling that old chestnut crossed with a hot potato, sustainability.  If we do not yet “trust” digital resources the way we do analog ones, at least part of the reason for this is that we have not had them around for hundreds of years, and do not yet feel secure that we will have them around for many years or decades more. How can we validate a scholarly career on the basis of something that may be perceived as ephemeral? 

Is it possible, then, that the answer (or at least an answer) to having digital scholarship accepted as equally legitimate and valid as a peer-reviewed article is the development of sustainable structures for our research? As Jerry pointed out, the framework we have for encouraging, publishing, and sustaining print-based humanities scholarship has become so embedded in the academic and institutional systems that support it as to be nearly invisible. This framework, developed over centuries with substantial institutional support, was not called into question until it began to break. One might have presumed that our non-digital colleagues might have looked to digital publication as a way out of the current difficulties; as a way of building new institutional structures to support in the first instance traditional research activities while exploring new models made possible by digital formats. But rather, the opposite has happened. There has arisen instead a bunker mentality clinging to the high old ways as assiduously as the British clung to the Raj.

Is the charge of the transience of digital scholarship warranted? Or is it our expectations that need to be adjusted? One example will suffice: the New Opportunities Fund, which operated in the UK from 1999-2004. NOF was a massive digitization program funded from the UK National Lottery Fund, awarding £50 million to 155 projects. Its goal was to digitize and make available resources for lifelong learning. Five years after the end of the program, Alastair Dunning carried out a survey to ascertain the fate of these projects.1 The results of his survey are instructive:

10 have been absorbed into larger projects;

According to a blog post by Dunning, opinions on the success of the program have been divided: “some have seen it as pioneering; others saw it as a poor use of money, which did not really reap the expected dividends.”2  Although nearly 20 percent of the websites are no longer available, nearly 80 percent are, and when examining projects with more than £300k of funding, 87 percent are still available (34 of the 39 projects). 

If we compare this result to more traditional forms of print-based scholarship, these digitization projects are not doing too badly. We would not fault a traditional publisher if 83 out of 155 of their academic titles did not go into a second print run. However, if 40 of them did go into a second printing (the equivalent of the first two bullet points) with only 20 percent no longer in print five years later, the commissioning editor would be seen to have an outstanding track record. 

Rather than justify why less than 100 percent of the projects are still maintained as active research, perhaps we should be asking, as Jerry states in his opening talk, how do we ensure that digital scholarship remains part of the scholarly life-cycle? How do we ensure that the interfaces, web applications, or digital objects are maintained for future use by the scholarly community? We should be devoting more resources to the development of sustainable digital scholarship rather than accepting the fragility of the structures we create due to short-term or soft funding; resources that once created are situated outside the traditional funding and institutional structures in the humanities. Moreover we need to find long-term funding strategies for supporting the personnel and resources needed for these projects despite the fact that they are more typical of a science lab than a humanities project:  programmers, servers, web developers, metadata specialists, to name but the most obvious. 

In Europe, the response to this new level of support has been varied. In many respects it is similar to that of North America, although the institutional framework from within which this work is carried out is different. There is chronic underfunding for digital activities. Library and cultural heritage institutions attempt to fund digitization activities from their operating costs, but the pressure to maintain traditional core services, outreach, and a (robust) digitization program tips budgets to the breaking point. Moreover, cultural heritage institutions not only do not have a clear mandate to preserve digital materials (particularly when they are already preserving the analogue object), but frequently lack the expertise and resources to create and maintain preservation-level digital objects. Funding in Europe, like the United States, in the past seemed to thwart attempts by institutions—be they digital humanities centers, university libraries, or cultural heritage institutions—to take the long view and build infrastructure to support ongoing digital activities. Funding tends to be short term for specific projects encouraging, even today, the creation of digital silos rather than integrated resources. But these trends may be changing.

The United Kingdom was the first country to take a long view. In 1995, the Joint Information Systems Committee funded the Arts and Humanities Data Service (AHDS) at a national level to support data creation, awareness and education, collections, and preservation.3  At its close thirteen years later, it employed ca. 25 staff and received funding from two national agencies of approximately £1,000,000 (with another million raised from project work). It was the most established, longest running digital creation, training, and curation program in the world. When funding was withdrawn at a single vote by the Board of the Arts and Humanities Research Council, a tremor was sent throughout the digital humanities community. Nevertheless, the model was in place long enough for digital humanists in other European countries to lobby for similar infrastructures. 

In Ireland, the Digital Humanities Observatory (DHO) was established in 2008 (only months after the cessation of the AHDS) to develop an all-island inter-institutional research infrastructure for the humanities by building a platform for the coordination and dissemination of humanities research, teaching, and training. Like the AHDS, the DHO has a three-pronged mission: to be a knowledge resource for digital scholarship in the areas of training, consultations, and the development of infrastructure.  The DHO was funded as part of a larger consortium involving the majority of higher institutions on the island of Ireland through a funding mechanism that supports large-scale inter-institutional frameworks, the Programme of Research in Third Level Institutions (PRTLI). 

This funding provided the vehicle for humanities researchers to seek funding for a national infrastructure rather than create mini digital humanities centers across Ireland. Nevertheless, the focus of the grant was not so much on developing strategic national e-resources but on funding individual researcher's projects, albeit with many of them delivered in digital form. Thus one of the challenges the DHO has faced is creating a coherence from these disparate projects within one repository. 

Yet, like many digital humanities projects, the DHO was funded for three years without a clear business model or sustainability plan. There was the expectation that further rounds of PRTLI would allow for repeat funding or that European monies would be made available to fund core activities. But PRTLI is not designed to be a funding stream for the long term, and does not extend funding grants or give repeat funding to successful projects. Instead, projects are expected to have somehow become “sustainable” after a limited number of years’ core funding. This should be achieved either by being mainstreamed into their host institution or by making a research breakthrough that leads to licenses, patent and unlimited cash. Even for the sciences this model is a pipe dream. What happens instead in these cases is that core funding is replaced with piecemeal project funding, which is far more burdensome to administer and diverts staff from “core activities” the infrastructure was designed to carry out. While the DHO has made a significant impact amongst humanities scholars in Ireland (with over five hundred people attending our events in eighteen months and some one hundred fifty project consultations), its future is uncertain, although a business plan is currently being developed. In the eighteen months since the announcement was made about the withdrawal of funding from the AHDS, as well as the current difficulties of the DHO in the wake of the demise of the Celtic Tiger in Ireland, it is at best shortsighted and at worst irresponsible to fund national digital humanities platforms that have preservation mandates on uncertain funding streams. 

The French, however, have taken a different approach. In 2007, TGE Adonis was funded under the auspices of the Centre national de la recherche scientifique (CRNS) to support access to digital documents in the humanities and social sciences. Its priorities are three-fold: building a partnership between the various actors in research on human sciences; the digitizing of archives; and the digital publishing of scientific journals in the field.4  Its activities thus span the digital life-cycle from digitization to archiving. It is an integral part of the TGIR, a national infrastructural research program funded by the Ministry through 2014, crossing all disciplines. 

Although this humanities infrastructure is nascent, there is reason to believe that funding will be available for the long term. A similar physics infrastructure has been funded for over twenty years. Additionally, twenty percent of TGE Adonis’s staff are public servants who are not on short-term contracts, further indicating government commitment. There is evidence that the French government is committed to the need for a national infrastructure in support of the human and social sciences as per their announcement last December in which they committed to funding four new initiatives (http://www.tge-adonis.fr/?Trois-nouvelles-grandes&decoupe_recherche=TGIR). More significantly, there appears to be a clear intention on the part of the government to somewhat reconfigure the research landscape, in particular the “balance of power” between the CNRS  and other players in higher education (such as the Universities and the Grands Ecoles). This in turn means that questions about digital infrastructures are high on the agenda and questions of appropriate governance and funding models are under active consideration. The model that seems to be emerging is characterized by “mutualisation,” i.e., sharing and partnership beyond the CNRS, rather than centralisation.5 

The DHO and TGE Adonis exist against the backdrop of the development of European-wide infrastructures in the humanities. The European Science Foundation has a Forum on Research Infrastructures. Its standing committee in the humanities recently agreed to publish a policy document on humanities infrastructure, particularly digital infrastructures, which could mean additional funding streams. More important, perhaps, are DARIAH (Digital Research Infrastructure for the Arts and Humanities) and CLARIN (Common Language Resources and Technology Infrastructure). DARIAH and CLARIN are on what is known as the ESFRI Roadmap, which lists Research Infrastructures of pan-European interest that are being developed to meet the long-term needs of European research communities, covering all research areas across the entire Union. These infrastructures have at their core a commitment to enhance science and technology development at the international level, to support new ways of doing research in Europe, and to contribute to the growth of the European Research Area.

DARIAH and CLARIN are now in what is known as their preparatory phase and are seeking funding for the construction phase. DARIAH aligns itself probably more closely with traditional digital humanities, with a mission to enhance and support digitally-enabled research across the humanities and arts. It will do this by developing and maintaining an infrastructure in support of ICT-based research practices, from training to resource development to content management, delivery, and curation. Funding for these pan-European infrastructures are fairly complicated, with national funding agencies allocating monies to particular research infrastructures on the ESFRI Roadmap. If a country chooses not to support a particular infrastructure, then individuals and institutions from that country cannot take a primary role in its execution and development. Moreover, for countries to take a leadership role in an infrastructure on the ESFRI Roadmap, they must already have expertise in the area, with a reliable and stable national infrastructure. 

Thus there is a bit of a chicken-and-egg situation: countries must have the economic will at both the national and European level to commit to an infrastructure. They must have a credible national presence to be part of a European presence that will, no doubt, bring additional monies back to participating countries. Both Ireland and France have committed to DARIAH's next phase, thus making their national infrastructures (the DHO and TGE Adonis) crucial. But what if these were to fail? Clearly the soap opera of national/supernational infrastructural approaches to sustainability will be running on for some years yet.

Our inability to enable trust in digital resources is one of the major stumbling blocks to the reform of scholarly communications we wrote of earlier in this paper. But there are many other cultural barriers in place as well. Our attachment to our norms of scholarly communication is also a limitation, but it is also a part of who we are as scholars.  In many humanists’ eyes, the monograph is still the primary acceptable mode of discourse between scholars, with articles and book chapters coming in second, and very little else on the list.  Are we too locked-in to the hegemony of the .pdf, to creating not just authoritative virtual copies of our heritage documents and artifacts, but of our entire communications system as well?

If we apply a science paradigm, a digital humanities scholar could be compared to an experimental physicist, as someone who designs processes and instruments to find the answers to their research questions.   But the most striking difference between the experimental humanist and the experimental physicist lies in the fate of these processes and instruments after the article on the findings they enabled has been written: they are transcended, perhaps licensed to another for further use, perhaps simply discarded.  Why are we so different about our electronic data?  Would it be enough for humanistic scholars as well to draw their conclusions and let it go either to be developed by someone else or to mildew?  Or is there something inherently different in the nature of our data, that we should be so attached to its survival?  For example, we expect to receive credit for scholarly editions—why should we not receive it for digital scholarly editions?  Are the data collections created by humanists inherently more accessible and open than an experimental physicist’s algorithm or shade-tree spectroscope?  Are we not just creating slaves and drones, tools to meet our need, but instead, as McGann puts it elsewhere, contributing to a “Global Library,” something that invites users across time and space to access the information within and use it to answer their own questions and undertake their own experiments?

If so, then there is another pachyderm in the parlor to address, which is the conundrum of reuse.  We all know the excitement of the digital project, the feeling that comes as the tide of understanding begins to break over us with each new iteration of the interface or each refinement of the search terms.  The day we send the URL around to our scholarly colleagues is a great day, and we bask in the congratulations they return to us.  But then what?  Usage studies confirm that after our close friends and colleagues have an initial trawl of our project data, rate of reuse of digital projects is actually disappointingly low.  Is it that the act of organizing a digital collection or a dataset is always already too powerful an act of editorialism, thereby marking that intellectual territory as no longer open for original investigation?  Is it that in a world where the size of the ocean of analog data means that the mere choice to digitize a manuscript or a collection leaves a watermark that reads “noli me tangere”?  Or is it that we simply don't have the infrastructure to communicate about works produced in this manner?  Stripped of publishers’ lists, of their marketing channels and peer review and quality control systems, are we failing the next generation of scholars by creating too many resources in the wild?

Culture change is always the most difficult part of any corporate project, and in this case we have neither the profit motive nor the bounded scale of operations to hold our community together while we try to change minds not just about the value of digital editions, but of scholarly editions of all kinds. Generations of big projects, Europe’s DARIAH and Project Bamboo not excepted, seem to struggle with the notion that the right tools will turn the scholarly Sauls to Pauls, and bring them in their droves into the digital fold. Others put forward the notion that generational change will bring us along regardless of our efforts for or against changes in modes of scholarly communication. But younger scholars have a long road of apprenticeship to endure before they can call the tune rather than merely dance, and many of them will be well and truly institutionalized by the time they can take a bold stance on what they produce from their research. The elephant in the room may therefore in fact be an iceberg, with many layers of analog problems filling the depth beneath the digital ones we tend to focus on. Change can come, and organizations like the DHO and the Trinity Long Room Hub are well placed to facilitate changing attitudes from above and below. If, to paraphrase another old bit of rhetoric, we can get beyond the entrenched attitudes of the stonemasons and the bricklayers, perhaps we will all be able to see that what we are building is a cathedral.

Notes

^{1 .} http://web.me.com/xcia0069/nof.html. Thanks to Alastair Dunning of JISC and to Mark Hedges, who graciously shared their summary of the data with us.² http://digitisation.jiscinvolve.org/2009/03/11/success-or-failure-of-a-digitisation-programme-nof-some-evidence/.³ Alastair  Dunning, “The Tasks of the AHDS: Ten Years On.” Ariadne 48 (July 2006). http://www.ariadne.ac.uk/issue48/dunning.⁴ http://www2.cnrs.fr/en/843.htm.⁵ Our thanks to Lou Bernard and Laurent Romary for the help with information about TGE Adonis.