“Institutional Repositories: Insights and lessons from a Dutch University Library”

(1)

“Institutional Repositories: Insights and lessons

from a Dutch University Library”

Master thesis, MscBA, specialization Business & ICT

University of Groningen, Faculty of Economics and Business

Date: 27-10-11

(2)

ABSTRACT

This thesis has set out to gain insights and lessons from an institutional repository (IR) evaluation at the University of Groningen. The insights and lessons are used to inform a (system selection) advice for the University of Groningen. Using the regulative circle method of van Strien (1997) first a

diagnosis was made, three techniques were used; Key informants were consulted in order to model the current IR situation at the University of Groningen, A survey was held to gather IR experiences of other Dutch universities, Expert reviews were used to reveal trends, motivations, and behaviors towards IR , in order to discover critical IR success factors. The diagnosis provided insights that have led to a design of a preferred IR situation. The design was used to derive (system) requirements and to compare available IR software packages. Most suitable packages for the University of Groningen are the open-source solutions DSpace and EPrints. Although the software and infrastructure are important for a well functioning IR, it is just the beginning. Success of IR‘s depend heavily on usage, therefore further research should be focused on motivations for (non) usage.

1 INTRODUCTION

1.1 Background & Motivation

The success of scientific research is correlated with the accessibility of the published results of the research. Quick developments in information and communication technologies increase the ability to communicate and share research results. However, to get access to published results one needs to be licensed to view the digital published material. The high cost associated with these licenses exclude researchers that are connected to smaller research institutions, and form a barrier to the co creation and communication of scientific knowledge.

These barriers have led to an increased popularity in Institutional Repositories (IR) among research institutions. An IR can be defined as set of services that a university offers to the members of its community for the management and dissemination of digital materials created by the institution and its community members (Lynch, 2003). These services, consist of preservation and dissemination of the institutions‘ digital research output. Researchers have evaluated the impact of Open Access (OA) publishing, but the results so far are ambiguous. Studies by Xia & Meyers(2011) and Gargouri et al. (2010), have shown that OA availability leads to more article views which has a positive impact on citation count. However, Davis (2011), reveals that although the total amount of views increase there is little impact on citation count. Hence, the literature does not provide a clear view of the impact OA and IRs have on research rankings.

In 2003, the Dutch universities and KNAW 1research institutes started with an OA project called Digital Academic Repositories (DARE). The goal was to enhance the access to and visibility of Dutch scientific research. A joint platform of bundled IRs was created. The amount of attention given to IR development has placed the structured and bundled Dutch IR systems ahead of those elsewhere in the world (Enserink, 2009). However, most researchers at universities are not aware of the existence of IRs in their organizations and/or do not use them (Creaser, 2010). Therefore, when the platform of bundled IRs was finished, two national projects were launched, Keur der Wetenschap (2005)2 and HonDAREduizend (2006)3. These projects were to enhance the visibility and usage of the IRs. When the two mentioned projects were completed in 2007, still only an average of 9.5 percent of all

publications was freely accessible (Van der Graaf, 2009). This shows that usage and awareness of IRs is still low.

1_{Koninklijke Nederlands Akademie van Wetenschappen (KNAW) is an umbrella organization of research} institutes that advises the government on science policy.

2_{Keur der Wetenschap (2005) was a national project within DARE. The goal was to enhance the visibility of} leading Dutch research

(6)

1.2 Problem Definition

The University of Groningen aims to provide ICT facilities and services to support research and education, and aims to increase and/or maintain international visibility. The international visibility of universities is commonly based on rankings. To get a high ranking, universities have to showcase and disseminate their research results. Since 2003, the University of Groningen offers a repository service using WildFire software, which is an in-house developed software package. WildFire is developed by the University of Groningen Library and aims to enhance the access to, and to preserve university research output. However, in this digital age demands on the software package are increasing rapidly. File formats change, both in size and type, and faculties have more and diverse demands for the system. Keeping up with the growing list of system demands and requirements is causing more and more difficulties.

The overall objective of this thesis is to evaluate the University of Groningen IR situation, and identify available IR solutions, in order to provide a (system selection) advice to the University of Groningen. The goal of the evaluation is twofold, that is, there is a scientific and practical goal. The practical and primary goal of the evaluation is to investigate which IR software would be a suitable alternative for the current WildFire software. Subsequently, IR experiences of other institutes, and IR success factors have been investigated. The three studies are used to design a preferred IR situation. First, the design is used to derive a list of (system) requirements. Second, the derived list of requirements is used to inform a review of existing IR software packages. Lastly, the review is used to provide a (system selection) advice to the University of Groningen. The scientific goal of this research is to reflect the results on existing theory, and to find fruitful directions for further research. The following main research question guides this research;

“Which IR software would be a suitable alternative for the currently used WildFire software at the University of Groningen?”

Seven sub questions are formulated to answer the above practical research question;

Literature review

1. To what extent does existing literature explain the success of IR?

Diagnosis:

(7)

Design:

3. What IR systems do other Dutch universities use, and how do they deliver repository services? 4. What are the critical success factors for IR systems?

5. How should the IR process work?

6. What are the requirements for the preferred IR situation? 7. Which IR systems meet these requirements?

(8)

2 THEORETICAL BACKGROUND

Institutional Repositories (IR) facilitate information-sharing, they are a small part of the so called OA movement, and are still evolving. The OA movement is a social movement in academia, dedicated to the principle of open access, to information-sharing for the common good. This section reviews the literature to find what determines IR success.

IR systems were first founded to fulfill two important functions. First, they enable the university to expand access to research output, thereby reducing the monopoly power of journals and publishers. Secondly, they enable the university to showcase and disseminate their research output. This helps to demonstrate the scientific, societal, and economic relevance of their research activities, thus increasing the institution‘s visibility, status, and public value (SPARC,2002).

A lot of research and attention has gone out to OA publishing, and to IRs. Earlier work focused on the discovery of success factors. For IR to be successful, they have to be used. Therefore, Harnad (2008) has focused on usage, impact factors, and reward systems and explained that researcher careers (their salaries, promotions, tenure, funding, prestige, prizes) depend on their research impact. Universities (which co-benefit from the research funding, progress, and prestige), reward research impact. Researchers are rewarded for publishing their work in high impact journals. However, these journals are based on exclusivity rather than open access availability. Harnad (2008, p. 1) described this as the research access/impact problem: ―The research access/impact problem arises because journal articles are not accessible to all of their would-be users; hence, they are losing potential research impact. The solution is to make all articles open access‖. There are two ways to make articles OA available, the golden road, and the green road. The golden road is publishing your article in an OA journal. The green road is publishing your article in an non-OA journal and also self-archive it in for instance an IR. Although the authors have permission for self archiving, Harnad (2008) found that only 10-20% of articles have been self-archived. He concluded that the self-archiving method with the greatest

potential to provide OA, is self-archiving in one‘s own university‘s IR. Furthermore, Harnad (2008) argues that self archiving needs to be mandated by researchers‘ employers and funders.

The lack of an IR standard, makes evaluating other IR situations very valuable. Collecting experiences of other institutes might lead to valuable new insights. Furthermore, exploring the trends and

developments in IR services helps to form a picture of motivations, attitudes, and behaviors towards usage in general. The experiences of other institutes provide valuable insights into how they

(9)

teaching objects, rather than to products and by-products of research. Xia & Opperman (2009) explored the context of the IRs through the analysis of their content elements. They found that a large percentage of the IR content was student work. They also noted that M&B only institutes were using new strategies for recruiting and collecting repository content, moving away from the traditional model that relies on faculty self- archiving. Research has shown that different models for archiving content lead to differences in meta data quality (Kurtz, 2010). Kurtz found that librarian-overseen archives had the most complete and accurate records. The samples were drawn from multiple IRs and examined for metadata quality using the criteria of completeness, accuracy, and consistency.

Case studies provide valuable insights into IR situations of individual institutions. Case studies show how the institutions have approached the implementation and development of their IRs. Suzanne Bell, the IR administrator of the University of Rochester, described their experiences with the creation of their new IR in an interview (Hutchens, 2010). They have conducted user research to find the actual needs (Foster, 2005). Foster explained that for an IR to be successful it must be filled with scholarly work of enduring value that is searched and cited, because without content, an IR is just a set of empty shelves. This motivated Suzanne Bell to understand faculty, in order to improve content recruitment. She found that researchers wanted to showcase and share their research, and that they wanted options for versioning and sharing their work. Other user-based research found that commitment to IRs depends mainly on building trust with faculty, and solving copyright concerns (Kim, 2011). To

increase IR participation, universities should focus on digital preservation and copyright management. Lastly, Davis (2007) did a case study at Cornell University focusing on non-usage of their IR. Faculty members gave many reasons for not using the repository services including copyright issues, fear of plagiarism, and the usage of other modes for disseminating information. He concluded that the faculties each had their own culture, largely defined by their reward system, and that IRs needed to address the cultural diversity in order to be successful.

In conclusion, the literature shows there is no dominant IR standard yet, and that OA publishing and IR systems are still in their infancy. Furthermore, the literature does not provide a clear view on what determines IR success. Related work has mainly focused on the IR experiences of other institutes, and on OA publishing and IR usage. Because an extant explanation on the success of IRs is lacking, this research aims at deriving new insights by exploring the University of Groningen IR situation.

(10)

3 RESEARCH DESIGN

This research is shaped using the regulative cycle technique developed by Van Strien (1997). Figure 1 visualizes this approach. After defining the problem a diagnosis/analysis of the situation is made. The diagnosis/analysis forms the basis for a (re)design.

FIGURE 1: Regulative Cycle (van Strien, 1997)

This is a design-oriented research. Additionally, the results of the diagnosis and design, are used to explore the IR concept and to contribute to the scientific body of literature on IR. Exploration is not only focused on presenting a collection and arrangement of facts, but also aims on finding connections between them that are considered to be relevant (De Groot, 1972). According to De Groot two

methods can be used within an exploratory study namely; literature studies, and empirical studies. Literature, as shown in section two, has lacked to give a clear definition. Therefore, empirical studies are used to explore the IR concept, and to develop a better understanding of how to cope with IR.

As part of the diagnosis, key informant consultations are used to model the current IR situation at the University of Groningen. Analysis will lead to a deeper understanding of the IR situation, and forms the starting point for the redesign.

(11)

preferred IR situation. Subsequently, a list of (system) requirements is derived from the preferred IR situation. The list is used to review existing IR software packages and to inform a system selection advice.

The last two steps of the regulative cycle are implementation of the redesign an evaluating the redesign. These steps are not included in this study. Figure 2 provides an overview of the research structure.

(12)

4 DIAGNOSIS: THE CURRENT IR SITUATION

This research aims to find a suitable alternative for the currently used WildFire IR software at the University of Groningen. After having defined the problem and research questions, in section 1 the ‗problem choice‘ phase, the next step is the diagnosis phase. The current IR situation at the University of Groningen is diagnosed. The diagnosis will show how the University of Groningen copes with IR, and provide a deeper understanding of the IR concept. Knowledge of the current IR situation, and a deeper understanding of the IR concept and related problems, are necessary in order to create a (re)design.

An overview is made of the IR background and setting, in order to provide a view of how the IR is embedded in the University of Groningen. The IR setup is diagnosed to find out how the system is structured, and interlinked with other systems. Analyzing the IR content will reveal what the IR is used for. A diagnosis of the IR processes contributes to the understanding of the functioning of the IR, and to improving the performance.

4.1 Method

4.1.1 Key informant consultations

Key informant consultations are used to collect the necessary data. Key informants, as a result of their personal skills, or position within a society, are able to provide information and a deeper insight into what is going on around them (Marshall,1996). The key informants used in this study are the IR manager of the University of Groningen, an ICT developer who has expertise on repositories and IR programming, and the head of the Library Infrastructure and Innovation Division of the University of Groningen.

(13)

4.1.2 Actor Activity Diagramming

The overview of the current IR situation included the creation of process models. These models were created using the Actor Activity Diagramming (AAD) method. AAD decomposes business processes in terms of actors, activities and transitions. By modeling processes using AAD the effectiveness (reaching the final state) and efficiency (the efforts needed to reach the final state) of the business process can be assessed. This assessment creates indications for a redesign towards an ideal business process, thereby predicting the effect on effectiveness and efficiency (Schaap, 2001). AAD is derived from Role Activity Diagramming (Ould, 1995).

AAD keeps modeling simple by modeling on purpose. The models consist of actors and transitions, using a limited number of symbols. AAD is to be distinguished from often used modeling tools like flow or interaction diagrams, which have their origin in the field of developing information systems (Scheer,1999). These modeling tools aim at a complete and true representation of the domain to be considered in order to define databases or build information systems. This purpose causes the schemes to be complex and not well suited for a discussion about effectiveness and efficiency of business processes (Schaap, 2001).

4.2 Results: IR Overview

4.2.1 IR background

The development of Dutch IRs started with the DARE project. The goal of DARE was to enhance visibility and traceability of Dutch scientific research. All Dutch universities and KNAW research institutes now have their own IR. The joint National Academic Research and Collaborations Information System (NARCIS) 4 offers access to the content of the IRs enhancing visibility and traceability. The National library of the Netherlands (KB) duplicates the IRs content for the preservation of Dutch scientific research.

The IR is hosted at the University of Groningen Library. The University of Groningen Library has an important role in supporting research and education at the University of Groningen. The last 10 years have been full of developments and the increasing digitization and communication has led to big changes in the library world. The University of Groningen Library recognizes that their traditional role is changing, and focus is shifting towards digital library services.

The University of Groningen Library, now focuses on their IT infrastructure. The amount of content, and the list of functional and technical requirements for their IR system is continuously growing.

(14)

Meeting these demands and administrating the in-house developed software package (WildFire) is becoming increasingly difficult. Most time is consumed on solving problems and keeping the system running.

4.2.2 IR setup

The current in-house created IR software (WildFire) consists of three basic parts; a visual

representation on the web, an admin tool, and the ‗Universitair Digitaal Aanleverloket‘ (UDA). UDA is used for the delivery of content to the repository by the researchers, or their assistants. The admin tool is used by local administrators for uploading content into the repository. Furthermore, the system applies to the agreed Open Archives Initiatives Protocol for Metadata Harvesting (OAI-PMH) used by NARCIS and the KB when harvesting the repository.

All Dutch universities and KNAW research institutes have an IR for sharing, communicating, and preserving digital content. Besides the IR, they all use a Current Research Information System (CRIS) for the registration of research information. Research information in a CRIS describes the current and conducted research, it includes FTE records and the amount of publications. It can provide reports of individual researchers and faculty totals. The research information is used for reporting purposes and visitations, which forms the basis for the division of the university‘s research budget. The CRIS software used at the University of Groningen is called Metis5.

Together the IR (WildFire, with actual research output) and CRIS (Metis, with research information) provide the input for the so called ‗Medewerkers Pagina‘s‘ (MePa). MePa is an online tool used by researchers for showcasing their achievements and research output. MePa provides general

information about the researcher, his/her function at the university, and a publication list providing an overview of research they have conducted.

FIGURE 3: Overview Repository Environment SOURCE: Key informant University of Groningen

(15)

IR structure

The IR of the University of Groningen, is in contrast to other IRs, comprised of 94 smaller repositories. The various repositories are all accessible via one common portal6.

The university chose to develop WildFire because it enabled them to provide customized repository services for the different research groups and faculties. They adapted the repositories to specific wishes of respective users/groups. Furthermore, these repositories are locally administrated. Not only does this lead to differences in appearance, but also in content recruitment, content types, content metadata and content quality as well. The customized repository services lead to difficulties in

administration, locally made changes lead to problems when changes are made to other systems and/or when systems are updated.

IR linkages

The IR is based on a data and service model and has linkages to several other systems (see fig.4). These other systems, known as harvesters, harvest the repository content using the OAI-PMH standard for metadata harvesting. The university is responsible for the data layer. They are free to choose the software and structure of the IR as long as they conform to the agreed set OAI-PMH requirements . On top of the data layer is the service layer, everybody can harvest the repository and offer that information through services.

FIGURE 4: OAI Protocol

(16)

The University of Groningen offers meta data harvesting using OAI-PMH, as this enables the dissemination of research output. The National library of the Netherlands, or ‗Koninklijke

Bibliotheek‘ (KB), does not only harvest metadata but it harvests all digital content within the IRs of Dutch universities for preservation purposes. Besides the external linkages, MePa uses the system for the creation of personal publication lists. All current linkages with the IR system can be made using the OAI protocols.

4.2.3 IR content

The IR is used for preservation and dissemination of all sorts of research output. The IR of the University of Groningen holds different kinds of content, ranging from peer reviewed articles to student theses.

However, not all information stored in the IR can be defined as research output. When the university started to offer repository services they took a growth strategy, meaning there were little requirements on content. A content analysis based on the meta data shows that merely a quarter of the content is defined as article (28%), and a quarter of the content is labeled as images (23%). The remaining half largely consists of dissertations (17%), and even a relatively large percentage is labeled as ‗no

type‘(17%). The requirements for content deposits, and the quality control of supplied meta data varies per repository, causing differences in content and meta data quality.

4.2.4 IR processes

Within the University of Groningen, the delivery of research information for Metis, and the delivery of full text articles or other digital content for the IR, are two separated activities. These two activities vary per faculty. Faculties are free to choose their own delivery method and protocol. The University of Groningen does not mandate the delivery of IR content; only the delivery of research information for CRIS is mandated.

Below, the delivery of research information and the delivery of research content are modeled. For both the processes, an Actor Activity Diagram(AAD) is created for visual representation (fig. 5 and fig. 6). These processes are modeled at the Faculty of Theology and Religious Studies. The University of Groningen Library had already started a project at the Faculty of Theology and Religious Studies. The University of Groningen Library selected this faculty for their relatively small size, and their

(17)

Collect research information/publication data for Metis

The collection of research information is an annual process. The faculty employee responsible for the registration of research information sends a delivery request to the researcher. The request contains a digital Microsoft Office Word form which is send to the researcher by e-mail. The researcher fills in his/her publications and adds the correct metadata and if possible the digital location of the

publication. When the researcher is finished, the researcher returns the form to the faculty employee. The faculty employee processes the form. When the records are complete, the faculty employee copies the information, from the Microsoft Office Word form, and creates/updates them in Metis.

FIGURE 5 AAD: Collect research information/publication data for Metis (Source: University of Groningen faculty GGW Date: 12-07-11)

Deposit full text article/research content for IR

(18)

adjust the file format. The library employee obtains the correct metadata by doing desk-research or by contacting the author. When the library employee is done the repository is updated.

(19)

4.3 Conclusions

In 2003, the University of Groningen started a repository service with the joint DARE project. At that time they chose to built their own software which now needs more and more administration. The self built structure of the IR differs from other institutes. WildFire consists of 94 coupled repositories. These repositories were customized to fulfill local needs. However, these customized repositories now lead to difficulties in administration. Locally made changes lead to problems when changes are made to other systems and/or when systems are updated. The IRs are linked to a couple of systems, most important are CRIS and MePa. Other systems harvest the IR using the OAI protocols. Furthermore, the content that resides in the IRs is not only research output. Merely a quarter of the content is defined as article (28%), and almost a quarter of the content are images (23%). The remaining half largely consist of dissertations (17%), and files labeled as ‗no type‘ (17%). The quality of the materials cannot be guaranteed and metadata is not always complete. Two processes are closely related to the IR, collecting research information, and content depositing. Because of the different structures within faculty these processes are not standardized. The delivery differs per faculty and checking and correcting the delivered data is a time consuming activity.

(20)

5 DESIGN: IR AT OTHER INSTITUTES

All Dutch universities have repositories in place. Their experiences with IR can provide valuable information for the redesign and (system selection) advice. The lack of a IR standard makes the experiences of other Dutch universities, with IR software and services, a valuable source of

information. By evaluating their experiences more can be learnt about available IR systems and how they are used in practice.

A survey is an excellent research method for collecting experiences and for answering who, what, where, how many, and how much questions. Therefore, a survey was held among all Dutch universities and the KNAW research institute, with exception of ‗special‘ universities in fields like theology or humanities. These ‗special‘ universities have a different organizational structure and are therefore excluded. The survey aimed to answer questions in three categories namely:

Which repository systems are used?

Which IR systems are used, and how? Acquiring information about the IR services offered, and the IR software used to deliver them.

Functional and technical demands of the repository systems?

What are the functional and technical demands of IR systems? Acquiring information about selection criteria, considered software alternatives, and their experiences with customization of the software.

Administrating demands of the repository systems?

Is there any support available for IR systems, and what are the administrative demands? Acquiring information about available software support, and the amount of time spend on IR system

administration

The gathered data was analyzed and used to create an overview of what IR systems other institutions use, and how they experienced these systems. This overview will provide ideas for possible solutions and serves as an input for the (re)design in chapter 7.

5.1 Available IR systems

(21)

Below an overview of the different available IR software systems is provided. The information is derived from the Registry of Open Access Repositories (ROAR)7.

TABLE 1: Repository software

SOURCE: ROAR http://roar.eprints.org/view/software/ 20-09-2011

Repository software Installations (>5)(2198) Type

ARNO 6 Open-source

Bepress (DigitalCommons) 123 Commercial (Hosted)

CDS Invenio 17 Open-source

ContenDM by OCLC 6 Commercial

DigiTool 9 Commercial

DiVA 25 Open-source (Joint project)

DSpace 936 Open-source

EPrints 412 Open-source/Commercial (Hosted)

ETD-db 30 Open-source

Fedora 40 Open-source

Greenstone 13 Open-source

HAL 13 Open-source (Joint project)

MiTOS 16 Commercial

MyCoRe 6 Open-source

Open Journal System 12 Open-source

Open Repository 19 Commercial (Hosted)

OPUS (Open Publications System) 44 Open-source (Joint project)

Other software (Various) 455 Various

7

(22)

5.2 Method

5.2.1 Survey

To acquire an overview of the IR software used by other Dutch universities, this study used a web-based survey. A key benefit of using a digital web-web-based survey tool is its ability to reach respondents easily despite their location, and to have responses returned quickly (Dillman, 2000). Truell (2003) did a review of resources used for research, in this research he found that web-based survey tools had high response rates compared to paper based surveys, responses ranging from 34,5% to 84%.

The survey held consisted of nineteen questions, and was created in collaboration with the University of Groningen Library. The survey questions were structured in three groups, aiming to answer which systems are used, what the functional and technical demands are, and what the administrating demands are. The survey consisted of open ended questions, and multiple choice questions. The multiple choice questions provided the option for the respondent to add a personal answer. To maximize the response, a pre announcement was sent. The pre announcement was sent by the University of Groningen

repository manager who maintains contacts with IR managers of the other Dutch universities. A week before the survey closed, a reminder was sent to the participants that had not replied yet. The survey was in Dutch, the questionnaire is added to this document in the appendices (appendix 10.5).

5.3 Results: Survey Overview

The survey was sent in July 2011 and closed in August 2011. Eleven universities and the KNAW participated in this study, the Radboud University Nijnmegen (RU) and the Open Universiteit (OU) could not participate in this study, the employees responsible for the IR were unavailable during the survey period. Delft University of Technology (TU Delft) was only able to fill in the general

(23)

TABLE 2: Survey Respondents Institute

Delft University of Technology (TU Delft)

Eindhoven University of Technology (TUE)

Erasmus University Rotterdam (EUR)

Koninklijke Nederlandse Akademie van Wetenschappen (KNAW)

Leiden University

Maastricht University (UM)

Tilburg Universtiy (UvT)

University of Amsterdam (UvA)

University of Twente (UT)

Utrecht University (UU)

VU University Amsterdam (VU)

Wageningen UR (University & Research centre)

5.3.1 Which repository systems are used?

Three of the fourteen respondents use custom built repository software. Three respondents use Fedora software, TU Delft uses a combination of Fedora software with a custom build interface. The UvA has a Fedora installation in place which is not used as IR. Furthermore, there are three respondents

operating on DSpace, three use ARNO, and two use EPrints.

(24)

TABLE 3: Repository Services

Institute Services

Easy access to

publications

Tool for generating publication lists Workspace for researchers to work together Other Eindhoven University of Technology (TUE) X X X - Erasmus University Rotterdam (EUR) X X - X (Automatic metadata, uploads to other repositories) Koninklijke Nederlandse Akademie van Wetenschappen (KNAW) X - - - Leiden University X X - - Maastricht University (UM) X X - - Tilburg Universtiy (UvT) X X - - University of Amsterdam (UvA) X X - - University of Twente (UT) X X - - Utrecht University (UU) X X - - VU University Amsterdam (VU) X - - - Wageningen UR (University & Research centre) X X - X (Reporting tool)

The size and growth rates of repositories vary per institution. Furthermore, there are substantial differences in the total amount of items and in the amount of full text items. This is due to different content recruitment strategies for the repository. Some institutions even mandate their researchers to provide digital versions of publications for the repository.

5.3.2 Functional and technical demands of the repository systems?

This section covers the functional and technical demands of repository systems. It contains an overview of the selection criteria of the respondents, and an overview of their experiences with customizing repository software.

(25)

implementation against development capability, an internationally used system, usability, and technical specifications. Table 4 provides an overview.

UM further elaborated on their selection, and explained that EPrints was a serious alternative at the time. They have chosen to use ARNO because they had close ties with UvT, who is the developer of the software. Also, the absence of other EPrints users was a reason not to go for their software.

TABLE 4: Alternatives & Selection Criterea Institute* Current Software Considered Alternatives Selection Criterea Erasmus University Rotterdam (EUR)

Custom ? Technical specifications, innovation, usability

Koninklijke Nederlandse Akademie van Wetenschappen (KNAW) EPrints Fedora DSpace

Experiences of other institutes, ease of installation and configuration, linkage to CRIS (= source system)

Leiden University DSpace DigiTool (Ex Libris) ?

Maastricht University (UM)

ARNO/Fedora EPrints (2003) Speed of implementation against development capability (support) University of Amsterdam (UvA) ARNO ? ?** University of Twente (UT) EPrints ARNO DSpace Fedora

Usability and technical specifications

Utrecht University (UU) DSpace DSpace Eprints

An internationally used system. Technical specifications.

* VU, UvT, TUE, and Wageningen UR did not consider any alternatives.

** Saskia Woutersen from UvA explains she did not work there at the time. They do however look at alternatives for their ARNO installation.

Repository systems have to be tailored to specific requirements that vary per institute. Table 5 provides an overview of the experiences of the respondents with customizing their repository installation. All respondents except UT altered their software. Of the remaining 10 respondents, 7 needed additional knowledge for their software. Of the 7 that needed additional knowledge 4 acquired the knowledge elsewhere in their organization, 3 of them acquired the additional knowledge

(26)

TABLE 5: Customization Experiences Institute Current Software Out of the box software?

Modifications Experiences with customizing the software Extra knowledge needed? Knowledge acquired Eindhoven University of Technology (TUE)

Custom No Cache Very well, changes are made by our own experts Yes, Cache Externally

Erasmus University Rotterdam (EUR)

Custom No (Software developed by themselves), over time various modifications.

Software developed by themselves Yes, Pyhton, C, RDF Internally

Koninklijke Nederlandse Akademie van Wetenschappen (KNAW)

EPrints No Self-built interface for CRIS Software is not so complex, we have the advantage of an experienced programmer that knows the system well. Having an custom built part means that updates / upgrades of the software can cause unexpected problems (which already did).

Yes, Perl, MySQL, XML/CSS, OAI-PMH, MODS

Internally

Leiden University DSpace No Embargo protection, document structures, statistics

DSpace 1.3 and earlier, proofs to be relatively easy to modify. From DSpace 1.6 we (along with Atmire) worked on a more structured and manageable environment. That did make everything more complex for the developers.

Yes, Java,Perl Externally

Maastricht University (UM)

ARNO/Fed ora

No IP shielding Perl knowledge was sufficient No -

(27)

University of Amsterdam (UvA)

ARNO No - Difficult, especially with updates and new versions. Then they have to check that everything still works. ARNO stopped development, so they won‟t get any new updates.

Yes Externally University of Twente (UT) EPrints Yes - - - - Utrecht University (UU)

DSpace No User interface We do not use the presentation side of DSpace, we built a new interface. The presentation side of DSpace was at the time insufficient, in our opinion. We are now reconsidering that choice, since DSpace has made significant steps. An additional interface requires extra maintenance and management.

Yes Internally

VU University Amsterdam (VU)

DSpace No Metis-DARE link, OAI/PMA link, GGC link and the ability to batch import items

Configurations are not that complex. Program code is complex. Documentation is very scarce. Systems are vulnerable following updates due to customization.

Yes, HTML, CSS, Java, SQL Internally Wageningen UR (University & Research centre)

(28)

-5.3.3 Administrating demands of the repository systems?

Besides installation and implementation, repository systems need to be administrated. Administrating the system is an ongoing activity. To create an overview of the administrative burden, respondents were first asked to specify the available support, followed by their administrative activities and the amount of time spent on these activities.

Two institutes use EPrints software, KNAW and UT. They responded that EPrints offers all the specified support activities. EPrints software is updated annually, and if needed, the developer could even host the system for you. ARNO software, used by three respondents, does not offer updates and the only form of support is through personal contacts with other users. Respondents that use custom build software also have little support, TUE has some technical support through an external partner and Wageningen UR notes that there is a user helpdesk available for their system. VU, UU, and Leiden University have an DSpace installation. DSpace has user and technical support forums, and technical questions can be sent to an support e-mail address. DSpace is at least updated annually. Leiden University and UU both use an external party (Atmire) for support and installation of updates.

TABLE 6: Available Repository Support Institute Current software Support User Forums User Helpdesk Technical Forums Technical Helpdesk Other Update Freq-uency Eindhoven University of Technology (TUE) Custom - - - X X (Intersystems Cache) Annually Erasmus University Rotterdam (EUR) Custom - - - - - No Updates Koninklijke Nederlandse Akademie van Wetenschappen (KNAW) EPrints X X X X X

(System can also be hosted by the developer.)

Ad hoc

Leiden University DSpace X - X - X

(Atmire Leuven.) Annually Maastricht University (UM) ARNO/ Fedora - - - - X (Personal contacts) No Updates Tilburg Universtiy (UvT) ARNO - - - - X

(We have no real users, for the ones we do have, we

have relatively short lines.We do not need support, but we offer it if

(29)

University of Twente (UT) EPrints X X X X - Annually Utrecht University (UU) DSpace X - X - X

(Atmire, is used for upgrades.)

Annually

VU University Amsterdam (VU)

DSpace - - X - X

(An e-mail address where you can go with questions.)

1 or 2 times a year Eindhoven University of Technology (TUE) Custom - X - - - When required

IR maintenance and support is necessary in providing services for the whole institution. Maintenance and support can consist of different activities: user support, entering content, troubleshooting, software development, etc.. These activities can be carried out by faculty or more likely at the University Library were the repository is administered. Table 7 provides an overview of maintenance and support activities, and the time spend on these activities, for each of the respondents.

(30)

TABLE 7: Activity & FTE spend

Institute Activity & FTE User Support Entering

Content Trouble-shooting Software Development Other Total FTE TUE (Custom) - X - - X (Repository is filled via Metis ) 0.2 EUR (Custom) X X (3.0 including user support) X X (1.0 including troubleshooting) - 4.0 KNAW (EPrints) X - X X (0.3 + 0.3 development + information management) X (Communication depositing (open access and digital preservation), policy (0.2)) 0.8 Leiden University (DSpace) X (0.4 functional management, copyright questions 0.4, and general communications and support 0.4) X (1.0) X X (0.2) - 2.4 UM (ARNO/ Fedora) X - X X (ARNO+ Wish 0.2 Fedora 0.6) - 0.8 UvT (ARNO) - - X X X (Technical and application management (0.1)) 0.1 UvA (ARNO) X X X - X (Control of Imports (uploads and correction, granting DAIS + theses entire procedure) (3.0) General (0.5)) 3.5 UT (EPrints) X (0.5) X (2.0) X (0.5) - X (Functional changes) (0.5) 3.5 UU (DSpace) X (0.5) X (Usually done by faculty using Metis 0.3, theses 0.3 by UB) X X X (Regular maintenance and management, including harvesting, backups, etc. (0.8 including troubleshooting)) 1.9 VU (DSpace) X (0.2) X (0.2) X (0.05) X (0.4) X (Obtaining Content (0.3), policy (0.3), technical management (0.2)) 1.65 Wageningen UR (Custom) X - X - - No answer given

(31)

* Added/subtracted 3.0 FTE for a „Entering Content‟ activity specified by UvA under „Other‟.

** This does not include the time for troubleshooting specified by EUR under „Software Development‟ and the time specified by UU under „Other‟ activities.

*** The total amount of FTE specified is 18.65, added 0.2 unspecified FTE from TUE.

5.4 Conclusions

The survey gathered experiences of other Dutch research and KNAW institutes with repository software. The collected experiences provided a picture of how other institutions cope with IR.

The respondents use different IR software systems, only three of the fourteen respondents use custom built software, and three use ARNO software, software created by UvT, UT, and UvA. All others use more commonly known and internationally used repository systems, like DSpace, EPrints, and Fedora.

Most of the IRs are only used to provide easy access to publications and for the creation of publication lists. There are, however, substantial differences between the size and growth rates of the repositories. This is due to different content recruitment strategies, some institutions even mandate their researchers to provide digital versions of publications for their repository. Furthermore, experiences of other institutes also show that customizing the repository systems often leads to problems when systems are changed or updated. However, most of the administration time is spend on entering and correcting content.

The gathered experiences of other institutes show that huge customization efforts should be avoided, because these often lead to problems. Furthermore, the focus should be on content recruitment policies, to grow the repository, and to guarantee quality of delivered materials. Most time can be saved by improving the delivery of research information and content, this will lead to a reduction in time needed for entering additional information and correcting content.

(32)

TABLE 8: Repository Software Conclusions SOURCE: Repository Survey

Repository Software

Advantages Disadvantages

ARNO  Proved to be stable and reliable

 Customizable input/output formats

 Little additional knowledge required

 Close link with developer (UvT)

 No updates

 Little support (User helpdesk only)

Custom Software

 Can be build according to wishes

 Customizable input/output formats

 No updates

 No support

 Need for programmers

 Need for specific knowledge

DSpace  Annual updates

 Various types of support

 Various input/output formats

 International user base

 Easy configuration

 Custom parts can lead to trouble with updates

 Complex program code

EPrints  Annual updates

 Various types of support

 Various input/output formats

 International user base

 Easy configuration

 Simple program code

 System can be hosted

Fedora*  International user base

 Simple program code

 Little additional knowledge required

(33)

6 DESIGN: WHAT ARE CRITICAL SUCCESS FACTORS FOR IR

IR services and the supporting IR software keep evolving, while there is no dominant standard yet. Therefore, experts were reviewed to gain information about IR and to identify critical success factors.

The expert reviews were conducted by doing interviews. Three experts were interviewed. First, drs. P.J.B.M. (Peter) van Laarhoven who is head of Faculty Library Services & Collections at the

University of Groningen. His expertise includes knowledge of collection creation, licenses & license administration, citation analysis, and open access. Second, drs. G.J. (Gert Jan) Bokdam who is the project manager of the Current Research Information System (CRIS) of the University of Groningen. And lastly, mrs. (mr. or Master of Laws (LLM)) E. (Esther) Hoorn who is an information specialist with expertise on copyright and open access.

The collected information was used for the identification of critical success factors for IR. These critical success factors provide ideas for possible solutions and serve as an input for the (re)design in chapter 7.

6.1 Method

6.1.1 Expert review

An expert review is a good alternative for formal user studies. The expert review method tends to produce qualitative results and requires fewer participants than controlled experiments (Torry & Möiler, 2005). Interviews are very suitable for gathering personal opinions, the semi structure of the interviews provided room for an open conversation with the interviewees. The open form of the interview enabled them to present their personal opinions without constraints.

The interviews provided expert opinions of how to cope with IR. These new insights were used as an input for the redesign and list of requirements. The interviews were guided by an interview protocol. The interview protocol is added to this document in the appendices (appendix 10.10). The interviews were semi-structured, guided by predefined subjects but without strict questions, and were conducted in August 2011. The interview structure consisted of three parts, the introduction, an open part, and a check and end.

(34)

and subjects from the first question, in order to find critical success factors for IR. The check and end section is used by the researcher to check if all necessary topics are covered.

6.2 Results: The Interviews

6.2.1 Interviewing the head of Faculty Library Services & Collections Peter van Laarhoven When the OA movement started Peter worked as the license manager for the University Library, and was responsible for acquiring licenses for scientific databases and journals.

At that time, publishers annually raised their license fees, growth rates sometimes ran up to fifteen percent. The rapid cost increases forced universities, and their libraries, to choose between the

different journals, now known as the ‗serials crisis‘. According to Peter OA publishing was one of the first real attacks on the business model of publishers.

The creation of an IR fitted perfectly in the OA movement. Peter mentioned that: ―Although there is no scientific proof for it, the OA movement made publishers drop their tremendous growth rates in license fees‖. Even the publishers themselves are now trying to gain profit from OA publishing. They provide open access services also called ‗open choice‘, where the author pays a fee for the

dissemination of their work.

The publishers‘ business model only survives because universities keep paying the tremendous license fees. As long as university ratings are based on journal related impact factors university boards will keep stimulating their researchers to publish in these high impact journals instead of OA. The Berlin Declaration that the university signed, which stated that publically funded research should be made publically available, can be seen as just a statement. Universities will never be publishers, however, according to Peter, this does not mean we cannot try to archive university output.

The researcher wants to do research not administration. ―To facilitate and stimulate the archiving of scientific university output‖, Peter said, ―We should combine the delivery of full text articles with the delivery of research information for the CRIS. This will reduce the workload for the researcher or research administrator‖. Other options for reducing workload could be realized by pre loading publication information for the researcher. Publications can be collected from web of science or other sources, researchers only have to choose their own publications and supplement them. Also providing more options for access control for content would be stimulating.

(35)

enough. This will help in growing the repository, thereby, improving the visibility of the University of Groningen.

6.2.2 Interviewing the Current Research Information System project manager Gert Jan Bokdam Traditionally libraries collect information from external sources, thereby, managing outside in. Due to the OA movement and the digitalization of research results the role of the library is slowly changing from not only managing the outside in but also managing the inside out, as IRs are created to disseminate information. According to Gert Jan two questions need to be answered to perform this new role effectively. What content should be made available, and how should we make it available?

Universities have to justify how they spend their budgets. A visitation commission checks the amount of publications, the necessary activities, and the money spend. The allocation of research budgets for the university and faculty are based on this research information. Therefore, the university collects all research information in a Current Research Information System (CRIS).

As mentioned earlier, researchers are stimulated by the university for publishing in high impact journals. Getting their work published in a high impact journal is rewarding for the researcher because this gives them status within their respective field and boosts their career. The high impact journal also validates the work with the research community. According to Gert Jan researchers are afraid for OA publishing because it lacks the quality control. Furthermore, they are afraid that publishing in a IR will devalue their work with the possibility of harming their career.

OA publishing is something that is on the move but still needs time to mature. Gert Jan said: ―When the advantages of OA become more clear and OA becomes a shared thought, then a growing number of publications will be OA‖. According to Gert Jan, the researchers that gain the most in the current reward model are not against OA in principle. However, they are not going to give up their privileges voluntarily. A mandate to publish their work OA could work, but might better be avoided, as forcing them will not be appreciated.

(36)

The answer to the questions; what content to make available, and how to do it, is not simple. The content question is probably the easiest. All publications listed in CRIS are important for the

university, therefore, a full text of all these publications should be made available. How this should be done is more difficult, however, combining the delivery of content for the IR with the delivery of research information for CRIS seems a viable option. Furthermore, according to Gert Jan, the library could also play an important role in the quality control of bibliographic research information, and in making the full text available with the help of the IR.

6.2.3 Interviewing the information specialist of the University Library Esther Hoorn

Esther Hoorn studied Law in Leiden, and worked there at the University and the Ministry of Justice. She now works at the University Library of the University of Groningen as an information specialist, and has expertise on copyright and OA. She helps researchers with publication strategies and

copyright issues.

According to Esther, the research community is slowly discovering the advantages of OA publishing. OA is on the move, however, just like Peter, she points out that the business model of publishers is slowing down OA. The publishers‘ business model is based on exclusivity. Authors give up their copyright to the high impact journal in exchange for an increase in status and recognition in their respective field. Esther explains it are the universities themselves that pay publicly funded money for this exclusivity and thereby support the publishers‘ business model.

The only way to make changes to the current business model is by altering the reward structure for researchers and the impact factors for journals. Esther mentioned that a solution would be to create high impact open access journals. She said: ―It is important to stimulate researchers to publish OA‖, this could be done for instance by encouraging them to deposit in subject repositories. Subject repositories are field related and thus create communities and act as quality control. An example she gave was the subject repository of the particle physics researchers conducting research with the Large Haldron Collider (LHC) at CERN in Geneva. They have set up their own repository for their

community, so they could share their large research results. The only access condition is to also OA publish your own work. Afterwards, they self-arrange the dissemination and publication of the research.

(37)

experienced with OA publishing. The library, whose expertise lies in scientific find ability, could use these experiences with OA publishing to perform a consulting role for researchers.

According to Esther, the library can probably never perform the desired role of high impact publisher. As final option, the university could mandate the delivery of content to the repository, unless the researchers have solid reasons for non-delivery. Other Dutch universities, already have similar mandates. A mandate will lead to open communication with researchers about reasons for non-delivery.

The IR cannot comply to all wishes of all different research fields and communities within the university. Esther stresses that it is important to implement IR software that has a active user community from which we could learn. Some of the open-source software systems have active user communities that constantly update and develop the software. These communities are experienced with OA, and with the demands different research fields have, therefore, the university could benefit a lot from joining a community.

6.3 Conclusions

IR experts were reviewed in order to form ideas for the (re)design. Firstly, they were asked what OA and IR mean for the researcher/University of Groningen? This allowed them to set the context and subject. Secondly, they were asked how they should cope with that situation. Their solutions were used to define critical success factors for IR, table 9 provides an overview. The critical success factors are used in the (re)design.

TABLE 9: Critical Success Factors

Lowering the barriers to delivery  Combined delivery of research information/content

 Pre loading publication information for the researcher, thereby, reducing workload

Providing more options for ‗Access control‘

Irrelevant content in the IR  Researchers should only store what is important in the IR. Only publications listed in CRIS are allowed storage capacity.

Illegal content and copyright concerns  The library should have an advisory function for Open Access publishing and Copyright questions. (Access control)

Quality of the metadata  Researchers should self archive, and provide meta data for their own publications.

 The library should act as an controller, they should check metadata for completeness and accuracy.

IR usability and usage  IR needs to be structured by subject to make browsing easy and to stimulate deposits.

(38)

7 REDESIGN

Following the regulative circle approach the next step is to create a redesign. The redesign is made using the new insights derived from the diagnosis phase (chapter 4), the survey (chapter 5), and expert reviews (chapter 6). A preferred IR situation is modeled, and together with the acquired insights, used to construct a list of (system) requirements. These requirements are used to review available IR software. The review is used to provide a system selection advice for the University of Groningen.

7.1 How should the IR process work?

To better understand the IR environment, chapter 4 provided an overview of the current IR situation. The underlying processes were modeled using AADs, which provided an detailed overview of the process steps. The current IR situation is used as a starting point for the (re)design

Expert reviews were presented in chapter 6, these provided a list of IR success factors. The interviews, have led to a better understanding of the overall IR state and direction. They have provided insights in the IR; evolution path, usage, quality, content, underlying business models, and attitudes towards IR. These new insights were used to form critical success factors.

The gathered information is used to review the current IR situation. This review has led to a design of a ‗preferred IR situation‘. The preferred IR situation is used to derive (system) requirements. The requirements are used to inform a review of available IR software packages, and to provide a (system selection) advice to the University of Groningen.

7.1.1 What does the preferred IR situation look like?

The preferred IR situation combines two processes, the delivery of research information, and content depositing in the IR. Combining these two will lead to less administrative work for the researcher. Combining the workflows will most likely have a positive impact on IR deposits as well. Depositing research information for CRIS is mandated. The IR deposit will a become part of the delivery of research information, thereby, reducing the administrative burden for the researcher.

(39)

accessible via CRIS and not via the repository. The control of research information is done annually by the research office. Therefore, this process is not included in the overview.

The library also benefits, namely by standardizing the workflow and delivery formats, and letting the researcher supply their own publication list. This will lead to a reduction in time needed for entering content and correctional work.

If the researcher has any questions regarding copyright issues or OA publishing, he/she can address these to the library. The library is a specialist in this field and can easily provide advice for the researcher. Informing the researcher and providing him with open and clear information will help in lowering the barriers to deposit. Furthermore, the researcher should be able to choose the access conditions for his publications. Providing the researcher more options for access control will also help to remove barriers to deposit.

The library has expertise in archiving and the creation of bibliographic data. Therefore, the library is very suitable to provide a monitoring role. This means ensuring the quality of the bibliographic data and content of the publications. When the library approves the publications, then CRIS will be updated, and when supplied, the full text will be archived in the IR.

In the current situation, that is based on a growth strategy, all sorts of information and content can be archived using the IR. Content analysis has shown that the current IR is partly filled with irrelevant content of which the quality cannot be guaranteed. All items in CRIS are of sufficient quality. Therefore, in the new situation, all items in the IR should be traceable to CRIS. Creating this one on one relationship will guarantee the quality of the content stored in the IR.

Besides ensuring the quality of metadata and IR content, it is important that the IR is well searchable. Therefore, the IR should be arranged by subject, instead of university component. Researchers are primarily interested in research in their own field. Providing them the ability to exchange relevant data with their colleagues in the same research field, will be a stimulus for the researcher to publish his/her work in the IR.

Furthermore, by choosing for an internationally used software package with a large and established user base, the university can learn from best practices. The different demands and requirements of different research groups and departments within the university show commonalities across

(40)

TABLE 10: Preferred IR Situation Process Steps SOURCE: Key Informant Consultations & Expert Reviews

Actor: Activity Condition Description

Researcher – Personal Metis

Check Publication list None The researchers checks a personal pre- loaded list of publications. The list is obtained from 3rd parties like Web of Science & Scopus.

Correct Publication list Wrong/Missing Publications When there are wrong publications in the list, or when publications are missing the researchers corrects the list.

Supply Metadata for Publications

None The researcher supplies/adds metadata to the listed publications.

Researcher - Library

Questions concerning Copyright or OA

Unknown Copyrights, Help with Publication Strategy

When the researcher has questions concerning copyright or OA publishing he/she can get advice from the library.

Upload Full text None The researcher uploads the full text‘s to Personal Metis.

Specify Access Control None The researchers specifies access rights for the delivered content.

IS: Personal Metis – Researcher

Copy of Delivery None Personal Metis sends a

confirmation about the deposited materials to the researcher.

Library – Personal Metis

Check for Completeness None The library checks the delivered publication list and bibliographic data for completeness.

Library - Researcher

Questions concerning Publication list

Wrong/Missing Information When necessary the library contacts the researcher about the missing information.

Correct Metadata Wrong/Missing Metadata The library corrects wrong information, and adds missing information.

Correct File Format Wrong file format If necessary the library can adjust wrong file formats.

(41)

Metis

IS: Personal Metis – CRIS

Update CRIS None Personal Metis updates CRIS (Metis) with the gathered research information.

IS: Personal Metis – IR

Upload Full text to IR None Personal Metis uploads and archives the full text file in the repository

IS: Personal Metis – Researcher

(42)

FIGURE 7: Preferred IR Process

“Institutional Repositories: Insights and lessons from a Dutch University Library”