A framework for the organisational effectiveness of funding agencies in the science sector

I

A FRAMEWORK FOR IMPROVING THE ORGANISATIONAL

EFFECTIVENESS

OF FUNDING AGENCIES ON THE SCIENCE SECTOR

I

~I

I

►

I

/0

l

i':

'

I

~, !

....

r:..

"''

r,1

.

'

~-

·-

Thesis submitted for the degree

PHILOSOPHIAE DOCTOR: BUSINESS MANAGEMENT

at the Mafikeng Campus of the North West University

Supervisor: Professor PD Gerber

Co-supervisor: Professor S Swanepoel

DECLARATION

I, Rakeshnie Ramoutar-Prieschl, declare that this dissertation, entitled: "A framework for the organisational effectiveness of funding agencies in the science sector", is hereby submitted to North West University in fulfilment of the degree, Doctor Philosophiae, in Business Management, and has not been submitted in any form for a degree at any other university. This is my own work in design and execution, and all materials and other sources of information have been duly acknowledged and cited.

Signature:

R Ramoutar-Prieschl

ABSTRACT

Research funding agencies are quasi-public organisations mandated by specific legislative acts or laws. Although they are independent entities, they are still dependent on government for financial resources. Through the resources they manage, funding agencies are able to facilitate and drive research and development. Although funding agencies exist in the midst of a system compounded by numerous external players, their responsibility is to optimally utilise limited resources. These resources are invested, through the use of grant awarding processes, to encourage research productivity from those scientists who are recipients of grants. However, funding agencies, like all other public organisations, face challenges such as the: poor allocation and utilisation of resources; weak management practices; weak research-performing institutions; unpredictable political, economic and ethical environments; and poor morale and motivation among public employees. Given this context, it is difficult yet critical to establish a useful, reliable and valid set of effective indicators for evaluating the organisational effectiveness of funding agencies. This study therefore endeavoured to address this very issue by firstly providing insights into the factors and subfactors associated with the balanced scorecard, with the aim of explaining, from a theoretical perspective, how each of these factors influence the organisational effectiveness of funding agencies. Secondly, this research sheds light on some of the key elements necessary for improving the organisational effectiveness of funding agencies in the science sector by presenting a conceptual framework that was based on the research findings from several countries internationally. The research questions that were addressed in this study, were as follows:

1) Do the factors and subfactors cluster according to the balanced scorecard that was proposed in this study?

2) What are the main factors influencing/impacting the organisational effectiveness of funding agencies?

3) Can a conceptual framework for improving the effectiveness of funding agencies be developed?

In addressing the research questions and objectives of this study, a sequential explanatory mixed methods approach was adopted. Initially, a theoretical model in the form of a five factor balanced scorecard was proposed, which was used to develop a survey, geared towards evaluating employee perceptions relating to the organisational effectiveness of funding agencies. From this phase of the study, a total of 188 responses were received, and these

the factor analysis revealed that four interrelated and interdependent factors, play a contributing role in influencing the organisational effectiveness of funding agencies. These factors include: environmental and stakeholder management; financial management; organisational business processes; and organisational learning and growth. It was further found that each of these factors play an equally important role in influencing the organisational effectiveness of funding agencies. The outcomes from the first phase of the study, were subsequently verified and validated in the secondary qualitative phase comprising eight semi-structured interviews with the top management of funding agencies and representatives from other organisations closely associated with funding agencies. The interview transcripts were subjected to thematic analysis. The outcomes from this phase of the study, confirmed the quantitative research outcomes, and helped the researcher to gain a better understanding of the research subject, which guided and refined the development of both the conceptual model and implementation roadmap that are

presented in this study.

l

u:~RY)

Keywords: Organisational effectiveness; balanced scorecard; funding agencies; mixed methods

ACKNOWLEDGEMENTS

First and foremost, I would like to thank Professor Gerber for his faith and belief in me, and for always being my pillar of strength and encouraging me no matter what the hurdles were that I had to face on this journey.

I am grateful to Dr Liezel Korf, not only for her statistical prowess, but also for helping me

understand this study so much better. Her questions, observations and advice on work-life

balance were truly valued.

I am extremely indebted to my husband for his endless support as well as his unwavering love and encouragement throughout this endeavour.

I am grateful to my darling daughter, Alina, for teaching me patience, endurance and dedication.

My heartfelt appreciation is extended to my parents, Mr and Mrs Ramoutar, as well as my aunt, Mrs Ramouthar, for always taking care of the little things that would otherwise consume my day. Thanks for affording me the opportunity to be a child again and focus on completing my studies.

To my brothers, thank you for always keeping me sane and grounded.

Thank you to my in-laws, Mr and Mrs Prieschl for their support and words of encouragement. Thanks to all my family and friends-their support has been something that I will always treasure.

I would like to extend my gratitude and appreciation to my former Dean and CEO, Dr Albert van Jaarsveld for his inputs and his support of this research undertaking.

In addition, I would like to thank the National Research Foundation for financial support through

the "Employee Part-time Educational Assistance Scheme".

In addition, I would like to thank my line manager, the leadership and senior management of the National Research Foundation for accommodating my study schedule and their support throughout this journey.

I would also like to thank North West University for making the necessary supporting resources

available for the completion of this degree.

Finally, I would like to thank all the respondents and/or participants in the survey as well as the interviews - without them, this study would not have been possible.

TABLE OF CONTENTS DECLARATION ........................................ 0 ABSTRACT ....... i ACKNOWLEDGEMENTS ........................... iv CHAPTER 1: INTRODUCTION ... 1 1.1 lntroduction ............. 1 1.2 Background ......... 1 1.3 Research rationale ... 2

1.4 Research problem statement ................ 3

1.5 Research objectives ...... 4

1.5.1 Investigative questions ... 4

1.6 Scope of study ..................... 4

1.7 Study outline .................. 5

CHAPTER 2: LITERATURE STUDY ........ 6

2.1 Introduction ......... 6

2.2 Research funding ......... 6

2.3 Funding agencies ............ 6

2.3.1 Types of funding agencies ... 7

2.3.2 Practices of funding agencies ... 9

2.3.3 Challenges facing funding agencies ... 10

2.3.4 Research funding practices across several countries ... 13

2.4 Funding agencies as organisations ......... 19

2.4.1 Introduction to organisations ... 19

2.4.2 Evolution of organisational theories ... 20

2.5 The organisational ecology of funding agencies ...... 25

2.6 Organisational effectiveness ........ 26

2.6.1 Introduction ... 27

2.6.2 Evaluating organisational effectiveness ... 27

2.6.3 Summary of organisational effectiveness ... 31

2.7 Identifying the factors influencing organisational effectiveness ........ 32

2. 7 .1 Organisational learning and growth ... 34

2. 7 .2 Organisational business processes ... 55

2.7.3 Organisational customer focus ... 81

2.7.4 Organisational financial perspective ... 99

3.1 lntroduction ... 131

3.2 Research approach ... 131

3.3 Research design ............ 133

3.3.1 Quantitative research design ... 133

3.3.2 Qualitative research design ... 139

3.3.3 Ethics ... 142

3.4 Summary of the research methodology ... 143

CHAPTER 4: RESEARCH FINDINGS ...... 145

4.1 Introduction ...... 145

4.2 Quantitative data collection ........ 145

4.3 Quantitative data analysis ... 147

4.3.1 General findings for all respondents ... 14 7 4.3.2 South African findings ... 17 4 4.3.3 Comparative studies ... 180

4.3.4 Summary of findings: quantitative study ... 191

4.3 Qualitative analysis ... 193

4.4.1 Interviews ... 193

4.4.2 Analysis ... 195

4.4.3 Summary of findings: qualitative study ... 224

4.5 Integration of the quantitative and qualitative findings ... 225

4.6 Conclusion ... 226

CHAPTER 5: DISCUSSION AND RECOMMENDATIONS ... 227

5.1 Introduction ............ 227

5.2 Findings of the quantitative study ... 227

5.3 Findings of the qualitative research ... 228

5.4 Integration of the literature review with the study outcomes ... 229

5.4.1 The balanced scorecard ... 229

5.4.2 Environmental and stakeholder management ... 229

5.4.3 Organisational learning and growth ... 232

5.4.4 Financial management ... 235

5.4.5 Organisational business processes ... 237

5.4.6 Summary ... 240

5.5 Propositions for a conceptual framework ... 240

5.6 Recommendations for implementation ... 244

5.7 Revisiting the research questions ... 246

5.8 Recommendations for future research ...... 248

5.9 Limitations of this study ......... 249 vi

5.10 Conclusion ... 249

REFERENCES ... 250

ANNEXURES ... 305

ANNEXURE A: LINKING THE SURVEY QUESTIONS TO THE FACTORS AND SUBFACTORS ... 306

ANNEXURE B: ENGLISH SURVEY ... 309

ANNEXURE D: INTERVIEW PROTOCOL ... 340

LIST OF TABLES

Table 2.1: Summary of the key contributing factors that have been highlighted in the literature

to influence the returns linked to R&D investments (own source) ... 17

Table 2.2: Summary of the factors and subfactors that were reviewed and used to inform the development of the measuring instrument ... 130

Table 3.1: Summary of the data collection sources and strategies ... 143

Table 4.1: Summary of quantitative data collection ... ... 14 7 Table 4.2: Composition of the sample across the different funding agencies ... 149

Table 4.3: Duration of employment ... 150

Table 4.4: Qualification level of respondents ... 151

Table 4.5: Summary of the responses received from the survey ... 152

Table 4.6: Summary of responses per statement in the web-based survey ... 153

Table 4.7: Descriptive statistics for the five factors: all respondents ... 156

Table 4.8: KMO and Bartlett's test ... 157

Table 4.9: The factors and subfactors theoretically linked to the adapted balanced scorecard ... 158

Table 4.10: Table4.11: Principal component analysis reporting eigenvalues ...

!

...

·

NW U·

.. :

·

J

· .. 160

Factor analysis outcomes using principal axis factoring . .

LJBRA.

RY ..

.

r: Table 4.12: Descriptive statistics of the four factors ... 164

Table 4.13: Overall mean of the four factors ... 164

Table 4.14: Standard deviation and standard errors for the four factors ... 165

Table 4.15: Independent one-sample t-test... .. ... 166

Table 4.16: Factor means across the participating funding agencies using the average response mean values ... 168

Table 4.17: ANOVA results based on years of service ... 170

Table 4.18: AN OVA results based on the qualifications of all respondents ... 171

Table 4.19: Scheffe's test of multiple comparisons ... 173

Table 4.20: Duration of service: South African respondents at the National Research Foundation ... 175

Table 4.21: Qualification level: South African respondents at the National Research

Foundation ... 176

Table 4.22: Descriptive statistics for the four factors for the South African respondents ... 177

Table 4.23: Standard deviation and standard error for the four factors for the South African respondents ... 177

Table 4.24: Levene's test for testing the homogeneity of variances for South African respondents ... 178

Table 4.25: AN OVA results for South African respondents by years of service ... 179 Table 4.26: AN OVA results for the South African respondents by qualifications ... 180 Table 4.27: Composition of the sample for respondents across South Africa, Africa, Norway and Chile ... 181 Table 4.28: Descriptive statistics for the respondents South Africa, Norway and Chile ... 183 Table 4.29: Descriptive statistics depicting the factor and country means for South Africa, Norway and Chile ... 185 Table 4.30: ANOVA results for respondents from South Africa, Norway and Chile ... 186 Table 4.31: Scheffe's test of multiple comparisons for respondents from South Africa, Norway and Chile ... 188

Table 4.32: Spearman's rho correlations for respondents from South Africa, Norway and Chile ... 191

Table 4.33: Summary of major and minor themes ... 195

Table 4.34: Breakdown of the sample of participants in the interviews ... 196 Table 4.35: Summary of theme 1 ... 199

Table 4.36: Summary of theme 2 ... 206

Table 4.37: Summary of theme 3 ... 212

Table 4.38: Summary of theme 4 ... 218

Table 4.39: Summary of theme 5 ... 222

Table 5.1: The transformation of the balanced scorecard that can be used to improve the organisational effectiveness of funding agencies ... 24 7

LIST OF FIGURES

Figure 2.1: Summary of the major broad factors impacting on the effectiveness of funding

agencies ... 18

Figure 3.1: Summary of the sequential explanatory method used in the current study ... 133

Figure 4.1: Demographic profile of respondents based on years of employment at the funding agency ... 150

Figure 4.2: Demographic profile of respondents based on highest level of qualification ... 151

Figure 4.3: Summary of the responses from the survey ... 152

Figure 4.4: Factor means for the five factors linked to the balanced scorecard ... 156

Figure 4.5: Scree plot of eigenvalues ... 159

Figure 4.6: Factor means for the four factors ... 165

Figure 4.7: Factor means across the participating funding agencies using the average response ... 169

Figure 4.8: ANOVA results based on years of service: all respondents ... 171

Figure 4.9: AN OVA results based on the qualifications of all respondents ... 172

Figure 4.10: Demographic profile of South African respondents based on years of employment. ... 175

Figure 4.11: Demographic profile of South African respondents based on qualification ... 176

Figure 4.12: Factor means: South African respondents ... 178

Figure 4.13: Respondents across the three funding agencies ... 182

Figure 4.14: Factor means across the three funding agencies ... 183

Figure 5.1: A proposed conceptual framework for the organisational effectiveness of funding agencies ... 242

Figure 5.2: An implementation map for the organisational effectiveness of funding agencies ... 246

ACRONYMS ANOVA BMW CONICYT DFG e-mail EPSRC FDA FNB FT GDP GRC IBM IP IT KOSEF MOST NIH NRF NSF OECD PEST PhD R&D RCN Roi UK UN USA Analysis of Variance Bayerische Motoren Werke

Comisi6n Nacional de lnvestigaci6n Cientffica y Tecnol6gica: Chile Deutsche Forschungsgemeinschaft

Electronic Mail

European Physical Sciences and Research Council Food and Drug Administration

First National Bank Financial Times

Gross Domestic Product Global Research Council

International Business Machines Corporation Internet Protocol

Information Technology

Korean Science and Engineering Foundation

Korean Ministry of Science and Technology

National Institutes of Health National Research Foundation National Science Foundation

Organisation for Economic Cooperation and Development Political, Economic, Socio-cultural and Technological Philosophiae Doctor

Research and Development Research Council of Norway Return on Investment United Kingdom United Nations

CHAPTER 1: INTRODUCTION

1.1 Introduction

This thesis explores the factors linked to the organisational effectiveness of funding agencies across several countries and attempts to contribute towards a global discussion on funding agencies and the factors that influence their effectiveness. An in-depth exploration of the

literature guided and shaped the research design and process of this study. This thesis begins

by providing an introduction to the study, followed by an explanation of funding agencies and a definition of organisational effectiveness, prior to describing the research process, its outcomes and key findings and recommendations. The main objective of the study was to develop a

framework that could be used to guide funding agencies in improving or enhancing their

effectiveness in the science sector. Firstly, a theoretical model based on the balanced scorecard was proposed that stipulates established measures from the existing organisational effectiveness literature. This model was empirically tested as an initial step towards developing a conceptual framework geared towards improving the organisational effectiveness of funding agencies in the science sector. The conceptual framework was subsequently verified and validated through a secondary qualitative process comprising semi-structured interviews.

1.2 Background

Research and development (R&D) are considered driving forces towards establishing a knowledge economy aimed at contributing to a country's economic wealth. It is on this premise

that many countries have raised R&D investments in order to boost scientific and technological

advances over the years (Lee, Park & Choi 2009). Playing a central role in driving scientific and

technological outputs are organisations such as funding agencies that direct national scientific

research by investing in research projects that meet specific requirements and criteria (Braun

1998). Funding agencies can therefore be considered protagonists in the distribution of public resources and structure the way research is conducted by the stipulation of criteria and conditions linked to research grants (Braun 1998). Funding agencies also play a key leadership role in stimulating interest in young people to pursue careers in science and technology and developing a diverse labour force with the necessary skills to navigate in a knowledge economy {Lee, Park & Choi 2009). Given the magnitude of their mandates and the level of accountability, it is not surprising that the effectiveness of such organisations in addressing these objectives has been questioned {Lee, Park & Choi 2009).

The management of science by funding agencies, from the very outset, is influenced by and mediated in concert with the environment (Morgan 2006, Braun, 2003; Braun, 1998). In turn, these interactions are influenced by economic, political and global factors. In essence, these agencies receive inputs, primarily R&D funding from government, which, using grant awarding processes, are transformed into outputs, that are generated by those scientists who are recipients of research funding. Feedback loops are also used in such organisations and are employed to assess the impact of research funding on scientific productivity from those scientists who are recipients of grant awards (Leydesdorff, Wagner 2009, Lane 2009, Jacob, Lefgren 2007, Arora, Gambardella 2005). Unravelling the complexities associated with funding agencies, and their interactions with the environment, does not lend itself to a simple explanation.

l

In the past two decades, governments in developing countries have taken steps to improve the effectiveness of public sector organisations under the guidance of good governance and new public management, which were first introduced in the United States of America (USA) (Waheed, Mansor & Ismail 2011 ). New public management is an intervention by government that aims to formalise the interplay between government, the market and society. The philosophy of this tool is underpinned by a set of management techniques that is used to reform public sector organisations and their management (Waheed, Manser & Ismail 2011 ). In addition, new public management has been informed by the literature on good governance, which calls for more accountability and better performance (Prybil 2006). The concept of organisational effectiveness in the public sector is therefore guided by the concepts of new public management and good governance. In addition, organisational effectiveness is geared towards organisations becoming more efficient, effective and relevant to the needs of customers and other stakeholders (Waheed, Mansor & Ismail 2011, Peterson et al. 2003, Lusthaus 2002). Finally, it has been proposed that the provision of public access to regular and reliable information not only increases transparency, but also improves public accountability (Waheed, Manser & Ismail 2011, Peterson et al. 2003, Lusthaus 2002). For improvement to be the desired outcome, when measuring the effectiveness of public organisations, it is important for customer satisfaction to be a key performance measure (Waheed, Manser & Ismail 2011 ).

1.3 Research rationale

Funding agencies must identify, link and align resources with external sectors to be better able to meet the needs of the science community. This includes brokering partnerships with industry,

such partnerships include the cross-fertilisation of ideas, access to expertise and technology,

the improvement of services and the increased ability to acquire additional resources. Forging

such partnerships also attempts to ensure the pursuit of academic excellence as well as the

incorporation of the needs of all stakeholders into the organisation's strategic planning. Such

interactions have the potential to reshape national science policy (Tiu 2001 ).

Organisational effectiveness has been widely studied in the private sector but to a lesser extent

in the public sector. In the latter case, a lot of research undertakings have focused on the

effectiveness of government departments, universities and non-governmental organisations.

What is lacking in the literature is specific attention to the effectiveness of organisations such as

funding agencies.

This doctoral study is therefore not only a unique undertaking but also aims to investigate the effectiveness of funding agencies in the science sector by assessing the perceptions of funding

agency employees. Its findings will be used to develop a conceptual framework that hopes to

contribute to a global discussion on the effectiveness of funding agencies and their role in influencing scientists to be more productive, while simultaneously addressing the demands of

government. To address this research focus, the study was guided and shaped by the literature

on organisational effectiveness.

1.4 Research problem statement

Few studies have attempted to measure the effectiveness of public organisations and these

studies have focused on analysing the effectiveness of hospitals (Prybil 2006, Kershaw,

Kershaw 2001 ), municipalities (Ngcelwane 2008), universities (Mukerjee 2014, Rasheed et al.

2011, Zangoueinezhad, Moshabaki 2011, Owsley-Stevens 2010, Thiruchelvam 2004 ),

non-governmental organisations (Fowler 2013, bin Haji Zain 2010) and government or government

departments (Khan, Ukpere 2014, Meyer 2014, Stanton, Webster 2014, Hoque, Adams 2011 ).

Even fewer studies have attempted to investigate the effectiveness of public funding agencies,

of which a number of these have either focused on the effectiveness of specific funding

programmes or developed various methods to evaluate research productivity (Leydesdorff,

Wagner 2009, Lane 2009, Jacob, Lefgren 2007, Arora, Gambardella 2005). This study is a

unique undertaking in that there is limited, if any, reference in the literature, to understanding

the organisational effectiveness of funding agencies, particularly in middle-income countries, by

identifying a useful, reliable and valid set of factors for evaluating the organisational

effectiveness of funding agencies. This study also contributes towards the development of an conceptual framework geared towards both understanding and improving the effectiveness of these agencies.

1.5 Research objectives

The aim of this research study was to identify a useful, reliable and valid set of factors for evaluating the organisational effectiveness of funding agencies using the balanced scorecard

and a mixed methods approach. The outcomes of the study aided in the development of an

conceptual framework geared towards contributing to the global discussion on the effectiveness

of funding agencies. It also makes strategic recommendations, in the form of an

implementation map, to government and funding agencies on how best to address the factors

that influence the effectiveness of funding agencies.

1.5.1 Investigative questions

1) Do the factors and subfactors cluster according to the balanced scorecard that has been proposed in this study?

2) What are the main factors influencing/impacting the organisational effectiveness of funding agencies?

3) Can a cocneptual framework for improving the effectiveness of funding agencies be developed?

1.6 Scope of study

This study focused primarily on the perceptions of employees about the organisational effectiveness of funding agencies, across several countries using a mixed methods approach. Although the scope of this study originally extended itself to include the perceptions of funding

agency employees, government officials and researchers, the leadership of the few funding

agencies that were agreeable to participating in this study, indicated that the scope must be

limited to the perceptions of funding agency employees only, and exclude those of government

officials and researchers. The study therefore did not extend to evaluating the perceptions of

other stakeholders, such as government officials and researchers who are recipients of grant

1.7 Study outline

The outcomes of this doctoral thesis will be described in six distinct and demarcated chapters.

Chapter 1 introduced the study and provided the rationale for the proposed research

undertaking. Further, it focused on the contextual background to the balanced scorecard, which

framed the research approach in this study and briefly introduced funding agencies. The

research problem statement and the investigative questions that this study endeavoured to

answer were also discussed.

Chapter 2 contains the main literature review and provides a comprehensive overview of the

theoretical concepts underpinning this study. In addition, it describes the theoretical factors and

subfactors as well as their impact on the organisational effectiveness of funding agencies.

Chapter 3 describes the research methodology, approach and design of the study. It also

describes the sample population and the sampling procedures followed. Finally, this chapter

presents the method for data collection as well as the data analysis tools that were employed.

The issues pertaining to reliability, validity and ethical considerations are discussed.

Chapter 4 expands on the research findings and provides the statistical data that resulted from

undertaking this research study. The outcomes presented here determine whether the research

questions were adequately answered.

Chapter 5 includes an interpretation of the research findings by integrating the literature review

with the study outcomes. In addition a conceptual framework is presented, that highlights the

key factors associated with the organisational effectiveness of funding agencies. The

developed framework is discussed in detail in this chapter. It also includes recommendations to

government and funding agencies, in the form of an implementation map, on how best to

improve the organisational effectiveness of funding agencies. The limitations of the study are

acknowledged and discussed in this chapter.

CHAPTER 2: LITERATURE STUDY

2.1 Introduction

This chapter focuses on three bodies of literature. Firstly, funding agencies and funding

practices across several countries are discussed. Secondly, organisational theory is briefly

discussed. As part of this discussion, reference is made to the relevant literature in describing

the balanced scorecard, which forms the basis of the development of the survey instruments

that were used in this research. Thirdly, the literature relating to organisational effectiveness is

visited. As part of the conclusion to this chapter, the complexity associated with understanding

the organisational effectiveness of research funding agencies is unravelled by identifying and

describing the various factors and subfactors that were discussed and presented in the

literature.

2.2 Research funding

A focal point of this research study is on research funding due to the considerable investments

that are directed towards funding R&D activities across the R&D value chain with the view that

the long-term direct and indirect benefits will accrue economic returns (Lane, 2009; Leydesdorff

& Wagner, 2009; Goldfarb, 2008; Boardman & Ponomariov, 2007; Jacob & Lefgren, 2007;

Arora & Gambardella, 2005; Ruegg & Feller, 2003). R&D activities are usually funded from

two major sources, private enterprise and government (OECD, 2008). The funding of research

by private enterprises is primarily motivated by profit, which incentivises researchers to

concentrate their energies on projects that are perceived to be profitable. Hence private sector

investments in R&D tend to be focused on development and commercialisation projects (OECD,

2008). Government-funded research, however, facilitates the generation of new knowledge and

fosters collaborative networks (Defazio, Lockett & Wright, 2009). With minor exceptions,

government tends to invest significantly in basic and applied scientific research (OECD, 2008).

In many instances, government investments in R&D are managed by funding agencies which

play a key role in the disbursement of these funds in the form of research grants (Goldfarb,

2008; Braun, 2003; Van der Meulen, 2003; Braun, 1998; Van der Meulen, 1998).

2.3 Funding agencies

Research funding agencies are quasi-public organisations mandated by specific legislation and

are accountable to a board of directors (Braun, 1998). Although they are independent entities,

(Hodgson & Calatrava, 2006). Although funding agencies exist in the midst of a system compounded by numerous external players, their responsibility is to optimally utilise limited national resources. The resources invested by funding agencies, using grant awarding processes, drive the research process and encourage research productivity by the scientists who are recipients of research grants (Defazio et al, 2009, Jacob & Lefgren, 2007; Rowlands & Olivieri, 2006; Arora & Gambardella, 2005).

The mechanism for funding research usually involves researchers applying for financial support in the form of research grants from funding agencies. This involves funding research projects through a competitive process, which involves the evaluation of projects against set criteria by a peer review committee of experts in a specific field of research (OECD, 2008). The awarding of

these grants entails a lengthy process as the funding agency enquires about the researcher's

background, the facilities used, the equipment needed, the time involved, and the overall potential and impact of the scientific outcome. Ultimately, only the most promising projects receive funding (OECD, 2008).

As a consequence, these public organisations possess the power to structure, constrain and enable specific behaviours that address national strategic priorities (Hodgson & Calatrava, 2006). This type of behaviour tends to be driven by government departments that form the primary sources of financial investment to funding agencies. The respective government departments allocate financial resources to the funding agency on the basis that specific strategic objectives must be met and key performance areas addressed (Braun, 1998). However, the level of government and political authority influencing the role and mandates of funding agencies differs because almost all funding agencies have a considerable amount of

latitude in determining and implementing their strategies (Braun, 1998).

2.3.1 Types of funding agencies

In attempting to understand the various types of funding agencies that have been reported in the literature, two differing perspectives are presented. In 2007, Lepori and co-authors, proposed a classification system relating to the nature of funding agencies with their relationships to government. The authors proposed four major types of funding agencies

(Lepori, Van den Besselaar, Dinges et al, 2007). These included national agencies that form a

direct part of government administration, intermediary agencies that are independent in their management and decision-making processes, regional agencies that are part of the local state

administration and have regional relevance, and international agencies that form part of the

European Union, such as intergovernment agencies.

An alternate categorisation system was suggested earlier by Braun (1998), which related the

type of research that funding agencies aim to steer, with the level of influence of national

government on their activities. In essence, Braun (1998) identified three types of funding

agencies. Firstly, he proposed science-based funding agencies that support all disciplines of

science as individual silos. This type of funding agency fosters mainstream research by

selecting and funding reputed scientists in a disciplinary arena, thereby avoiding the rise of

hybrid communities that could compromise the existing distribution of power in the scientific

system (Braun, 1998). Examples include the German Research Community, the American

National Science Foundation and the French Centre National de la Recherche Scientifique.

The second proposed type of funding agency includes political funding agencies that are

obliged to respond to the general and multifaceted problems raised in parliament or ministries.

These are rare funding agencies and an example includes the German Ministry of Education

and Research (Braun, 1998). Political agencies, according to Braun (1998) are forced to create

hybrid communities because they need unconventional solutions to politically defined problems.

Such agencies can very well lead to the differentiation of the science community (National

Research Council, 2005; Braun, 1998). Thirdly, Braun (1998) proposed strategic funding

agencies that promote research in a particular problem area such as health and environment.

Such agencies also respond to particular problems raised by the scientific community,

laypersons, industry or politicians. Strategic funding agencies not only promote disciplinary

research, but also develop strategies to apply basic research in a wider context and, in most instances, support clinical research that is often neglected by science-based funding agencies

(Braun, 1998). Examples of such agencies include the various research councils in the United

Kingdom (UK), Grands Organismes de la Recherche in France and the National Institute of

Health in the United States of America (USA).

l

NWU

I

LIBRARY

·

Despite the differences between the various types of funding agencies as described in the

literature, it can be deduced that the role of all funding agencies, regardless of their mandates

and allegiance to national government, is essentially based on the same principles - that is, to

ensure that the needs of the research community are addressed and that competitive evaluation

practices are adopted to ensure that there is a return on investment in R&D that aligns with

2.3.2 Practices of funding agencies

In managing the public investment in R&D, several approaches have been utilised to evaluate scientific or research proposals that are submitted to funding agencies by research applicants. The main criterion, which forms the cornerstone for the evaluation of proposals, relates to scientific merit, which is measured by the research productivity of the applicant (Rowlands & Olivieri, 2006; Braun, 1998). Productivity can be defined as the measure of scientific activity

that is able to generate various quality research outputs, particularly research publications

(Rowlands & Olivieri, 2006). The productivity of applying researchers can be measured by using

the quantitative method of bibliometrics. According to Ruegg and Feller (2003), bibliometrics is defined as the quantitative study of literature as reflected on bibliographies and utilises quantitative analysis and statistics to describe patterns of publication in a given field or body of

literature (Weingart, 2005). The challenge funding agencies face in employing this method is

that bibliometrics only addresses a single type of R&D output, namely publications (Lee et al, 2009). This method therefore poses a challenge to funding agencies, particularly in developing countries that are required by government to consider and report on other research outputs that

form part of national performance indicators, including student training and innovative outputs

that have the potential to draw economic benefits to the country.

A commonly employed method utilised by funding agencies to evaluate the merit of research proposals, is the peer review method, which is based on the perceptions of well-informed experts on various quality dimensions of R&D (Ruegg & Feller, 2003). Referees are co-opted by the funding agency and usually two referees present a research proposal to the rest of the participants of the peer review group (Braun, 1998). This opens the floor to dialogue and opposing views by the other panel participants. The research proposal is usually rated against set assessment criteria defined by the funding agency. Although peer reviewers can gain consensus on proposals that are either outstanding or poor, it is difficult to reach consensus on proposals that score in the middle range, which is a major limitation associated with the peer review system (Weingart, 2005). An additional limitation is that this method is both costly and an inherently subjective one that is likely to be biased, depending on the interests, experience, competence and knowledge of the evaluators (Lee et al, 2009).

Another method, employed by funding agencies, but to a lesser extent than the peer review process, is the anonymous mail review. Anonymous referees or reviewers are invited by funding agencies to decide on the credibility of the proposal and the research applicant in accordance with the same set of assessment criteria that are used in the peer review process. In the mail review system, the referee makes a decision by himself or herself without being exposed to the

opinion of other reviewers (Lee et al, 2009). Usually two or three mail reviews are requested on

the same research proposal in order to compensate for opposing views. The final decision on

the investment potential of a proposal is then made by funding-agency personnel.

Each of the above-mentioned practices form part of the core business of funding agencies and

possess certain drawbacks that need to be weighed. As such, some funding agencies have

developed alternate methods of measurement. These include computer-modelling programs

such as data envelopment analysis {Lee et al, 2009), which allow funding administrators at

funding agencies to assess all criteria involved in the decision-making process. Such techniques are widely utilised by the US and European funding agencies (Lee et al, 2009; Jacob & Lefgren, 2007; Jaffe, 2002). However, these too suffer from limitations in that all the historical data on the applicant needs to be electronically available and the software program

itself is expensive to purchase (Jaffe, 2002). Further, these programmes need to be adapted to

ensure alignment to each country's national funding priorities (Weingart, 2005).

The challenge with the use of any of the above-mentioned methods, is that each practice can

be either time consuming or expensive, or both. Nevertheless, in the absence of generally

accepted or agreed methodologies, funding agencies continue to base their grants awarding

processes on peer review, despite this method being open to both questions and flaws

{Leydesdorff & Wagner, 2009).

2.3.3 Challenges facing funding agencies

In addition to the criticisms from the research community, relating to the processes adopted by

funding agencies to award research grants, these public organisations also face several other

challenges.

Firstly, there is a worldwide drive across many countries to adopt new public management,

which aims to make public agencies more transparent, especially with respect to their

expenditure behaviours by applying a set of performance measurement indicators. Performance

management tools attempt to make objectives, performance outputs and resource utilisation

clear in public sector organisations, including agencies, by integrating financial and nonfinancial

information (De Waal, 2010). The new public management tools however, have been received

with mixed reviews. On the one hand, these tools have been shown to improve overall

2010). Furthermore, it appears that across public funding agencies, the use of these tools has not as yet been fully implemented and explored, especially with regard to their alignment with the nature and complexity associated with the mandates of funding agencies.

Secondly, a major contributor impacting on the effectiveness of funding agencies is the external environment and its management by agencies themselves. Funding agencies tend to exist in a dynamic environment driven by the need for resources and accountability (Braun, 1998; Abernethy & Brownell, 1997). The external environment that impacts on the effectiveness of

funding agencies involves two major stakeholders, namely government, on the one hand, and

the scientific community, on the other. Funding agencies by virtue of their processes and their

mandates significantly influence the scientific community of a country by determining those who will be recipients of grant awards (Braun & Guston, 2003). However, in delivering on the demands of the scientific community, they themselves need to address the demands of the political environment that is accountable for the organisation's financial resources (Wagner, 2011 ). In essence, while dealing with educational and research management, funding agencies face the challenge of converging the interests of scientific and political stakeholders (Braun, 2003).

A third challenge is the need for funding agencies to remain autonomous. Although they act as

intermediary organisations between government and universities, which include the scientific

community, these agencies require a certain level of autonomy with respect to both sides. Part of the influence on funding agencies by the scientific community is explained by the peer review process, the primary instrument used to evaluate and select fundable projects. Despite the role of scientists in advising funding agencies, through the peer review process, of what projects are fundable, the positions of funding agencies and scientists in the quest for the distribution of funds are different (Braun, 2003). Furthermore, funding agencies represent independent organisations that are governed by specific mandates, and are not mere agencies that aspire to fulfil the demands of the scientific research community alone. The basic interest of funding agencies therefore is to maintain the organisational capacities for organisational survival, and this not only demands a close relationship with scientists and government, but also a certain degree of independence. Boundaries are therefore necessary and need to be maintained in order for these organisations to operate effectively and efficiently (Braun, 1998).

Compounding the above challenge is the fact that boundary lines are often blurred owing to the inextricably interdependent relationship that funding agencies share with the external environment. A case in point is the recent practice of funding agencies to replace personnel with

scientific staff (Braun, 2003; Braun & Guston, 2003; Braun, 1998). The reasons for this are

twofold. Firstly, the matters of funding agencies are so specific that they require a reasonable

knowledge of the scientific field; and secondly, in order to succeed, funding agencies require

personnel with good contacts in the scientific community who are familiar with the values, norms

and procedures of scientists (Braun, 2003; Braun, 1998). This institutional symbiosis means

that no goal can be achieved without the active cooperation of both sides. However, these

agencies are under growing pressure from government to demonstrate greater accountability and accomplish more with fewer resources, in order to improve their organisational productivity

and enhance their impact on the research-productivity capacity of the country (National

Research Council, 2005). This interplay has numerous implications for the role played by

funding agencies and has raised many concerns about the efficiency and effectiveness of R&D

spending by funding agencies. Compounding this issue is the absence of a universal indicator

that measures the return on R&D investments.

Drawing on the above-mentioned list of challenges that face funding agencies, it is not

surprising that South Africa faces similar constraints. The South African National Research

Foundation is considered to be an intermediary agency between the policies and strategies of

the South African government and research performing institutions or universities. The

objective of the South African National Research Foundation is to "promote and support

research through funding, human resource development and the provision of the necessary

facilities in order to facilitate the creation of knowledge, innovation and the development of all

fields of science and technology" (South Africa, 1998). In 2010, an institutional review of the

National Research Foundation was conducted, which highlighted that the government agency's

effectiveness was constrained by its decision-making structures and the allocation of its

resources in balancing grants and overhead costs have impacted on both its efficiency and

effectiveness in the science sector (Cruz, Duncan & Ward, 2011 ). Given that the review was a

consultancy report, no proposed clear solution was recommended for the funding agency to

enhance its effectiveness. Across other developing countries, the challenges associated with

the effectiveness of public sector organisations include the following: poor allocation and

utilisation of resources; weak management practices; weak research-performing institutions;

unpredictable political, economic and ethical environments; and poor morale and motivation

among public employees (Waheed et al, 2011 ).

Given this context, it is difficult yet critical to establish a useful, reliable and valid set of effective

that emerge from the literature review, that have been pinpointed as playing a significant role in

influencing the effectiveness of public sector organisations.

2.3.4 Research funding practices across several countries

This section describes research funding practices across several countries, which appear to be

largely influenced by political demands and economic constraints. In addition, understanding the

state of a country's funded research system, requires due acknowledgment of its goals,

research choices, policies, outcomes and challenges. The direct involvement of government in

research decision-making is growing, and this, in most instances, has led to the establishment

of agencies that appear to closely link funding to specific goals (Niven, 2008).

The first example is the USA, which during World War II, enjoyed great success in harnessing

its then, fairly small science and technology cohort, to develop new weapons and meet military

needs (National Research Council, 1999). At the end of the Cold War, there was a shift in

thinking towards pushing resources to support R&D activities. Over the years, the USA has

significantly increased its investments in R&D and its subsequent scientific and technological advances have positioned it as a global leader.

However, in 2006, Bybee and Fuchs (2006) conducted an assessment that revealed that although the USA has led the world in scientific discovery and technological innovation for over a century, nations from Europe to East Asia were on a fast track to exceeding the USA's performance to date. The challenge the R&D sector in the USA faces is declining investments, as a percentage of Gross Domestic Profit (GDP), in the natural and physical sciences, as well as mathematics and engineering (Bybee & Fuchs, 2006). From 1994 to 1998, the number of Asian students who chose to pursue their doctoral studies at US universities, declined by 19%. At the same time, the number of students who chose to pursue doctoral studies at universities in their own countries nearly doubled. This, according to Bybee and Fuchs (2006), indicated that

these countries, particularly, China, South Korea and Taiwan, were quickly growing their own

higher educational capabilities that appeared to successfully retain their skilled youth. The USA share in research articles published worldwide declined from 38% in 1988 to 31 % in 2001. Europe and Asia, however, were responsible for a major portion of the growth in scientific papers in this time period (National Research Council, 2005). Another finding was that, the

world's fastest-growing economies were on track to achieving R&D investments on par with the

USA. China, South Korea and Taiwan increased their gross R&D investments by about 140%,

from 1989 to 2001, while the USA only increased its investment by 34% (Bybee & Fuchs, 2006).

The decline in R&D investment in the USA has impacted on the performance of funding agencies. The National Institute of Health (NIH) and the National Science Foundation (NSF) are the two agencies that allocate almost US$30 billion annually to the basic and applied sciences (Jacob & Lefgren, 2007). Since 2000, there has been a decline in the annual budget allocation by government of these two funding agencies and the associated consequence was a decrease in the average grant award size, which was not large enough to support a large portion of promising research projects (Arora & Gambardella, 2005). Such cutbacks have played a significant role in contributing to the USA's performance during this period {Leydesdorff & Wagner, 2009).

A similar trend was observed in Australia. In a publication by the Organisation for Economic Co-operation and Development (OECD) in 2008, it was reported that Australia's performance in terms of scientific and technological activity in the higher education sector was declining because of a smaller budget for R&D (Quirk, 2004). This was substantiated with data from 2001 when Australia's investment in R&D was 1.53% (Quirk, 2004). At this stage, Sweden had the highest investment in R&D as a percentage of GDP, at 4.27%, followed by Finland (3.4%), Japan (3.09%) and the USA (2.82%) (Quirk, 2004). Subsequent to 2001, however, increases in business sector investment in R&D were noted, and this has managed to offset Australia's modest research performance in the public sector (Bybee & Fuchs, 2006). Hence Australia has continued to enjoy a considerable share of world publications, which increased steadily from 2.85% in 1999 to 3.18% in 2008 (OECD, 2008).

Finding innovative solutions to addressing declining national R&D budgets, however, is not necessarily a feat that has been successfully achieved in some middle-income countries. Although Argentina has highly-trained human resources, some of whom have received several Nobel science prize awards, its skilled research staff is aging and there is limited development in many of the scientific disciplines (Villegas, 2013). Argentina also faces the challenge of having few linkages between its academic and private sectors and a fairly limited number of research activities at universities are able to meet the needs of the production sectors. These are the consequences of a complex history associated with the political and economic landscape in Argentina (Ribeiro, Albuquerque, Franco & Moura, 2009). From 1991 to 1998, Argentina experienced high growth in GDP, which was briefly interrupted by the tequila crisis of 1995. In late 1998, the economy entered into a stagnation period, which was followed by a deep decline in GDP in 2001 and 2002, in the midst of a huge financial crisis (Chudnovsky, 1999). These economic challenges have had dire consequences for the nation's scientific

articles published from 197 4 to 2008. During this time, other Latin American countries such as

Brazil, Mexico and Costa Rica had an increment of 3 449%; 1 440% and 1 153% respectively

(Ribeiro et al, 2009). However, patent production was described as a linear trajectory for Brazil,

which experienced a rise of 171 % in the number of patents granted by the USA Patent and

Trademark Office (Ribeiro et al, 2009). Argentina experienced a decline in the number of

patents granted, which dropped from 60 in 1974 to 51 in 1982. The country's expenditure on

R&D as a percentage of GDP revealed that in 2006, it spent 0.46%, and this investment

supported a headcount of 3.37 full-time equivalent scientists per thousand people.

In comparison, South Korea another middle-income country, experienced growth rates that

were regarded as impressive in terms of publications and patents (Ribeiro et al, 2009).

Whereas in 197 4, South Korea was classified as a country in which the scientific infrastructure

was too immature for minimum technological production, by 1998 this country was

characterised by its consistent connections and interactions between science and technology.

An analysis of the trends in Asian countries indicated that during this time period, there was a

higher and regular increase in its scientific and technological production (Ribeiro et al, 2009).

Although starting their trajectory at a higher level than South Korea, Latin American countries,

however, were not capable of reaching the status that South Korea had achieved between 1998

and 2006.

l

NWU

I

LIBRARY_

The success associated with South Korea can be attributed to the establishment of the Ministry

of Science and Technology (MOST), which in 1967 committed this country to the long-term

development and funding of science and technology, including support for research (Hwang,

1990). The ministry is responsible for several funding agencies, including the Korean Science

and Engineering Foundation (KOSEF), which focus on addressing the loss of skilled individuals

to Western countries, particularly in the USA. KOSEF's effort to reverse the brain drain is highly

concentrated on providing motivations and incentives for the recruitment of skilled scientists and

engineers in public sector R&D institutes. With this intervention in place, there has been an

influx of skilled workforce returning to South Korea. The intervention implemented in South

Korea was characterised by dual approaches that focus on researcher motivation and reward

systems. Firstly, it aimed to attract skilled Korean scientists and engineers back home, and

secondly, to create a conducive environment to protect professionalism (Yoon, 1992), which

included revising the salary scale for repatriates, to be on par with cabinet and National

Assembly members. Interestingly enough, the private sector piggy-backed on the interventions

initiated by government and offered incentives such as a "PhD allowance" on top of a basic

salary for employees pursuing doctoral studies. Furthermore, laws were passed to encourage

engineering services and industry sector R&D as well as ensure that R&D in the public sector is protected (Yoon, 1992).

In each of the cases presented above, there appears to be an observed enforcement of measures to ensure the efficient use of public resources, which clearly articulates the demands from government for a higher level of accountability with regard to the use of public funds. In addition, there is a strong push towards ensuring that research funding is linked to research productivity; key performance indicators, that are defined upfront in contracts with government departments; and the introduction of research funding instruments that are aimed at addressing political priorities (Lepori et al, 2007; Lepori, 2006).

In summary, it appears that across various countries, the evaluation of the effectiveness of research funding either through private sector, government or agencies needs to be interpreted with respect to stakeholder mandates, their level of investments, national policies, political and socioeconomic priorities, institutional structures and stakeholder networks in specific countries (Lepori et al, 2007). These challenges are tabulated in Table 2.1 below.

Table 2.1: Summary of the key contributing factors that have been highlighted in the literature to influence the returns linked to R&D investments (own source)

Contributing factor Associated criterion Country

Government investments USA

South Korea Australia

Financial investments Brazil

Private sector investments _Mexico

European countries Calibre of staff and students USA

South Korea Australia Higher education capabilities

Research productivity Brazil

Mexico

European countries

Feeding innovative R&D outputs to the private

sector for development and commercialisation South Korea

Academic-private sector opportunities Australia

linkages Brazil

Developing skilled graduates that are European countries absorbed by the private sector

Legislative environment that is supportive of USA

R&D investments South Korea

Political and economic climate

Long-term commitment from government, that Australia

(fiscal policies and science Brazil

policies) articulates the strategic vision for R&D Mexico investments in areas of identified national

priorities European countries

South Korea Products and services supporting the long- Australia

Customer products and services Brazil

term vision ensures sustainability is addressed

European countries South Korea Organisational processes The unifying link to other factors ensure Australia

effective delivery on R&D spending Brazil

European countries

In each of the above case studies, there are linkages to the political and economic climate that appear to have a strong influence on the ability of a country to steer science forward.

Based on the summary provided in table 2.1, there is a reaffirmation that activities in the

external environment, namely the national and global arenas, play a key role in influencing the

effectiveness of R&D spending.

Finllll:ial Perspective Effectiye R&D Funding Business Processes

Figure 2.1: Summary of the major broad factors impacting on the effectiveness of funding agencies

Furthermore, emanating from the discussion thus far, there appears to be a lack of literature

relating specifically to understanding the effectiveness of funding agencies in terms of delivering on their mandates, which in turn ties in with their delivery on performance indicators linked to

R&D investments. This further substantiates the uniqueness of the current research studies.

Thus far in this chapter, there has been a strong focus on the role of funding agencies and the contextual environment relating to R&D investments across several countries. The remainder of this chapter therefore focuses on the factors that have been implicated in influencing the

effectiveness of organisations in general. In summary, the rationale is to attempt to marry the

2.4 Funding agencies as organisations

In order to begin the process of identifying and understanding the factors that influence the

effectiveness of funding agencies, it is imperative to understand two key elements, namely that

(1) funding agencies are organisations; and (2) organisations do not exist in a vacuum. All

organisations exist in a contextual setting that needs to be clearly understood, prior to proposing

any framework for enhancing their effectiveness. It is therefore imperative that time is afforded

to briefly describe the literature on organisations and organisational theory.

2.4.1 Introduction to organisations

An organisation, by virtue of its definition, is a structured social system, composed of individuals

operating within an organised system of parts that are highly integrated in order to achieve an

overall goal (Firth, 2013; Parsons, 2013; Greenberg, 2010). Studies in the field of

organisational and management sciences, commonly involve the assessment of two

fundamental concepts, namely organisational effectiveness and organisational development.

Organisational effectiveness is a measure of the extent to which an organisation is able to

realise its goals using limited resources (Matzler & Abfalter 2013; Cameron, 1978).

Organisational development, however, is commonly viewed in the literature as an improvement

strategy that increases effectiveness at the individual and organisational levels, through applied

behavioural science principles and practices (Cummings & Worley, 2014; French, Bell &

Zawacki, 2000; Pasmore, 1988) In addition, this concept refers to the attempts made by

organisations to be better able to attain their short- and long-term objectives (French et al,

2000).

When analysing organisations as integrated systems, with due consideration afforded to the

concepts of organisational effectiveness and development, the fundamental approach to such

analyses tends to be informed by expert social inputs that suggest that organisations are

established in order to better serve human needs (Giddens, 2013; Berger & Luckmann, 1966).

This view provides a description fitting of funding agencies that are mandated by government to serve the needs of scientists, the majority of which are based at universities in a country. It is

perhaps relevant therefore to trace the evolution of the theories informing the analysis and

understanding of complex organisations.