Monitoring in Public Service Triads

(1)

MASTER THESIS

Monitoring in Public Service

Triads

University of Groningen, Faculty of Economics and Businesses,

MSc Supply Chain Management

July 6, 2015

By

S. Westrik

Student number: 2576554

E-mail:

S.Westrik@student.rug.nl

(2)

2

PREFACE:

There are many people I would like to thank for supporting me during the process of writing my thesis. First of all, I want to express my gratefulness to dr. Manda Broekhuis, for always providing me with useful feedback, to plan spare with time for discussions and drawing-sessions, and to participate in an interactive session at the RIGG. In addition, I would like to thank dr. Taco van der Vaart, for his critical review and suggestions. Beside the supervisors, I also would like to thank Ineke Jansman (co-researcher) for providing her feedback on my research proposal.

Secondly, I would like to thank the RIGG, for offering room and time. Without their support, I was not able to collect data. In particular Janine Groeneveld, Bert van Krieken and Grietje Kalsbeek for their time and effort to enable this research. I also would like to thank Tom Grevers and Joel Zwart for their useful ideas and critical review.

Last, but certainly not least, I would like to thank my family and friends for their support during the process of writing my final thesis.

(3)

3

ABSTRACT:

This research focuses on designing a monitoring instrument that facilitates buyers in a service triad, in monitoring outsourced public service. In order to design a monitoring instrument, the specific goals and requirements of the monitoring instrument are identified. Further, the role of subcontractors and buyers’ customers in contract monitoring, in service triadic setting, are clarified.

A design method study has been conducted to design a monitoring instrument. Derived from literature, a list of first goals and requirements for a monitoring instrument were identified. Data was collected via interviews, field notes and small talks and an interactive meeting with two representatives of the RIGG. The interviews were conducted with all triadic stakeholders; healthcare providers, clients and municipalities. Moreover, monitoring experts were interviewed to include their monitoring expertise and prior experience in monitoring Dutch youth care. Via an interactive meeting with two representatives of the RIGG it was attempted to find patterns in the stakeholders’ perspectives on the design requirements and the design itself of the monitoring instrument. Subsequently, the goals and requirements and identified patterns were translated in a first design of the monitoring instrument. The first design of the monitoring instrument comprises four steps; (1) goals of monitoring, (2) key decision areas, (3) selecting indicators and (4) process requirements in monitoring. The latter step includes the following process requirements; the exploitation of monitoring capacity, the frequency and intensity and the role of subcontractors and buyers’ customers in monitoring. The second step, key decision areas, is subdivided into four areas. Each key decisions area represents a contingency perspective on the monitoring instrument, in which users have to select what to monitor. Hence, this monitoring instrument not only includes subcontractors’ performances, but also transformation targets developed by the RIGG and monitoring risk. In this way, the monitoring instrument is characterized by its comprehensiveness.

(4)

4

List of figures

Figure 1 Process of designing a monitoring model ... 9

Figure 2 Service Triad (Van Iwaarden & Van der Valk, 2013) ... 9

Figure 3 Purchasing process (van Weele, 2014) ... 11

Figure 4 Design science research process model (Peffers et al., 2006) ... 15

Figure 5 Basic actions in designing (Van Aken, Berends & van der Bij, 2012) ... 17

Figure 6 Goals of monitoring ... 22

Figure 7 First design monitoring instrument ... 36

Figure 8 Key decision areas ... 49

Figure 9 Transformation targets (RIGG) ... 50

List of tables

Table 1 Stakeholder interviews ... 18

Table 2 Quality of the research (Yin, 2009) ... 20

(6)

6

1. INTRODUCTION

Organizations are increasingly outsourcing products and services that become part of their value proposition to their customers from subcontractors (Van Iwaarden & van der Valk, 2013; Domberger, 1994). Outsourcing consists in transferring those products or services to a third party subcontractor and controlling the sourcing through contracts (Roberts, 2001). Settings, in which the buyer contracts a subcontractor, and the subcontractor delivers services directly to the buyers’ customer, are described as service triads (Li & Choi, 2009). The buyer in this triadic setting lacks direct control over service delivery since subcontractors directly deliver to the buyers’ customers. Therefore, the buyer depends on subcontractors for its business performances; poor performance from the side of the subcontractors immediately affects customer satisfaction (Van Iwaarden & van der Valk, 2013). Contracts and monitoring activities are often used by buyers to ensure that the services provided and the behavior of subcontractors are appropriate (Van der Valk & Van Iwaarden, 2011). But, buying organizations may have limited service-related knowledge, which is the extent to which the buyer has knowledge about the task being outsourced (Ouchi, 1979).This may especially be true in case of professional services that’s been characterized by high levels of unique knowledge, specialization and customization (Fitzsimmons et al., 2014). Subsequently, buyers may be unable to precisely define their needs and requirements from the outset of the contracting process and might not know what standards of practice to apply to monitor and evaluate the service (Selviaridis & Spring, 2013; Sharma, 1997). In such circumstances, subcontractors may not only determine what needs to be done and how it should be delivered, but also decide or be involved in the decision whether certain standards have been met (Sharma, 1997). In situations in which a professional service is delivered, developing detailed specifications and measures is challenging and the perceived risk of supplier opportunism increases (Ellram et al., 2008). Therefore, this research will focus on designing a monitoring instrument, that can be used to produce evaluative data in a professional service triad. This instrument can be used in determining what monitoring activities to put in place in order to evaluate subcontractors’ actions and performances.

(7)

7 have prior experience related to the service to be outsourced. Besides, buyers perceived uncertainty is often extensive when they face highly complex outsourcing task – such as professional services - due to their limited knowledge (Selviaridis & Spring, 2013). In addition, subcontractors may fulfil a substantial role in subsequent monitoring activities. Subcontractors for instance can periodically perform reviews regarding the selected performance indicators (Wholey & Hatry, 1992). However, little is known about the role of the subcontractors in these contract management activities yet, and more specific the monitoring activity. A better understanding of this is essential for buying firms in order to manage buyer-subcontractor interactions (Van der Valk et al., 2011).

Due to the characteristics of professional services, customer inputs are always required for service processes to take place (Sampson and Froehle, 2006). Because the customers are the providers of input but also recipients of the buyers outsourced services they might provide valuable insights regarding the outsourced service, for instance what customers’ exactly need, what they want to deliver on their own and how they experience the offering. In order to enhance control of the outsourced delivery, buying firms should therefore continue to interact with the end-customers (Li & Choi, 2009), also in the monitoring of subcontractors’ performances.

(8)

8 However, Dutch municipalities experience difficulties in regulating the transferred health care services, due to insufficient service-related knowledge and capacity (Yacht, 2015). This is also acknowledged by the report of VNG (2014) who concluded that municipalities in the Netherlands want to increase the involvement of clients in monitoring functions to create a better outlook on the provision of the outsourced health care services. This research will take place at the RIGG (Regionale Inkooporganisatie Groninger Gemeenten). The RIGG is responsible for contracting and monitoring healthcare providers, on behalf of 23 Dutch northern-region located municipalities. Main goal of the RIGG is to purchase ‘good’ health care services for a reasonable price against good conditions.

(9)

9

2. THEORETICAL BACKGROUND

The main aim of this research is to identify the criteria for a monitoring instrument, which ultimately will be translated in-to a first draft of a monitoring instrument. The goal of this monitoring instrument is to facilitate a public service buyer in monitoring outsourced services and provide multiple stakeholders evaluative information on subcontractors’ contributions.

Figure 1 Process of designing a monitoring model

2.1 Service triads

The importance of service triads stems from the fact that increasingly both public and private organizations contract suppliers to provide services that will be incorporated in the buyers value delivery and proposition to its customer (Van der Valk et al., 2009). The buyer, subcontractor and end customer form a triad (figure 2), in which buyers’ customer requests services from the buying organization, which has outsourced service production to a subcontractor, who in turn produces the service in direct interaction with the customer (Van Iwaarden & van der Valk, 2013).

There is an increasing risk for the buying firm regarding the delivered service quality by the subcontractor, because the performance of the subcontractor directly affects customer satisfaction. Service triads can be best described by explaining common characteristics. First of all, service triads are characterized by the ensuing relationships between buying organizations, subcontractor and the buyer’s end customers (Li and Choi, 2009). Each actor within the triad is connected to the other two actors, whereby the connection could be constant or periodic (Wynstra et al., 2015). This can be observed as transitive, which means that all three actors have direct ties with each other. The links between the actors often constitute flows of service, information and or/money associated with a specific service production process and involve social/relational elements (Van der Valk & van Iwaarden, 2011). Contractual relationships exist between the customer and the buyer, and between the buyer and the subcontractor (Madhavan et al., 2004). However, there is no contract between the end customer and the subcontractor (Van Iwaarden & van der Valk, 2013). Hence, the contract between a buyer and supplier in a business service triad, should reflect the objectives that the buyer needs to realize for the buyers’ customer (Van der Valk & van Iwaarden, 2011). Another characteristic is the risk of supply chain disintermediation, i.e. total exclusion from delivery for the

(10)

10 buying firm. The occurrence of supply chain disintermediation can be best described by exemplifying the development of relational structures in service triad settings (Li & Choi (2009). According to these authors the shifts in relationships in service triads can be classified in-to three stages; pre-outsourcing (bridge), during (bridge decay) and post-outsourcing (bridge transfer). In the final stage, bridge transfer, the subcontractor starts to interface with the customer. Now the service production and consumption primarily takes place between the subcontractor and the buyers’ customer, the buyer is more or less excluded from service delivery (Van der Valk et al., 2015). The buyer now has to more strongly rely on the contract as a means to design and govern the subcontractor-customer service encounter (Van der Valk et al., 2015). In this stage the subcontractor takes on a more powerful position since it now has access to information from both actors in the triad. To maintain informed and prevent the subcontractor from strengthen its bridge position, Li and Choi (2009) propose to create a permanent state of bridge. This permanent state of bridge means that the buying firm should maintain close communication with both its subcontractors and customers and obtain feedback regarding the performance of the subcontractor. Moreover, these authors propose to continue actively interacting with its customers and closely monitor the subcontractor performances, because, services being outsourced does not mean the buyer should leave everything up to the subcontractor (Sanders et al., 2007). The use of contracts and monitoring procedures minimize opportunistic behavior and assure compliance with the agreements (Van Iwaarden & van der Valk, 2013).

2.2 Contract monitoring

(11)

11 contracts often specify monitoring and reporting requirements that vendors must perform (Brown & Potoski, 2006).

Figure 3 Purchasing process (van Weele, 2014)

2.2.1 Contract monitoring in service triads

According to Van der Valk et al. (2009) the work in service procurement in service triads starts after purchase; contracts should be monitored in order to ensure proper delivery, since it directly affects customer satisfaction. Another important argument to monitor outsourced services in service triads, is to stay to some extent involved in the outsourced service. Li & Choi (2009) describe this as supply chain disintermediation, which may occur after the service is outsourced and the production and consumption of the process primarily takes place between the subcontractor and buyers customer (e.g. bridge transfer; Li & Choi, 2009). Hence, managing information flows, in order to assess either behavior or to measure outcomes, becomes a greater challenge for the buying firm (Li & Choi, 2009).

2.3 Goals of monitoring, issues and monitoring forms

2.3.1 Goals of monitoring

There are multiple monitoring strategies for overseeing the actions of subcontractors and to evaluate the quality of the services which are delivered (Van Slyke, 2007). An example of a monitoring strategy is to use a monitoring system. A performance measurement system is an example a of monitoring system, which is used by organizations to define its services or products and to develop performance indicators to measure its output (Bruijn, 2002).

(12)

12 monitoring information to communicate the value of public programs towards the public (Wholey & Hatry, 1992). Further, monitoring supports the management of contract and relations with for example subcontractors. Since the goal of any monitoring system is to provide managers with performance information about the service delivery, managers can realign subcontractors actions with targeted stakeholder values (Brown et al., 2005). At last, improving the communication by the use of integrated, and multidimensional performance information among the stakeholders may also reduce information asymmetries (Modell, 1997). These goals may apply to an organization, but will also enable comparison between organizations, i.e. benchmarking (Bruijn, 2002).

2.3.2 Issues in monitoring

Some returning issues, related to contract monitoring, are repeatedly mentioned by buying organizations. At first, buying organizations often encounter limited human capital as the biggest issue in monitoring contracted services. This can have the effect of limiting the ability of organizations to develop monitoring activities in order to hold contractors accountable for contract goals, quality and client satisfaction (Van Slyke, 2007). Secondly, it is found that the frequency in monitoring is often determined based on the importance of the service contract. The following example exemplifies this: ‘’In cases were the stakes were relatively high the frequency of reviewing increased. But, in situations in which the stakes were relatively low, public managers randomly select providers and performance reports for review’’ (Brown & Potoski, 2006). The infrequency of monitoring can lead to situations in where it is hard to evaluate the frequency, consistency and quality of service delivery among their contractors (Procurement Guidance, 2014). At last, buying firms which did not performed the outsourced service before often experience limited service related knowledge. This in turn hampers buyers to precisely define their needs and requirements and what standards of practice to apply to monitor and evaluate the service (Selviaridis & Spring, 2013; Sharma, 1997).

2.3.3 Direct and indirect monitoring

(13)

13 performing proxy monitoring ‘’buy’’ monitoring capacity by specifying in the contract that customers undertake monitoring tasks and then report the outcomes of their monitoring. To ensure that the provided information is reliable and accurate, buying organizations can test the reports through follow-up, site visits or spot checks (Procurement Guidance, 2014). At last, buying firms may decide to check the quality of the service delivery of the subcontractor by for example regular follow-ups with their customers. This approach will give the most accurate representation of the service delivery quality in its natural setting. Often a combination of the above mentioned forms of monitoring are used to ensure that the contract outcomes are achieved (Procurement Guidance, 2014).

2.3.4 The content of a monitoring instrument

According to Bruijn (2002) the central idea behind performance management is that public organizations formulate envisaged performance and indicate how this can be measured, by identifying performance indicators. Performance indicators will provide information on trends and variations in performance. Indicators are defined as quantitative and qualitative statistical information which is used to assist in determining how successful an organization is in achieving its objectives (Hall & Rimmer, 1994). Useful monitoring depends on the agreement between policy making and operating levels on appropriate indicators of program performance (Wholey & Hatry, 1992). Hence, there should be a connection between the intended goals of monitoring and the related indicators.

Bruijn (2002) describes the difficulty in measuring public service performance, which especially holds for the ‘outcome’. Outcome, i.e. the final effect envisaged, depends on too many factors. Outcome indicators are indicators which refer to the customer value derived from a given service (Axelsson & Wynstra, 2002). Outcome indicators can include soft elements like client-satisfaction, which are hard-to-objectify (Bonnemeier et al., 2010). Moderately difficult to measure services may require a more extensive monitoring system with overlapping output and outcome performance measures (Brown et al., 2005). However, since monitoring performances is not a costless process, it is desirable that buying organizations are able to collect and provide monitoring information cost efficiently, at reasonable cost (Hall & Rimmer, 1994). Beside outcome indicators, process indicators are also frequently used in performance measurement. Donabedian (1980) describes process indicators as indicators which address logistical and technical issues. Examples of process indicators are; lead-time, throughput-time and waiting time. The measurable effects are most often measured via output indicators. Examples of output indicators are the number of customer served in period X, the amount of money spend for a particular service.

2.4 Customer and subcontractor involvement

(14)

14 and Froehle, 2006). Besides the role of customers as provider of input, i.e. information, belongings and it-self, during consumption and production of service processes, they may also fulfil an important role as assessor (Sampson & Froehle, 2006). In addition, buying firms increasingly make use of public surveys and consultative committees in order to provide feedback (Hall & Rimmer, 1994). This is a useful way to assess whether the services provided by subcontractors match the buyers’ customer needs. Also, customers may support buying organizations in setting goals for outcomes and monitoring results (Wholey & Hatry, 1992). In this way buyers’ customers are involved before subcontractors deliver the services.

When subcontractors’ service delivery buyers’ customers could fulfil the role of assessor. Since buyers’ customers are the direct recipients of the contracted services, they could provide valuable information towards the buying organizations. In a ‘supplier to customer direct service delivery context’ it often occurred that frequent and defined reporting about subcontractors’ service delivery was a noticeable factor in contracts between buying firms and their customers (Gunawardane, 2012). This in turn enabled the buying firm to monitor subcontractors performance more closely. An example of a practice performed by buying organizations are customer contact centres that ‘’allow companies to build, maintained manage customer relationships by solving problems and resolving complaints quickly, providing information, answering questions and being available usually 24 hours a day, 7 days a week, 52 weeks a year’’ (Prabhanker et al., 1997). This type of customer input may aid buying firms in their loss of insights on the provision of outsourced delivery. Buying firms may use these inputs in their monitoring activities, and so to stay ‘involved’ and put indirect control and create oversight on subcontractors service delivery. At last, buyers’ customers may support buyers by periodically reviewing the selected performance indicators (Wholey & Hatry, 1992).

Beside the involvement of buyers’ customers in monitoring activities, subcontractors could also fulfil a role. Before closing the contract, it is important that buying organizations ensure alignment with their subcontractors. This may require that both parties beside discussing the specific goals also jointly agree to the types of measures that would best represent successful service delivery (Brown et al., 2005). Selviaridis & Spring (2013) found that a higher level of interaction, before closing the contract, between buyer and subcontractor will lead to a better contract outcome.

(15)

15 interaction between the buying organization and subcontractor is beneficial to the outcomes of outsourcing (Gadde & Hulthen, 2009).

3. METHODOLOGY

The main aim of this research is to identify goals and requirements for a monitoring instrument, which ultimately will be translated in-to a first draft of a monitoring instrument. In order to develop a monitoring instrument a design method study was conducted. This approach is primarily aimed at discovery and problem solving as opposed aggregation of theoretical knowledge (Holmström, 2009). In addition, design science research is aimed at developing knowledge for professionals in the field (Van Aken, 2004). Six stages, in a nominal sequence, are identified by Peffers et al. (2006) in performing a design study; problem identification, defining goals and requirements of a solution, design and development, demonstration, evaluation and communication (figure 4). This research includes the first three stages of the design science process.

The first stage includes the problem identification and motivation. This stage is elaborated in the introduction part of this research report. The second stage, the identification of the goals and requirements for a monitoring instrument, is to some extent fulfilled in the theoretical section (chapter 2). Here, the goals of monitoring, performance indicators, process requirements for monitoring and the role of subcontractors and buyers’ customers in monitoring activities are discussed. However, stakeholders have to be interviewed in order to extent the identified goals and requirements from literature. The third stage includes the design and development of the monitoring instrument, which will be elaborated in chapter 5. During this stage, an interactive session is organized with representatives of the RIGG.

(16)

16 Due to the limited time, this research will not include the following design science process stages: demonstration, evaluation and communication. Next paragraph provides the context of this research, including the identification of the stakeholders. Subsequently, data collection and the design of the monitoring instrument will be explained.

3.1 Context and stakeholders

3.1.1 Context

The empirical setting of this research is the Dutch transition project. As mentioned in the introduction, Dutch municipalities are as of January 1 (2015) responsible for the whole continuum of universal, preventative and specialized care for all children, young people and families, including those with multiple problems. Since providing direct care is not Dutch municipalities’ core business, but yet it is incorporated in their value’s proposition, this service is outsourced. Hence, a triadic setting is established including Dutch municipalities (buyers), healthcare providers (subcontractors) and clients (buyers’ customers).

The RIGG (Regionale Inkooporganisatie Groninger Gemeenten) which, among other things, is responsible for contracting and monitoring healthcare providers, on behalf of 23 Dutch northern-region located municipalities. Main goal of the RIGG, is to purchase ‘good’ health care services for a reasonable price against good conditions. In addition, ensuring the quality of care is an essential goal. Contracts with healthcare providers are closed in 2014. The RIGG have decided to continue working with contracts which were closed in 2014 in order to ensure quality levels and to minimalize working pressures so the main focus is on the whole decentralisation process. Healthcare providers started delivering care as of January 1st_{(2015). Since then, contract management, including monitoring, of}

approximately 160 closed contracts takes place. However, many Dutch municipalities experience difficulties in establishing successful monitoring. Documents retrieved from the field of research denote the following obstacles (Roerick, 2013; Stals et al., 2014). First, a well-functioning data system for performance measuring is missing. Second, the large diversity in health care providers and health care services, requires all a different form of performance measuring and registration. Third, the importance of monitoring is yet not acknowledged by municipalities. Others projects like designing information systems have more priority at this moment. Hence, additional work pressures are prior to the monitoring activities.

(17)

17 instrument will produce evaluative information which can be used in the evaluation phase. This is the final phase in contract management, according to van Weele (2010).

3.2 Data collection and design

During the second phase of the design science process, goals and requirements for a monitoring instrument were identified. Subsequently, the design process phase started: problem analysis, developing requirements, synthesis, evaluation and design (Van Aken, Berends & van der Bij, 2012). These steps are subdivided in two loops of data collection (figure 5).

First, after the problem is analyzed (phase 1), and the goals and requirements for the monitoring instrument are identified; a process of synthesis will take place. This process will combine the obtained insights from both stakeholders and experts in-to a first design. This is the first loop of data collection (blue box). Afterwards, the design would be evaluated with the stakeholders and if the design meets all the requirements it will be designed in practice. This is the second loop in the process of designing (green box). It may happen that the proposed design does not meet all requirements and so more specifications, or another design is needed (option B). However, due to the limited time it was not possible to perform the second loop of data collection.

Figure 5 Basic actions in designing (Van Aken, Berends & van der Bij, 2012)

In order to identify the goals and requirements for a monitoring instrument, data from multiple sources is required. In this research, data was gathered from two sources:

1. Stakeholders: key users who can provide more insights in the goals and requirements for a monitoring instrument. The service triad in this research consists of the municipalities (buyers), healthcare providers (subcontractors) and client-representatives (buyers’ customers).

(18)

18 The data, gathered from the above mentioned stakeholders, is synthesized at the end of the first loop (Van Aken et al., 2012). The outcome was a first design of a monitoring instrument.

3.2.1 First loop of data collection

In the second step of the design science process data was gathered from stakeholders in order to identify goals and requirements for a monitoring instrument. Besides, interviews were conducted with monitoring experts to obtain more insights and knowledge in monitoring youth care. At last, a representative of the RIGG (Janine Groeneveld) reviewed the draft version of this research report. Stakeholders

Data collection was mainly performed by means of semi-structured interviews, consisting a set of predetermined open questions (Appendix A). The main topics included in the interviews are; general questions about monitoring, design and process requirements for a monitoring instrument, and the role of client (-representatives) and subcontractors in monitoring. Main goal of the interviews was to identify goals and requirements for a monitoring instrument that facilitates municipalities in determining what monitoring activities to put in place in order to evaluate subcontractors’ actions and performances. Semi structured interviews were used so that space for new insights is provided. Participants also had the opportunity to describe their ideal monitoring instrument and share their experiences with monitoring tools. 12 interviews were conducted with municipalities, healthcare providers, client-representatives and monitoring experts.

Table 1 Stakeholder interviews

(19)

19 analyzed. Transcripts which were approved by the interviewees were subsequently coded. A coding-scheme was developed, based on the discussed literature in chapter 2.

Experts

Two experts participated in a mixed interview. The first expert is currently performing research about monitoring general healthcare and innovation in the Netherlands (CMO-STAMM institute). The second expert is involved in multiple projects related monitoring youth care (C4youth project), performed by the municipality of Groningen and the UMCG (hospital). The goal of this interview was to obtain insights and knowledge about monitoring Dutch youth care; difficulties, the field and obstacles. The goals and requirements identified from the experts (table 3) were taken in-to account in the first design of the monitoring instrument.

Design

In order to enhance transparency in the process of synthesizing the identified the goals and requirements in-to a first monitoring design (figure 5), a card system is developed. Via this card system it was attempted to find patterns in the stakeholders’ perspectives on the design requirements and the design itself of the monitoring instrument. This card system is subdivided in four sections (the sequence of the interview structure); (1) goals of monitoring (2) the contingency perspective in monitoring youth care (3) indicators, (4) process requirements for monitoring. This latter section includes the exploitation of monitoring capacity, frequency and intensity and the role subcontractors and buyers’ customers in monitoring.

(20)

20

3.3 Research quality

The validity and reliability of this research is safeguarded in several ways. The quality of this research is ensured by fulfilling the quality criteria, as proposed by Yin (2009), see table2.

(21)

21

4. RESULTS

This chapter discusses the identified goals and requirements for the monitoring instrument, which are derived from the interviews with the stakeholders. Besides, the interview performed with two monitoring experts is included. In section 4.1, the goals of the monitoring instrument are discussed. Section 4.2 discusses the contingency perspectives on the monitoring instrument. Next, section 4.3, discusses the data regarding the indicators of the monitoring instrument. In section 4.4 the process requirements for monitoring are discussed. These process requirements include; the exploitation of monitoring capacity, the frequency and intensity in monitoring and the role of subcontractors and buyers’ customers. A summary, including the identified goals and requirements for the monitoring instruments is provided in section 4.5. In chapter 5 the author translated the identified goals and requirements in a first design of a monitoring instrument.

4.1 Goals of monitoring

This paragraph will discuss the findings regarding the goals of monitoring. Interviewees were asked to order from the most important to least important goal of the instrument five pre-formulated goals of monitoring. These goals were based on the literature review in chapter 2. In addition, interviewees were offered the opportunity to add goals. The five goals are; learning and improving, providing input for future contracts, accountability towards municipalities or clients and to determine awards of sanctions (subparagraph 2.3.1).

Municipalities (n=5)

The most important goal of monitoring according to all municipalities is: ‘learn and continuous improving’ (n=5). Followed by ‘accountability towards clients and municipalities’, and ‘the input for future contracts’. One said; ‘’if you focus on continuous improving, the rest will follow automatically’’. Most of the respondents argued that a monitoring instrument ultimately will provide more insights in their internal processes, which may are subject to improvement. An additional goal of monitoring was given by two interviewees; ‘’the instrument should be used to benchmark performances’’ (n=2). The least chosen goal of monitoring was; to determine awards or sanctions (n=4). Therefore, an interesting argument was provided; ‘’municipalities should be more convinced about intrinsic motivation of healthcare providers, to provide good youth care.’’ (n=1).

Healthcare provider (n=4)

(22)

22 providing ‘input for future contracts’ and ‘accountability towards clients and municipalities’. Again, to determine awards or sanctions is the least important goal, according to the healthcare providers.

Client representatives (n=2)

The client-representatives believe that ‘to learn and continuous improving’ is the most important goal of monitoring. Interesting argument therefore was; ‘’…to create a culture where continuous improving and to learn from each other should be a fundamental starting point’’. Client-representatives argued that ‘to award and sanction’ is the least important goal of monitoring. In addition, one argued that they like the ‘’high trust, high penalty’’ 1_principle.

Experts (n=2)

The experts opine that ‘learning and improving’ is the most important goal of monitoring. Moreover they added an additional goal of monitoring; ‘’signaling poor performance by subcontractors and adjusting direction in current policy’’. One stated; ‘’if you put ‘accountability’ on top, it has somehow an unpleasant taste... If you opt for that, you actually forget that you’ll use monitoring to improve your services’’.

To sum up; ‘to learn and improve’ is evidently the most important goal of monitoring, according to all stakeholders (n=12). Many stakeholders (n=7) share the view that ‘to award or sanction’ is the least important goal of monitoring. Hence, there is not only alignment within the stakeholder-groups, but also

1_{: This phrase means; ‘I will trust you, but if you harm this trust, you will be sanctioned’. This phrase should}

increase the level of trust from supervisor’s perspective, but also sanctions when trust is harmed.

1 2 3 4 5 6

Importancy sequence - monitoring goals

Learning and improving Accountability towards municipalities

Accountability towards clients Input for future contracts

To determine awards and sanctions Other, namely:

(23)

23 within this particular service triad. Figure 6 provides an overview of the most important to least important goal of the instrument.

4.1.1 Contingency perspective on monitoring instrument

Youth care covers a wide variety of different services; these vary on classifiers or determinants as intensity, residential or ambulatory care and whether care is provided in a legal framework or not. During the interviews, stakeholders were asked to consider whether a particular parameter could trigger the need for customization, i.e. care that ‘scores’ differently on this determinant requires an adjustment in the way of monitoring. Hence, in this subparagraph, the results about the contingency perspective on the monitoring instrument will be discussed.

Municipalities

(n=5)

Monitoring healthcare should be focused on identifying risks. Risk was a central theme in this discussion. Risk was related to multiple things; regarding the safety of children (n=2) and their environment (n=1), financial risk (n=1) and political risk (n=2). Financial risk includes the risk of exceeding budget by healthcare providers. One specified ‘safety’ to the ‘vulnerability of the children’ who receive care. The more vulnerable the particular patient in care, the higher their dependency. One employee of the RIGG mentioned; ‘’I think in case of high complex care you need to intensify the monitoring, because of their safety’’. At last, transformation targets formulated by the RIGG (paragraph 5.2) have to be monitored (n=2); ‘’you want to monitor the effect of transformation targets. For instance in financial shifts, and whether the number of patients in for example foster care de- or increases’’. Healthcare providers (n=4)

There was not a clear theme in this discussion. One provider stated; ‘’when you come to an agreement with a healthcare provider, you should put trust in their effort in providing good healthcare’’. Another provider mentioned; ‘’I think you need to consider, for each healthcare provider separately; what are the risks, and what additional information do we therefore need’’. Complexity should need not be an determinant in this monitoring instrument (n=1). Further, healthcare providers mentioned that there should be a difference between monitoring small and large healthcare providers, because of the administrative burden (n=2).

Client-representatives (n=2)

(24)

24 Experts (n=2)

Experts mention that they would increase and intensify monitoring based on the type of care. Besides, they mention both ‘risk’ as an important determinant in designing a monitoring instrument: ‘’organizations providing highly complex care are more sensitive for mistakes. This is detrimental to clients, but also for to the municipalities’’. Mistakes are in this context errors which are can be made while providing care to a client. Examples of mistakes are; wrong diagnosis, wrong prescriptions. One experts stated that healthcare couldn’t cost endlessly. Since the transition project in the Netherlands, municipalities deal with less budget for providing youth care. Therefore, financial risk should be monitored more closely (n=1). At last, one expert mentioned that the intensity and comprehensiveness in monitoring relatively simple youth care should be lower compared to high complex care. Hence, risk in terms of finance and safety and the type of care are important contingency factors in the monitoring instrument.

To sum up; risk is a central theme in monitoring youth care. But, it does not become clear what exactly is meant with this. Also, monitoring the use of resources, and achieved effects is something that should be taken into account according some stakeholders. Besides, complexity and safety are important contingency factors.

4.2 Content – indicators

Outcome and output indicators are often used in Dutch youth care to monitor healthcare services. In addition, process indicators are frequently used to measure performances. Each stakeholder is asked to propose useful indicators to monitor provided youth care services.

Municipalities mentioned: ‘target realization’, as an important outcome indicator (n=2). Moreover, ‘client satisfaction’ is found as an often used outcome measure to determine the quality of care (n=3). In addition, one mentioned that the ‘amount of care-returners’ is an important outcome indicator. However, municipalities admire that measuring outcome is complicated (n=2). ‘Costs’, relating to the healthcare expenses for a particular child in care, and the ‘amount of children in care’ are frequently mentioned output indicators (n=4). One municipality stated that acquiring and sharing these type of data isn’t hard, since healthcare providers already measuring this for themselves. Two municipalities shared thought about the use of input indicators; ‘’minimum quality criteria for a healthcare provider’’. The most often mentioned process-indicators are; throughput time (n=3), waiting lists (n=2) and responding time (n=1). Throughput time in this context is the total time of a treatment.

Healthcare providers (n=4)

(25)

25 satisfaction tests are performed by healthcare users individually. However, some target groups do not have the ability to perform these tests by themselves. Hence, care-takers support these healthcare users in performing the test. Besides, ‘effectivity’ is mentioned by healthcare providers as an important outcome indicator (n=2). Output data is described as hard data, which can be derived from providers themselves because they measuring these data already for themselves. Example of output indicators are; average costs per client (n=2), amount of clients in care (n=2), how much time is spend in providing care (n=1) and, how many treatments are provided in a certain period (n=1). Two healthcare providers mentioned that ‘costs’ and the ‘number of children in care’. One of these providers said; ‘’I think it’s all about the number of clients in care, and the related costs’’, and ‘’.. the average costs per patient are also very important to monitor’’. Healthcare providers mentioned quality criteria as input indicators. Examples of quality criteria comprise the BIG-registration of pedagogues and ISO certifications of health providers. Waiting time (n=2), waiting lists (n=3), throughput time (n=2) and responding time (n=2) are frequent mentioned process indicators.

Client-representatives acknowledge that measuring outcome is quite difficult. One mentioned that healthcare providers should not be sanctioned based on their finance and ‘production’ (i.e. serving as many clients as possible) immediately. This in turn will not enhances to the intrinsic motivation of healthcare providers to perform high quality care (n=1). The other stated that process indicators should be not in the lead, but supportive to the outcome indicators. Process indicators include waiting time (n=2) and waiting lists (n=2).

Experts (n=2)

The experts mentioned that the effect of provided care should be expressed in an outcome indicator. Moreover, they specify this by differentiating in different target groups. They do not specifically mention output indicators, but state; ‘’each indicator that shows the required resources for pre-determined treatment targets’’. Process indicators are also important; ‘’you want to prevent long waiting lists’’.

4.3 Process requirements

This paragraph discusses the process requirements for the monitoring instrument. The process requirements for monitoring are subdivided in four items; the exploitation of monitoring capacity, and setting the frequency and intensity, the role of the healthcare provider and the role of the client in monitoring.

(26)

26 design to be developed and current systems. This could be realized via a smart IT application (n=2) Moreover, achieving efficiency in exploiting monitoring capacity can be realized in the development phase of the monitoring instrument (n=1).

The frequency and intensity in monitoring can be based on multiple things. One said; ‘’the intensity of monitoring should depend on the financial expenses of a healthcare provider. When a particular treatment is very expensive, we would like to know; why does it cost that much?’’ and ‘’if we know on beforehand that healthcare providers will complain about their budget, and they expect that they will run short on budget, we would increase the frequency of monitoring’’. In addition, shortage on budget is also a trigger to increase/decrease the frequency of monitoring (n=1). Further; ‘’outcome indicators should be monitored each sixth month, and the basic output data on monthly basis’’. Output data is relatively easy to acquire, since healthcare providers already measuring this within their organizations (n=2). Another determinant is related to the transformation targets, formulated by the RIGG; ‘’If a healthcare provider is sensitive for one of these transformation targets, than I would increase the monitoring frequency. Then we are able observe the effects of the transformation target more closely’’. Others relate the intensity of monitoring to the type of clients/health care type (n=3). At last, trust is an important determinant in setting the intensity of monitoring; ‘’it all depends on trust; the higher the level of trust, the less we have to know’’.

Municipalities are to a large extent aligned about the role of healthcare providers in monitoring. They agreed upon that healthcare providers are allowed to think along, for example with follow-up actions based on monitoring results; ‘’think along, but not being decisive’’ (n=3). A sound board group is proposed to compose, in order to involve healthcare providers (n=2). In this way Further, one municipality stated; ‘’municipalities should not interfere with healthcare content related tasks too much. For instance what resources are require. This is something that has to be determined between healthcare providers and clients’’. However, there is a little disagreement about the role of clients in monitoring; ’…do we involve them or not? Is it relevant to incorporate their information?’’. One said that monitoring is ‘too far away’ for clients to properly understand. Another municipality mentioned that clients should have a platform in order to share their experiences, and ideas.

(27)

27 sharing monitoring information’’. Using smart IT applications could facilitate healthcare providers in sharing monitoring information more easily. This in turn will decrease the exploitation in monitoring capacity.

Healthcare providers are willing to cooperate, and think along with municipalities about the development of the instrument. Besides, being involved in the development-phase and subsequently providing monitoring information, providers would like to think along about follow-up actions based on the monitoring data. However, there is some disagreement about the role of clients in monitoring. Two providers say; ‘’I think its exaggerated to involve clients’’ and ‘’clients do not want to’’. Two others say; ‘’clients should be involved to determine quality standards’’ and ‘’the RIGG should have a direct line with clients to enhance cooperation’’.

The frequency of monitoring should be based on the complexity of care, and vulnerability of the children in care (n=2). One client-representative mentioned that, in order to create a culture where ‘learning and improving’ is a fundamental thing, healthcare providers should have an educational and feedback providing role. The other stated; ‘’beside providing monitoring information, providers may also think along about follow-up actions together with municipalities’’. Both stated that clients (-representatives) should have a decisive role in monitoring; ‘’…they should think along about how to transfer monitoring results into follow-up actions’’, and ‘’for me, clients should be involved in the development phase, the monitoring process itself and post-monitoring actions’’.

Experts (n=2)

(28)

4.4 Summary

(29)

(30)

4.5 Further development

This paragraph discusses the results about further development of the monitoring instrument. Explicitly, the role of municipalities themselves, healthcare providers and clients are discussed.

According to all municipalities (n=5), healthcare providers have to be involved in further development of the monitoring instrument. But; ‘’providers may think along with municipalities about further development, but definitely not being decisive’’. Main reason for their involvement is because they hold healthcare expertise (n=3). Besides, municipalities agree upon the role of clients; ‘’they should be involved in the development of the monitoring instrument’’. Another municipality mentioned that a mix of client-representatives should represent the clients in the development of the monitoring instrument as well as subsequent evaluation. At last, municipalities notice themselves as orchestrators of this whole process (n=3) since the provision of youth care is incorporated in their organizations value delivery and proposition.

Healthcare providers propose to support municipalities in for example the selection of indicators (n=2). One mentioned; ‘’I would like to be involved, very closely’’, and ‘’beside development, we would like to evaluate the instrument and give feedback towards the municipalities’’ and ‘’we really would like to know what municipalities eventually undertake with for example the monitoring results’’. The role of clients in further development of the monitoring instrument is particularly assessing the selected indicators and evaluating these measures. One healthcare provider suggests to create a mixed group which comprises client-representatives with diversified healthcare backgrounds to perform such evaluations (n=1). One provider stated that municipalities are responsible to organize meetings with stakeholders for the development and evaluation of the monitoring instrument (n=1). Further, providers’ experience this whole governmental decentralization project as a communal ‘thing’ (n=3). This means that input and involvement of municipalities, healthcare providers as well as client-representatives is expected.

(31)

(32)

32

5 SYSTEM DESIGN

This chapter provides a first draft of a monitoring instrument based on the input from the interviews from the stakeholders. This instrument is structured and subdivided in several steps; (1) goals of monitoring, (2) key decision areas, (3) indicators and (4) process requirements (figure 6). This chapter also includes additional data that is collected during the interactive meeting at the RIGG in design science stage 3.

5.1 Goals of monitoring

Derived from the results section (chapter 4), the most important goal of monitoring according to all stakeholders is to ‘learn and continuously improve’. In the second place, the monitoring instrument will provide accountability information towards municipalities and clients. In the third place, monitoring information will serve as input for future contract discussions. The least important goal of monitoring is to ‘award or sanction’ subcontractors. Because these four goals were most frequently mentioned, they are included in-to the monitoring instrument, and will goal as a starting point (figure 6, step 1).

5.2 Key decision areas

During the interactive meeting (stage 3 of the design science process ) a returning discussion about four issues that came up from the interviews with the stakeholders, especially when talking on what the interviews expressed on the contingency perspective of the instrument. These four returning issues or decision areas are; deviation in healthcare type, risk monitoring (financial, safety, political), monitoring transformation targets which are formulated by the RIGG and interfering in healthcare professionalism. From now on, these returning issues are labeled as ‘key decision areas’ (Appendix B).

Key decision area 1: monitoring type of care

To develop the monitoring instrument more in detail, stakeholders should agree upon the determinant that should be dominant, i.e. what kind of classification scheme should the monitoring instrument be customized. A client-representative mentioned a classification scheme which is used in general care, in the Netherlands. This classification scheme comprises; pre-care, community care, low complex and high complex care. After the interactive meeting, during stage 3 of the design science process, one representative of the RIGG stated that all possible care types can be labeled as treatable or controllable care, i.e. care or cure. The results of the interviews show that complexity and the level of dependency and vulnerability of the child in care are important determinants. Hence, several attempts are made in order to categorize healthcare. However, a seeding classification is yet not being used in Dutch youth care.

Key decision area 2: risk monitoring

(33)

33 related by the stakeholders to financial risk, the safety of children in care and their environment, and political risk. Risk related to the safety of the children in care and their environments often depends on the level of vulnerability and dependability. The higher the level of vulnerability, the more intensive monitoring information municipalities want to receive. Financial risk is particularly related to care expenditures and budgets. Key decision area 1 may complement key decision area 2 since stakeholders mentioned that particular target groups are equivalent for expensive care.

Key decision area 3: monitoring evolution in Dutch youth care

During the interactive meeting at the RIGG, it became clear that there was a desire to monitor the transformation targets. Due to the decentralization of the Dutch youth care, the RIGG (on behalf of 23 Groningen northern-region located municipalities) decided to formulate eight transformation targets related to the Dutch youth healthcare. These targets are aimed to enforce the facilities in the pre-care stages. Main reason for this is to decrease the amount of highly intensive youth care in the Netherlands. Each transformation target focusses on a particular issue in Dutch youth care. To each of these targets, a set of indicators will be related, including process requirements. The monitoring of each transformation target including their specific monitoring requirements could be developed on their own; this could be regarded as developing eight independent monitoring-units or ‘module’ (Appendix C). The eight transformation targets are:

(1) Further development of the coordination of availability and critical care (2) Renew and increase flexibility of foster care

(3) More general application of youth protection/youth rehabilitation measures in the allocated (forced) care

(4) A common screening process of high intensive care, including an integrated triage and more attention for care-returners/outflow.

(5) Dyslexia-compensation transformation

(6) To pass the ‘Sign of Safety’ as an approach to the local partners (7) 18-/18+. Attempt to find a bridge with educational care facilities

(8) Project ‘Safe Home’: ensure more safety for youth in the whole care chain Key decision area 4: monitoring and professionalism

(34)

34 and whether healthcare provider agree upon this issue. Hence, municipalities and healthcare providers should first achieve alignment regarding this issue. If municipalities want to know exactly what healthcare providers perform, in terms of treatments, a certain level of service related knowledge is required. In case municipalities decide to not intervene in the processes which are performed by healthcare providers, a black box is created. The consequence of this decision is that the monitoring will be restricted to the output of the black box. In case municipalities want to interfere, they have to dig deeper in how youth care is provided. But due to the limited task-related knowledge they experience (Yacht, 2015), this is unlikely.

By offering the possibility to ‘select’ key decision areas the monitoring instrument enhances the flexibility and multi-dimensionality. Hence, this monitoring instrument not only includes subcontractors’ performances, but also the transformation targets developed by the RIGG and risks. In this way, the monitoring instrument is characterized by its comprehensiveness.

5.3 Indicators

The third step in the monitoring instrument relates to the selection of (performance) indicators. As discussed in paragraph 2.3.4, many indicators can be used in order to measure performance. During the interviews, stakeholders were asked about their view in using output and outcome indicators in Dutch youth care. Effectiveness, i.e. target realization, was mentioned most frequently as an important outcome indicator. Second most frequently mentioned outcome indicator was client satisfaction. However, many difficulties arise while using this indicator since healthcare providers do not measure this in a uniform way which in turn makes it unable to benchmark the outcomes. Sometimes, client-satisfaction tests comprise a list with questions which will has to be filled in by healthcare ‘users’ and sometimes with support of their care-takers. This depends on whether the healthcare users have the ability to do it on their own. Hence, to this day, measuring client satisfaction is troublesome. Most frequently mentioned process indicators are; throughput time, waiting time/lists, start-time (responding time). However, the relationship between the key decision areas (2) and indicators (3) is too vague to appoint a specific set of indicators for a specific key decision area (figure 7). For instance, key decision area 2 (monitoring risk) contains safety-monitoring but this is yet not translated to specific indicators.

5.4 Process requirements

(35)

(36)

(37)

6 DISCUSSION, CONCLUSION AND RECOMMENDATIONS

This chapter provides a discussion and final conclusion of this research. Subsequently, the limitations and strengths of this research will be addressed, and recommendations about further development and implementation will be provided.

6.1 Discussion & Conclusion

The aim of this study was to design a monitoring instrument that facilitates buyers in monitoring outsourced services and provide multiple stakeholders evaluative information on subcontractors’ contributions. Goals and requirements are identified from each service triadic partners’ perspective; municipalities (buyer), healthcare providers (subcontractor) and clients (buyers’ customer), which in turn are translated in-to a first design of a monitoring instrument (figure 7).

(38)

38 Example 2: The intensity of monitoring depends on the financial expenditures of healthcare providers due to the financial risks. The higher the expenditures, the more monitoring information municipalities want to receive.

Further, results show that subcontractors and buyers’ customers might have a substantial role in the development of the monitoring instrument (Selviaridis & Spring, 2013; Brown et al., 2005), as well as in subsequent monitoring activities (Wholey & Hatrey, 1992; Sampson & Froehle, 2006). Buyers’ customers should fulfil two roles according to the results of this research. First , they need to be involved in the development-phase of the monitoring instrument. Buyers’ customers are the recipients of provided services, hence they should for instance have a say about selected indicators. While using the monitoring instrument, buyers’ customers could provide feedback towards buying organizations about subcontractors’ performances (Wholey & Hatry, 1992). The subcontractors’ role in the first design of the monitoring instrument is substantial too. Selviaridis & Spring (2013) mentioned that buying organizations facing limited service-related knowledge may involve subcontractors in setting requirement of the outsourced services. Data from my research shows that buying organizations are inclined to involve subcontractors in the development of the monitoring instrument, to in turn exploit their expertise. For example, subcontractors will be involved in the selection and definition of indicators, which in turn would best represent successful service delivery (Brown et al., 2005). Further, results show that municipalities want to involved healthcare providers in follow-up discussions which are based on the monitoring results.

Furthermore, the most important goal according to all stakeholders (‘to learn and improve’) translates to some of the subsequent monitoring instrument steps. Since the outsourced service was not performed by the buying organizations before, they experience limited service related knowledge. Therefore, subcontractors and buyers’ customer are involved in developing monitoring activities, as well as participating in these monitoring activities. Buyers’ customer might provide evaluative information on subcontractors’ performances and regarding the set of indicators. Buying organizations might involve subcontractors in the selection of indicators (Brown et al., 2005) to exploit the expertise. In addition, the relationship between the intended goals of the monitoring instrument and the selection of indicators, which is important to establish useful monitoring (Wholey & Hatry, 1992), is not explicit.

(39)

39 transformation targets and risks. This is possible due to the development of four key decision areas. Users are enabled to decide what ‘key decision areas’ they want to monitor. Hence, this monitoring instrument has a multi-dimensional character, and is characterized by its flexibility and comprehensiveness. This is for a greater part ensured by the development of four key decision areas. These areas enable users to define more precisely the focus and boundaries of the monitoring instrument by deciding the contingencies that should be applied in developing monitoring activities and whether the transformation targets should be monitored as well. Further, flexibility is ensured due to the fourth step in the monitoring instrument. For example, this step enables users to adjust the intensity and frequency in monitoring to their selected key decision area.

6.2 Recommendations

Preferably, municipalities (and the RIGG) should first align their expectations of this monitoring instrument. For instance, municipalities vary in thoughts about the role of clients in the use of the monitoring instrument. Some state that involving clients in such monitoring activities is ‘too far away’. Others propose to create a platform for clients in order to allow them sharing their experiences regarding subcontractors’ provided services. The findings from the interviews that took place during the first loop of data collection can be used by the RIGG in order to achieve alignment among the stakeholders. Second, the relations between the different monitoring instrument steps should be further researched. For instance, it is not clear what specific monitoring information is required when municipalities decide to adopt a specific goal or to focus on specific risks. In addition, specific indicators should be appointed the transformation targets, in order to monitor the effects. Third, stakeholders mentioned that they experience difficulties in measuring outcome indicators (e.g. client satisfaction, effectiveness). After the interactive meeting at the RIGG (third stage of the design science process) it was proposed to put more effort in gaining more grip on the measurability of outcome indicators. At last, it is proposed to evaluate the first draft of the monitoring design. Another meeting should be organized, including municipalities, healthcare providers and client-representatives in order to evaluate to the first design of the monitoring instrument.

6.3 Limitations

(40)

40 number of stakeholders could be interviewed, in order to identify the goals and requirements for the monitoring instrument. The stakeholders who contributed in this research were selected by employees from the RIGG. Healthcare providers were selected based on their size (€) and healthcare background. Municipalities were selected based on their expertise and prior experience in monitoring. The selection of stakeholders could therefore be biased.

(41)

41

REFERENCES

Anderson, E., & Oliver, R. L. (1987). Perspectives on behavior-based versus outcome-based salesforce control systems. The Journal of Marketing, pp. 76-88

Axelsson, B., & Wynstra, F. (2002). Buying business services. John Wiley

Bailey, P. et al., (2008). Procurement principles and management. 10th ed. Harlow, England: Pearson Education

Bonnemeier, S., Burianek, F., & Reichwald, R. (2010). Revenue models for integrated customer solutions: Concept and organizational implementation. Journal of Revenue & Pricing Management, vol. 9, no. 3, pp. 228-238

Brown, T. L., Potoski, M., & Van Slyke, D. M. (2006). Managing public service contracts: Aligning values, institutions, and markets. Public Administration Review, vol. 66, no. 3, pp. 323-331

Brown, T. L., Potoski, M., & Van Slyke, D. (2005). Managing the tools of government: Contracting and contract management in the new millennium. InPaper for the presentation at the 2005 National Public Management Research Conference, University of Southern California

Campbell, D. (2012). Public managers in integrated services collaboratives: What works is workarounds. Public Administration Review, vol. 72, no. 5, pp. 721-730.

Choi, T. Y., & Wu, Z. (2009). Taking the leap from dyads to triads: Buyer–supplier relationships in supply networks. Journal of Purchasing and Supply Management, vol. 15, no. 4, pp. 263-266

De Bruijn, H. (2002). Performance measurement in the public sector: strategies to cope with the risks of performance measurement. International Journal of Public Sector Management, vol. 15, no. 7, pp. 578-594

Domberger, S. (1994). Public sector contracting: does it work?. Australian Economic Review, vol. 27, no. 3, pp. 91-96.

Donabedian, A. (1980). Explorations in quality assessment and monitoring. The definition of quality and approaches to its assessment. Michigan: Health Administration Pres.

Monitoring in Public Service Triads

MASTER THESIS