PERFORMANCE MANAGEMENT AT DUTCH SECONDARY SCHOOLS

(1)

PERFORMANCE MANAGEMENT AT DUTCH

SECONDARY SCHOOLS

How performance management can affect the performance of Dutch

secondary schools: a case study

Master thesis, MSc Business Administration

Specialization: Organizational & Management Control

University of Groningen, Faculty of Economics and Business

June, 2013

Joost Haverkate

Student number: s1891057

joost_haverkate@hotmail.com

Supervisors university:

First supervisor: dr. S. Tillema

(2)

1

Acknowledgements

I would like to thank a few persons that helped me a lot during this process of writing my thesis and to whom I am very grateful:

 My supervisor from the university, dr. S. Tillema, for her critical but useful feedback on my thesis, during the whole process of writing it.

 The headmaster of my case school, for his permission to conduct a study at his school and inviting me to the ‘Study day’.

 The team leaders and teachers who I have interviewed, for being so open and for providing me with a lot of useful information.

 My fellow master student Fianne Meijer, for her useful feedback.

 My fellow student of economics, Eef Mittendorp, for checking the spelling and grammar.

(4)

3

1. Introduction

In modernizing the public sector, performance management is an essential element (Bouckaert and Peters, 2002) and its increased use is a widespread international trend (Pollitt, 2006). Bouckaert and Halligan (2006) even state that „performance management forms the core of public management, especially New Public Management (NPM).‟ As the name already suggests, the goal of performance management ultimately is to improve performance. Definitions of performance management differ, but a useful one comes from Rogers (1990, p.16): performance management means „an integrated set of planning and review procedures which cascade down through the organization to provide a link between each individual and the overall strategy of the organization‟.

In the Netherlands, managing performance has recently become more important for secondary schools. A few years ago, the Dutch council of secondary education started a project called „Vensters voor Verantwoording‟ (Windows of Accountability), which goal is to increase transparency, dialogue and quality. This project aims at collecting various types of quantitative data about Dutch secondary schools into one system. Based on these data, quality standards are developed and benchmarking is made possible. Both schools and their stakeholders gain insight into the relative quality of education on a particular school. In February 2010, as part of the project, a website was launched (www.schoolkompas.nl), meant for pupils in the last year of primary school and their parents. This website must assist them in choosing an appropriate secondary school. The website provides a list of all secondary schools, and compares every school to the national average on particular performance indicators. A list of these performance indicators is provided in Appendix 1. This project and its openness suggest that secondary schools should focus on performance even more than before, in order to attract a sufficient number of new pupils.

Measurement of the performance of Dutch secondary schools is executed by the schools themselves and by „Dienst Uitvoering Onderwijs (DUO)‟, which is the executing organization of the Dutch Ministry of Education, Culture, and Science. Both provide

„Vensters voor Verantwoording‟ with performance information. This project incorporates all the information into figures that are made available to different stakeholders. Schools can incorporate the information themselves as well, for example in internal reports. Schools should use the performance information to improve their performance. Based on all performance information and after further analysis and inspections by the Inspection of Education, the Dutch government can force a school to close its doors.

(5)

4

as well? Should the focus be on hard data like grades and results or on soft data like safety and a good learning climate or satisfaction among pupils? And, for example, when a school decides to focus on results of pupils, this brings some additional difficulties. Since several factors outside the school, such as the influence of family, may have a strong impact on results as well, it is difficult to show how much value the school added (Todd and Wolpin, 2003).

Previous research has investigated performance management on secondary schools. For example, Sarrico et al. (2012) investigated performance management within Portuguese secondary schools. However, to my best knowledge, there is no literature about performance management on secondary schools in the Netherlands. The Center on International

Education Benchmarking (CIEB), a program of the National Center on Education (NCEE) that researches the best performing education systems worldwide, lists ten top performing countries. The Netherlands is among these countries, indicating a relatively good educational system. For example, more than 90% of Dutch secondary pupils finish secondary school, indicating one of the highest completion rates of all countries.

However, despite the relatively good status, there will always be opportunities to improve performance, which is not only important for pupils, but also for the Dutch society as a whole. Furthermore, it is important for every individual secondary school to improve its performance, since performance information is more widely available than before, and potential pupils and their parents base their choice on this information. This report contributes to the existing knowledge by investigating performance management and its effects on performance at Dutch secondary schools. The research question that this report will try to answer is: How can performance management affect the performance of Dutch secondary schools?

Answering this question is important, because it gives insight into the usefulness of performance management systems for secondary schools. If performance management systems appear to have only a relatively small effect on performance, or no effect at all, it can be questioned whether current systems should be improved or performance management should be abandoned completely from the schools. The costs of projects like „Vensters voor Verantwoording‟ would outweigh the benefits in that case. On the other hand, if performance management systems appear to have a positive effect on the performance of secondary schools, then its use should be stimulated, in order to improve the performance of all secondary schools in the Netherlands.

(6)

5

The rest of this report is structured as follows. Section 2 gives an overview of the available literature on performance management, both in general and more specifically in the public sector and at schools. Section 3 describes the methodology of this research. It covers the type of research, types of data, and the methods of data collection and data analysis. Section 4 shows the analysis of the data obtained. Results are provided and outcomes are discussed. Section 5 is the conclusion of this research: the research findings are summarized and what is added to the existing knowledge is highlighted. Section 6, the discussion,

(7)

6

2. Literature Review

1. The role of performance management within New Public Management

New Public Management (NPM), which emerged in the 1970s, is a term commonly used for the restructuring of the public sector, both public organizations and public governance. It redefines the role of managers and employees within this sector, and relationships with stakeholders change (O‟Donnell et al., 2011). In short, NPM means that instruments and the style of managing from the private sector are introduced and used in the public sector (Ter Bogt and Scapens, 2012). The main purpose of NPM is to make the public sector more efficient and effective (Hood, 1991; Schubert, 2009), thereby improving

performance (Ter Bogt and Scapens, 2012).

NPM is an overarching term, covering a set of administrative doctrines that were important in the bureaucratic reform agenda of many countries starting in the 1970s. Hood (1991) identified seven of these doctrines. To give the reader an idea, among these doctrines were a „greater emphasis on output controls‟ and a „shift to greater competition in the public sector‟. Both are relevant for the sector of secondary schools in the Netherlands. There is an increased level of competition among schools, as a result of the more widely available performance information. In order to retain a good position in this more competitive environment, schools need to focus more on outputs that reflect part of their performance. Another doctrine relevant to this report is what Hood (1991) called „explicit standards and measures of performance‟. According to him, this means defining targets, goals, and indicators of success. He explains that a clear statement of these elements is necessary for accountability. Implementing these performance standards and performance measures can indicate what parts of performance need extra attention, in order to improve or manage performance. In line with Bouckaert and Halligan (2006), this shows that performance measurement and performance management are important components of NPM. Before the existing literature about performance measurement and performance management can be elaborated usefully, however, the concept of performance itself has to be explained.

2. Performance

As Armstrong and Baron (1998) state it: „If you can‟t define performance, you can‟t measure or manage it.‟ There is no simple definition of performance (Otley, 1999; Mwita, 2000). On an organizational level, performance is assumed to be linked with successfully meeting the objectives of the organization. Performance in this sense could be defined as „effectively implementing an appropriate strategy‟ (Otley, 1999).

(8)

7

performance indicators that the project „Vensters voor Verantwoording‟ uses, this study takes the position that performance is determined mainly by results. Behaviors are important, however, since they can influence results.

Managers should take several factors into account that influence performance. Armstrong and Baron (1998) name four of these factors. First, personal factors are

distinguished. These comprise, among other things, skills, commitment to the job and the organization, and motivation. Second, leadership factors influence performance. In this respect, support, motivation, and guidance by managers are important. Third, team factors are mentioned. The quality of colleagues and the amount and quality of interaction with them have an influence on performance. Fourth, system factors like the facilities that the

organization provides to its employees are important. Mwita (2000) added a fifth factor that influences performance. He states that contextual factors can have a strong influence on performance. Both internal and external pressures can force or limit an organization and its employees to act in certain ways.

Now that we have discussed the meaning of performance itself and factors influencing it, we can take a step closer towards performance management by first discussing the concept of performance measurement.

3. Performance measurement

To improve or manage performance, organizations need to measure performance. The ultimate goal of performance measurement should not be control, but learning.

Organizational learning is a process that uses results from history and translates this into routines, that should guide current behavior, which is target-oriented (Levitt and March, 1988).

The role of a performance measurement system is to focus attention on primary and secondary objectives (Atkinson et al., 1997; Mwita, 2000). Secondary objectives

(determinants) determine primary objectives (results). In this way, performance

measurement can allow the actual management of performance (Bititci et al, 2012). Put differently, a performance measurement system should form the core of a performance management system, which in turn helps an organization to reach its objectives (Sahoo and Jena, 2012).

(9)

8

Traditionally, the focus of most organizations was on the financial part of

performance. Using primarily financial performance measures, however, lacks the right focus and robustness necessary for good internal management and control (Atkinson et al., 1997). Therefore, in addition to these financial performance measures, the need for non-financial measures of performance appeared in both the private and public sector (Kloot and Martin, 2000; Mwita, 2000). Recent literature states that there should be a strong linkage between measures of performance and strategic plans (Otley, 1999; Kloot and Martin, 2000; Bititci et al., 2012). Also, measures of performance should prevent managers from cheating on these measures: their measured performance should only increase when their actual performance improves. Furthermore, performance measures should be simple and measuring costs need to be relatively low, in order to reap the benefits from the system (Heinrich, 2002).

4. Performance management

Now that we have discussed performance measurement, we can discuss the other two components that, together with performance measurement, form performance management: incorporation and the use of performance measures (Bouckaert and Halligan, 2006;

Bouckaert and Halligan, 2008). Without incorporation, measurement of performance is useless. In turn, incorporation is a prerequisite for the functional use of performance information.

Incorporation of performance measures means incorporating performance-related data into documents and procedures with the purpose of using them (Bouckaert and

Halligan, 2008). The reason for incorporating performance-related data is to make it possible to include this information in the process of decision making and potentially in the

organizational culture as well. The incorporation capacity of an organization is determined by the tools and techniques that can anchor information in documents and procedures. The use of performance information refers to debates and procedures that make use of performance information for policy and decision making (Bouckaert and Halligan, 2008).

Performance management thus is the overarching term for the measurement,

incorporation and use of performance information. Bititci et al. (1997) use another definition of performance management. They define it as „the process by which the organization

integrates its performance with its corporate and functional strategies and objectives‟. The essence of a performance management system is that it enables learning in an organization (Mwita, 2000), with the ultimate goal of improving performance. By trying to manage the performance of every individual, team and process, organizations try to meet their goals in an efficient and effective way (Sahoo and Jena, 2012). Related to the discussion about

(10)

9

As stated before, the main goal of performance management is ultimately to improve organizational performance. Walker et al. (2011) conducted a quantitative research to

investigate whether this theoretical goal was achieved in reality. Organizational performance was measured by performance scores on seven core service areas, given by external

evaluators. Performance management was measured by, what Walker et al. (2011) call, its components. In short, these components deal with the specification of operational goals, performance targets and objectives and whether there are deviations and how these deviations are communicated and corrected. Data from a set of English local governments were used and a significantly positive relation between performance management and organizational performance was found.

Several groups of factors can increase the likelihood of a performance management system being successful in improving performance.

First, according to Sahoo and Jena (2012), for a performance management system to be successful, it needs to be strongly aligned with the strategy of the organization. This is in line with Mwita (2000), who states that a performance management system should aim at improving results of employees‟ behavior by linking these to organizational goals.

Second, employees need to know what is expected of them (Sahoo and Jena, 2012). Participation of individual employees in the process of developing and implementing a performance management system can improve the effectiveness of performance management. This involvement creates trust and ownership towards the performance measures that are used (Sahoo and Jena, 2012), and decreases the level of resistance (Ter Bogt, 2004). It has also been argued that the beliefs of an individual about his own

capabilities positively moderates his performance (Bandura, 1995). If this view is correct, a performance management system should strive for the development of positive self-beliefs of the employees.

Third, top management has the power to increase the likelihood of a performance management system being successful. According to Sahoo and Jena (2012), there should be both agreement and commitment among top managers about the strategy and the ways to pursue it. Furthermore, according to Walker et al. (2011), innovativeness of management also increases the likelihood of a performance management system being successful.

(11)

10

useful feedback. Also, feedback should be strategic instead of short-term focused, because performance goals are on a strategic level as well.

Fifth, Locke and Latham (2002) and Buchner (2007) argue that performance goals should have a certain level of difficulty and specificity to have an impact on performance. Easy and vague goals are unlikely to lead to improved performance.

The sixth and perhaps the most questionable factor that is discussed here, is the use of a compensation plan. Sahoo and Jena (2012) mention that on the one hand a compensation plan can be used to avoid any discrepancies. On the other hand, however, they also note that when incentives are in place, they can lead to unwanted behavior, for example when

employees focus only on those elements that are measured and neglect the rest of their job. Several performance management techniques have been discussed in literature. A traditional performance management technique is a budgetary control system, in which performance is basically defined as profitability (Otley, 1999). Because the focus of a budget is more on control than on learning, it is not well suited for improving the performance of schools, however. A technique that enables management of both financial and non-financial measures of performance is benchmarking, in which an organization or business unit

compares itself with another organization or business unit on various aspects. This technique enables transfers of knowledge and innovation within and between organizations and

stimulates continuous improvement (Sahoo and Jena, 2012). The Balanced Scorecard (BSC) is a performance management technique that helps to translate a strategy into operational goals. The BSC adds three dimensions to the traditional financial perspective; these are the customer perspective, the internal business process perspective, and the learning and growth perspective (Kaplan and Norton, 1993).

One of the most remarkable differences between the private sector and the public sector with respect to performance management is the fact that outputs are often harder to measure in the public sector. Therefore, in the public sector, performance management was traditionally focused on inputs and costs (Kloot and Martin, 2000).

The current state of performance management in the Netherlands is that is used in public organizations already, but it is very likely that improvements can still be made. Ter Bogt (2004) argued that the use of performance management in the Dutch public sector was not optimal at the beginning of the new millennium. Interviews with Dutch aldermen

revealed that they used more informal performance information and performance

(12)

11

5. Empirical studies into performance management in the educational sector

This section reviews some empirical studies into performance management in the educational sector. It will conclude with a paragraph highlighting what this study wants to add to the existing literature.

Ter Bogt and Scapens (2012) did research into performance management at

universities in the Netherlands (Groningen) and England (Manchester). They describe how Dutch universities were not regularly evaluated on performance historically, but that this changed in 1982. Since then, the amount of funds depends on the „productivity‟ of research programs and on teaching performance. This „productivity‟ is measured by the number of publications in international academic journals, while teaching performance is assessed by course evaluations of students. According to Ter Bogt and Scapens (2012), measuring „productivity‟ in this way has some negative side effects, because both creativity and innovation are discouraged, as researchers will not take the risk of falling short on their quantity of publications. Measuring teaching performance in this way also has a negative side effect, as course evaluations are no longer perceived as „developmental‟ in order to improve the course, but as „judgmental‟ only.

Performance gained increasing attention due to increased competition between universities worldwide (Ter Bogt and Scapens, 2012). This increased focus on performance was found to be appreciated by most of the employees. However, it also led to pressure or even stress among employees. This was even more the case when evaluations of their performance were available for others.

Findings in England (Manchester) were similar. In both cases almost half of the respondents felt that the criteria on which their performance was evaluated were not clear enough. Also, in both cases, a majority of the persons is unhappy with the way in which performance is evaluated and major groups feel some kind of stress. And last, in both countries performance measurement has become more important at the universities.

Decramer et al. (2013) studied a Flemish university and focused on what they call “employee performance management”. This means that individuals in the organization are helped to understand what they contribute to the organization, and are stimulated to manage their own performance and the performance of the organization. They found that satisfaction among employees with the performance management system was positively associated with internal consistency of the system, the amount of communication and the tightness of

control. They also found differences in satisfaction among tenure types, which suggests that it might be considered to have a diversified performance management policy.

(13)

12

is that, at least in Portugal, performance management at secondary schools is

underdeveloped. They designed their study according to the three stages defined by Boukaert and Halligan (2006). Most schools had some kind of measurement. However, this was often not in a formal and systematic way. Incorporation was also far from being fully developed, making the process of going from measurement to actual use of performance information even more difficult. Little evidence was found that improvements were made based on the information of the performance management systems in place. Positive developments were observed concerning self-evaluation, but whether this was caused by internal motivation or external pressures was unclear. An example of such external pressures are school rankings published to the public, comparable with the increased transparency in the Dutch sector of secondary schools. Silva Portela et al. (2011) talk about this performance ratings and focus on an internet platform called Benchmarking of Portuguese Secondary schools (BESP).

Accountability of schools is promoted by this platform. Schools are encouraged to improve continuously and results are made more transparent to society. According to Sarrcio et al. (2012), their study highlights the need for developing more knowledge about performance management on schools.

(14)

13

3. Methodology

The goal of this research is to contribute to the existing knowledge on performance management and its effects on the performance of secondary schools in the Netherlands. The research approach followed in this study is theory development.

To make a contribution to the existing literature, a case study is conducted. This type of research is appropriate for investigating dynamics within a single setting (Eisenhardt, 1989). In this case, the dynamics of performance management within a Dutch secondary school are studied.

The effects of performance management on performance are investigated on three levels: the school level, the sections level and the level of individuals. Due to time constraints not all sections will be investigated; the focus will be on the economics section of this school. This enables conclusions on more levels, which is an addition to the existing literature, as other studies focused on a school level only (for example, Sarrico et al., 2012).

(15)

14

1. Case description

A secondary school in the eastern part of the Netherlands was selected as the object of study. This school was chosen, because next to all the regular levels of education, this school has some additional profiles. It is a public school that offers the opportunity to follow bilingual education, there is a special program for pupils talented in sports (there are only twenty of such places in the Netherlands), and there is an international school situated in the case school. The international school is meant for international pupils who are spending a limited period of time in the Netherlands, Dutch pupils who have received education on an international school abroad and Dutch pupils planning to receive education at an

international school abroad. These additional profiles can make managing the school more complex, possibly leading to extra difficulties or extra opportunities for a performance management system as well. This makes the school an interesting case object.

Figure 1 presents the organizational structure of this case school (location 1), which is part of a bigger organization consisting of eight locations. Although the eight schools have different locations, they do have a common long term policy and common core values. Ensuring quality and safety is the main aim of the long term policy. In addition, there are four common core values: creating opportunities and developing talents; being responsible towards society and towards each other; being transparent in communication and

accountability; and treating each other in a positive way. The organizational board and the management team of the school together take decisions about performance targets.

Every location has a headmaster in charge of a management team consisting of team leaders, who in turn manage groups of teachers and teaching assistants. The shaded area in Figure 1 is the object of this study. Concierges and administrative staff are also active at the case school, but due to their relatively minor direct role in educating pupils, and by that their minor role in performance management, they are not included in figure 1.

2. Method of data collection

Exploratory information was gathered from various websites. The website of the school itself provided the researcher with information about the strategy, mission and vision of the school. The website of the project „Vensters voor Verantwoording‟ was used as well, as it lists information about exam grades, satisfaction among pupils and their parents, the percentage of pupils finishing their study without delay, and other performance indicators. Both scores of the case school and the average national scores on these indicators can be found on this website.

(16)

15

provided me as a researcher with useful background information and it helped me in determining how to structure the rest of my study.

During the research period (February - June, 2013), the case school organized a „study day‟ for all its employees, in which the performance of the school was discussed. Two team leaders were responsible for a presentation about the results on a school level and on the level of sections, focusing on the exam grades of the year before, in particular on the two highest levels of education. The headmaster held a presentation about teams, highlighting that new teams would be formed, based on a few criteria of which the number of pupils covered by a team was the most important one. After these presentations, teachers of the same subject were encouraged to discuss ways to improve performance. Although the focus of this day was rather limited, as performance is much broader than just exam grades, it

provided me as a researcher with useful information that I could use mainly as background information. It highlighted especially the current level of (part of) performance and the indicators on which the school focuses. Information of this study day is also highlighted in the Analysis.

The main type of data acquired in this study is qualitative data about the performance management system and its effects on performance at the case school, which was obtained by conducting semi-structured interviews. After the before mentioned exploratory conversation with the headmaster, it was decided who to interview. Interviews were held with the (interim) headmaster, two team leaders and two teachers.

The headmaster was interviewed because he is ultimately responsible for the performance of the school. It concerns an interim headmaster, who was appointed in this function a few weeks before the study took place. This was not a limitation however, as he held the same position the year before and therefore knows how this school is trying to manage performance. From a total of six, the two team leaders responsible for a presentation on the study day were selected for the interviews. This can potentially lead to bias. However, the researcher determined that this was overcome by the researcher‟s expectation that these team leaders would be able to give more detailed information because of their focus on performance. Finally, two teachers of economics were selected to be interviewed, to represent the teachers in this research. This is important, because in the end they are the employees with the most direct influence on the performance of the school‟s pupils.

Interviews were not held with the head of the personnel department nor with the head of the administration. The head of the personnel department was not involved in the process of improving performance at the case school. In situations where the school collects

(17)

16

By interviewing employees from several levels in the hierarchy about the same phenomenon, data triangulation is assured, which provides increased confidence about the results (Jick, 1979) and improved reliability. Interviews were recorded to ensure that no information would be lost. On average, the interviews took about 45 minutes (see Table 1).

Date Interviewee Duration

25-04-2013 Headmaster 43 minutes 26-04-2013 Teacher 1 42 minutes 13-05-2013 Teacher 2 38 minutes 16-05-2013 Team leader 1 43 minutes 24-05-2013 Team leader 2 51 minutes

Table 1. List of interviews

Ferreira and Otley‟s „Performance Management Systems Framework‟ (2009) was used as a base for creating the interview questions, as the twelve original questions that form this framework „yield significant insight into the various aspects of performance management systems design and use‟ (Ferreira and Otley, 2009). These original questions were

transformed to make them more appropriate for the context of a secondary school. More questions were added, for example about the individual role of the interviewees within the performance management system at place. Interviewees were also asked for suggestions for improving this system and were encouraged to give their opinions about the current

performance management system and the use of it. Most questions are open ended, to make sure that the interviewees would not be directed into a certain direction or limited in telling their stories about the performance management system in place.

Appendix 2 provides the full list of questions used for the interviews. After conducting the interviews, extensive interview reports were made.

3. Method of data analysis

Reports from single interviews were coded. Codes were based on literature and the questions used in the interviews. Quotes from interviewees were first coded and then assigned to a certain heading or theme. Assigning data from different interviewees to the same theme enables a better and more comfortable analysis. It was analyzed whether there are important similarities or contradictions between the interviewees about a certain theme.

(18)

17

(19)

18

4. Analysis

Before the actual analysis, the mission of the case school and some information about the performance indicators in place are discussed, to demonstrate what the school wants to achieve and what it should take into account. The performance management initiatives in place at the case school can be broadly classified as resulting from governmental pressure, external pressure stemming from „Vensters voor Verantwoording‟ and initiatives of the case school or its employees itself. The analysis is built around this classification and, within this classification, around the dimensions of performance management: measurement,

incorporation and the use of performance information.

Mission of the case school

As pointed out in the Methodology, the case school is part of a bigger organization. Important core values are that every potential pupil is welcome, that everybody gets equal chances and that everybody is treated equally. In cooperation with partners, pupils are inspired to develop their talents. Ensuring safety and quality is the main aim of the long term policy. The school wants to create a climate in which everybody can focus on results in a pleasant way.

Performance indicators and their targets

The organizational board and the management team of the case school together determine which performance indicators to focus on and what performance targets will be set. The boundaries are set by the organizational board, but within these boundaries the management team is free to pursue the plans in its own way. After the management team, the teams or sections have an influence as well on the process of executing the plans.

1. Performance management and governmental pressure

Measurement of performance is, to a large extent, done by external parties. DUO (i.e. the executing organization of the Dutch Ministry of Education, Culture, and Science) and the Inspection of Education collect most performance data. They have an important influence on the performance indicators on which the school is focusing.

(20)

19

Especially in the superstructure [year 4 and 5 of HAVO and year 5 and 6 of VWO 1], we focus a lot on exam grades, also because the Inspection is very strict about that. The difference between exam grades at our school and the national average can’t be too large and the discrepancy between school exam and central exam is not allowed to be large.

– Team leader 1

This quote shows that the case school focuses on some indicators of performance, basically because they are required to do so by the government. Failing to meet this

requirement can have serious consequences for a school. There is no explicit maximum of the difference between the grades on the central exam of the case school and the national

average, but of course the higher the average grade of the case school, the better. For the discrepancy between the grades on the school exam and the central exam, however, the Inspection of Education sets a maximum of 0.5 points. In case that this criteria is not met for three years in a row, the school will experience enhanced surveillance by the Inspection of Education, which might ultimately lead to closure of the school.

We choose to focus on the exams because the discrepancy [the difference between school exams and central exams] at our VWO level simply is too large. The Inspection also stresses this point. In that area we are on a critical list. – Team leader 2

This quote also shows that governmental regulations, in the form of guidelines and targets from the Inspection of Education, have an important influence on which performance measures the school will focus on. Furthermore, with this quote, team leader 2 states what multiple interviewees point out: the main focus of the case school is on performance indicators on which the school has an „insufficient score‟ according to governmental institutions.

DUO and the Inspection of Education incorporate the data they collect in yearly reports that highlight their findings, which are provided to the school.

The school is required to read these reports. These contain the fixed data that the school is forced by law to report to the governmental authorities. The Inspection interprets these data and comes with an assessment. – Headmaster

1

(21)

20

These reports of DUO and the Inspection of Education showed that in past years the discrepancy between school exams and central exams was sometimes too large (more than 0.5 points). Last year, at VWO (the highest level of education) the exam discrepancy of this school was on average 0.49, while on the HAVO (the second highest level of education) the result was much better: 0.06. The teachers have an explanation for this relatively large exam discrepancy at VWO:

Former boards of this school were above all other things creating opportunities [for potential pupils]. They wanted to make a dollar out of a dime. In their eyes every pupil could attend VWO, but of course that’s not realistic. But if that’s your starting point, than you can expect large discrepancies: not every pupil can graduate on VWO. – Teacher 2

……….this led to the fact that pupils could flow to the next class too easily and by that we actually lowered our own standards. This happens automatically, you get weaker pupils on those higher levels and then you will adapt to the average of the class. – Teacher 1

By focusing on a better determination of pupils, the school tried to diminish this discrepancy. The term determination mainly refers to assigning new pupils to the right level of education. It also means being more strict and critical about letting pupils flow from, for example, HAVO 3 to HAVO 4. This was one of the main consequences of these reports. Another initiative to diminish the exam discrepancy was the modification of school exams, which are more like central exams than before, as the teachers argue:

In the superstructure we really aim at making our exams difficult enough. Too many fails no longer means that the difficulty level of the test was too high. We now look whether we can bring the level of the pupils to a higher level, so this really improved. – Teacher 1

I think that this led to higher grades on the central exams and slightly different education, slightly more exam oriented. Nowadays we also discuss the harder questions, while five years ago I might have skipped them. – Teacher 2

(22)

21

These measures, both a stricter determination and more central exam like school exams, which gained more attention in 2009 when the Inspection of Education introduced the escalation ladder, seem to have had a positive impact on the discrepancy of the case school. The escalation ladder shows that Dutch secondary schools are not allowed to have a big discrepancy (>0.5) for five years in a row or a very big discrepancy (>1.0) for three years in a row.

The average grade on the central exams has increased at the case school. The average grades on school exams slightly decreased, most likely as a result of an increase in the difficulty level (more like central exams). Table 2 highlights these numbers, which were collected from the website of „Vensters voor Verantwoording‟ and a spreadsheet discussed at the Study day.

2009-2010 2010-2011 2011-2012 Central exam HAVO 5.9 6.1 6.2 School exam HAVO 6.5 6.4 6.3 Discrepancy HAVO 0.42 0.20 0.06 Central exam VWO 6.0 6.2 6.2 School exam VWO 6.9 7.0 6.9 Discrepancy VWO 0.79 0.63 0.49

Table 2. Exam grades and discrepancy2

The discrepancy on VWO is still a point of concern, considering the fact that the limit is 0.5, while the case school scored 0.49. For the next year, however, the prognosis is positive. Official numbers from DUO are not available yet, but calculations by one employee

responsible for all numerical data and the researcher, show a (much) smaller discrepancy on VWO for the year 2012-2013.

Management stimulates teachers to improve scores on the performance indicators set by DUO and the Inspection. However, they do not make concrete plans. According to them, this is the task of the sections themselves, because they are more knowledgeable about their field and their pupils:

2

(23)

22

As managers we can only offer them a helping hand. We can display the data, we can have conversations, but the real content of the action plan must come from the teachers

themselves.………. Management can’t do that and I also think that we shouldn’t. For example, I have no idea what the economics section should change, I have no idea, I don’t know anything about economics [she has a background in biology]. – Team leader 1

It is likely that a school will work on those indicators on which governmental institutions place most value, but what is done to improve the performance on these indicators is less clear. Schools and, as this quote illustrates, teams or individuals within schools are free to make their own plans about how to achieve their goals (which are in part set by the Inspection of Education).

2. Performance management and external pressure (Vensters voor Verantwoording)

Besides the Inspection of Education and DUO, the project „Vensters voor

Verantwoording‟ determines to a large extent on which performance indicators the school focuses. Its internet platform provides a list of 20 performance indicators and the scores of all Dutch secondary schools on them. At the Study day it was announced that the case school wants to be among the best 50% of Dutch secondary schools. With the help of this platform, performance targets can be set in such a way that meeting these targets means meeting the objective of being amongst the best half of Dutch secondary schools. In cooperation with DUO and the Inspection of Education, the school itself has to provide „Vensters voor

Verantwoording‟ with data. In this way, the school is forced to measure performance, as the headmaster explains:

The pressure from outside has increased and is increasing, because society does something with quality policy………. That resulted in the project ‘Vensters voor Verantwoording’. This quality system is developed by the Council of Secondary Education………. If you decide not to join this project you have something to explain. Why not? In this way, society forces

education to become transparent. That means that you need to have proper measurement systems internally. – Headmaster

(24)

23

some other information about the pupils. Magister enables the transfer of data from the case school to „Vensters voor Verantwoording‟:

To a very large extent, this is done via Magister. A lot of numerical data come out of

Magister, for example educational time, and from here these data are put on ‘Vensters voor Verantwoording’. – Team leader 1

Magister offers a lot of tools, but at the case school only a fraction is effectively used at the moment, as the headmaster explains:

We have a very extensive and rich pupil support package [Magister], spanning everything from exam modules to communication with parents to the management box, you can do everything you want with it. Teachers can register their conversations, registrations of incidents can be saved, advancements, you name it, everything you can think of. Looking at the possibilities, only a portion of it is used. Teachers use it purely for registering grades. – Headmaster

Different reasons seem to play a role in why teachers do not make more use of this support package, like a lack of routine, a lack of knowledge and a lack of time:

Filling in the data and making this your routine is more of a problem than not knowing how to do it, I think. – Team leader 1

I think I’ve never had a course about how to use Magister in a smart way. One day I received my login data and I had to help myself: ask a colleague how it works. – Teacher 1

Partly, I can imagine that. When you’re giving class you don’t have time to do that [Filling in Magister]. – Team leader 1

These quotes show that if management wants teachers to make more use of Magister, various factors should be taken into account. They need to create time for teachers to do so and they have to make sure that everyone has knowledge about how to make us of Magister effectively.

„Vensters voor Verantwoording‟ incorporates the data it receives in figures and tables with comparisons with other Dutch secondary schools. Besides the information that is

(25)

24

management report meant for the organizational board, which contains confidential information about the school:

And from them, the board, we receive it as feedback. Like: those are the results and based on that you should build your conclusions. – Team leader 1

The management report is thus first sent to the organizational board, before the management team of the case school itself can see it. As the quote of team leader 1 shows, the organizational board already interprets the report and tells to some extent on what the case school should focus. However, both the management report and the public information have not led to concrete plans. This information is merely used as a point of reference; it highlights which performance indicators might need some extra attention in the future. The headmaster recognizes that this is a point of improvement:

Data driven is an unknown area for this school, but I think that is the case for many schools. For example a grade retention percentage of 30%, what does that mean? Have you observed that in time? Did you do anything about it? Did it lead to interventions with teachers? Or do you just think: the children became dumber, teachers are incapable, it happened only once? In general, that is the repertoire of people in schools when it concerns processes we have some control over, but of which the government has less strict rules for. – Headmaster

This quote of the headmaster again highlights that the case school focuses primarily on those performance indicators that the government finds important. Until now, the case school did measure the indicators of performance that are used for „Vensters voor

Verantwoording‟, but it did not actually use this information or the information it received from this project. The school is planning to make an important change, concerning this. The management team formulated a goal to be among the best half of all Dutch secondary schools. Trying to reach this goal is expected to lead to more use of the information of „Vensters voor Verantwoording‟. At the moment of writing this report, however, this is just a speculation.

3. Performance management initiatives of the case school

(26)

25

Visitation committee

An initiative of the organizational board was to hire a committee, consisting of an external chairman and some selected employees of the overarching organization itself, to visit the school. This collegial visitation focuses on specifically chosen points of reference for the case school. The findings of this committee have no official status, but according to the interviewees and the information provided to the personnel on the „study day‟, they form a helpful tool. The committee writes a visitation report, which provides the school with findings and recommendations concerning quality and performance.

It showed, for example, that some classes were a bit boring. Than you can sometimes see the links between these soft data and the hard results. – Headmaster

That some classes were perceived as boring is not a surprise. Perhaps more

interesting, therefore, are two other critical findings. The visitation report states that pupils are „insufficiently prepared to flow from substructure to superstructure‟, that means prior years of secondary education do not sufficiently prepare pupils for the last two years at the case school. This observation seems to be related to the observations of the teachers that the determination has not been optimal for many years. The visitation report also states that teams exist predominantly on paper (the teams are not actively used) and recommends new structured teams in which groups of teachers are assigned to groups of pupils. Team leader 2 strongly agrees with this last observation:

I have people in my HAVO team, who teach HAVO classes only three hours per week. The rest of the week they teach at VWO. I know that I’m responsible, because their name is written under my name and so they belong to my team, but I can’t observe what they do. I don’t hear from the pupils what they’re doing, so I don’t get any feedback. – Team leader 2

The recommendations in the visitation report led to a few plans. The effects of these plans are, at the moment of writing this thesis, uncertain, because most plans will be

(27)

26

This example of a visitation committee highlights that initiatives of the case school itself provide the school with additional information, compared to the performance information discussed before, that the school had to measure and use because of external pressures. It provides the case school with quite specific recommendations.

Pupils’ council and consultative groups

The pupils‟ council and consultative groups of parents are other initiatives of the case school with the objective of managing performance:

We also have, for example at our bilingual education department, a consultative group. A group of parents gathers once in a while at school to discuss with us how they perceive how things are done at our school………. Last year we had an active pupils’ council as well. This year we also have a council, but I have to admit that this one is not that active. Especially thanks to the person who chaired this last year, it had much effect. – Team leader 1

The main reason why this year‟s pupils‟ council is less active is the fact that most of the former members graduated last year and suitable successors were hard to find.

Findings of these ways of measuring performance are incorporated in internal dossiers. The same goes for suggestions or complaints of pupils in general. Information stemming from the pupils council is used relatively soon after the „measurement‟ of it. Sometimes the stage of incorporation is even skipped and initiatives take place almost immediately, as the examples that team leader 1 mentions show:

Last year, the exam pupils wanted to have an extra locker. When we found out that some lockers were left, we gave the exam pupils the opportunity to ask for one extra.

– Team leader 1

In contrast to what most literature tells us about performance management, this example shows that sometimes the phase of incorporation is not necessary.

(28)

27

Pupils also mentioned that not on every test the number of points available was written in the margin. We paid attention to this by going to our colleagues to remind them to do so in the future. That kind of improvements came from the pupils’ council. – Team leader 1

Mentioning the number of points available for all questions on a test will probably lead to higher grades, because pupils can focus more on questions with higher weights.

The use that is made of information coming from the consultative group of parents is comparable with the pupils‟ council. Information is reported in dossiers, but actual use of the information sometimes happens immediately as well. For example, a parent of a pupil who attended bilingual education remarked that one teacher had the same class for several hours in a row, because he taught them different courses. After this remark, some changes in the timetable were made to resolve this „problem‟. These kinds of actions not only have a positive effect on the pupils, but also on the satisfaction of parents, which is an indicator of

performance itself.

These are other examples that show that incorporation is not always a necessary condition for the use of performance information. General complaints of pupils are dealt with in a comparable way. The management team tries to solve any existing problems that appear from these complaints as soon as possible. Especially when managers hear the same kind of critique about a particular teacher several times, they will discuss this with the teacher in order for him to change his behavior.

Survey

DUO conducts a yearly survey among third year pupils about their level of

satisfaction. Because this provides schools with a rather limited view, as only one specific group of pupils is asked about its satisfaction, the case school decided to conduct a survey itself among pupils who are in their final year of secondary education. This was done in order to receive richer insights into pupils‟ perceptions of the performance of the school, because last year pupils experienced the whole process from entering the school to graduation. Data from this survey among last year pupils were incorporated in internal dossiers.

Findings from this survey resulted in some changes that are aimed at improving performance. Pupils appeared to be dissatisfied with the exam preparation of the school. This led to a new structured exam period, with more time for pupils to study and exam trainings offered by the school, as teacher 2 explained:

(29)

28

These extra initiatives of the case school, next to everything that for example DUO does, provide the school with extra information. Without its own initiatives, the case school would have less knowledge about what it can change to improve its performance.

Individual initiatives

The before mentioned examples of performance management initiatives were all on a school level. However, some teachers themselves try to measure and improve their

performance too:

For the last five years I do some research myself, I put all exam grades in tables and look for discrepancies.……… Internally, we do some results analysis, myself as well, but I don’t know to what extent others do the same. This is my own initiative. – Teacher 2

Whether this is done seems to depend mainly on the motivation of individual teachers. This initiative of teacher 2 was, according to himself, supported by the organizational focus on exam grades and discrepancies.

Myself, I make summaries since a few years and my pupils receive them at the end of their ‘career’ at this school. That’s some kind of extra service, but I do hope that thanks to this they are even better prepared [for their exams], by glancing through everything they need to know one more time. – Teacher 2

This shows that initiatives of the case school influence performance in two ways: directly by these initiatives and indirectly by influencing or supporting individuals to take some initiatives themselves. However, the headmaster thinks that there is room for improvement concerning „measurement‟ and self-evaluation of individual teachers:

A teacher never asks his pupils what they think about him and his classes. They [the teachers] experience that as a judgment. They should learn that it is completely unthreatening. That information is for themselves. But they are allergic to rankings and having an opinion about someone else. That is a no-go area. – Headmaster

(30)

29

performance of other sections. The sections with relatively low scores are motivated to perform better next year, because of this „public shame‟.

Looking at all the performance management initiatives of the case school discussed in this section, it can be said that the school is both reactive and proactive. The case school focuses on the performance indicators that are, according to government, important.

Especially these indicators for which the Inspection of Education sets explicit standards and the ones on which the school scores insufficiently at the moment gain attention. The

(31)

30

5. Conclusion

In order to answer the research question of how performance management can affect the performance of Dutch secondary schools, a case study was conducted. This chapter presents the main findings.

First, to a large extent, external pressures determine on which indicators of performance the case school focuses. These external pressures can be subdivided into governmental pressure and pressure from the project „Vensters voor Verantwoording‟.

The case school primarily focuses on those indicators of performance for which they are required to do so, especially on the exam discrepancy, because the Inspection of

Education sets a maximum for this indicator. Failing to meet this requirement can have serious consequences, possibly leading to closure of the school. Because other Dutch schools face the same governmental rules, it can be expected that the performance management initiatives of every Dutch school are strongly influenced by governmental pressure. It also means that the main focus of every school is likely to be on those performance indicators on which the school currently has an „insufficient score‟ according to governmental institutions.

External pressure stemming from „Vensters voor Verantwoording‟ also determines to a large extent which performance indicators the case school measures. Every Dutch school is expected to provide this project with information on 20 indicators. Therefore, it can be expected that studies at other Dutch schools yield similar findings concerning the role of this project. It might differ per school, however, to what extent the school uses the performance information on „Vensters voor Verantwoording‟. The case school did not make much use of this information so far, but by setting a goal of being among the best half of Dutch schools, they will make more use of it in future years.

Second, (informal) initiatives of the case school itself form an important part of performance management. At the very least, they form a useful addition to the formal initiatives of governmental institutions like DUO and the Inspection of Education. Reports from these institutions show some points for improvement, but are not that concrete as what can be obtained by initiatives of schools themselves. For example, formal reports of DUO can show that the exam grades are too low. Initiatives of the case school itself, most of them informal, can give insights into the reasons for these low grades. Schools that are trying to improve their performance therefore are expected to have some initiatives of their own in place, of which a visitation committee, pupils‟ councils, consultative groups and surveys or other forms of research are examples. In order to reap more benefits of performance management, schools should not limit themselves to formal instruments only.

(32)

31

contradiction to the existing literature (for example, Bouckaert and Halligan, 2008), this study showed that the second phase from the original three steps model measurement-incorporation-use can be skipped sometimes and information that is just „measured‟ can be used immediately, as the examples of the pupils‟ council and the consultative group showed. This is likely to be the case on other schools as well, especially when it concerns less formal performance management initiatives like the pupils‟ council. Skipping the phase of

incorporation saves work and time and might therefore be a useful help in improving performance. It might be a good idea, however, to incorporate some information after the actual use. For example, measures that took place based on the information should be written down and saved. By doing so, a school prevents that the same problems appear in future years.

Fourth, performance management initiatives on a school level can affect performance both directly and indirectly. Performance is directly influenced by these performance

management initiatives on a school level themselves. These school wide initiatives influence performance also indirectly, by stimulating individuals within the school to take some initiatives themselves. Whether this is done seems to depend mainly on the motivation of individual teachers. Schools can stimulate individual teachers to improve their own performance, for example by discussing the results per section on a meeting with all employees, as was done at the „study day‟ at the case school.

By these four conclusions, this study added to the existing literature of how

performance management can affect performance, in particular on secondary schools in the Netherlands. It is expected that Dutch schools focus primarily on those indicators of

performance that are determined by external pressures. For proper management of

(33)

32

6. Discussion

Contribution to the existing literature

This study contributes to the existing literature by looking at performance

management within one school at several levels. Performance management initiatives and their effects at the school level, the section level and the individual level were discussed, while previous studies focused on one level only. At the case school, most decisions concerning performance were taken on a school level. In turn, the sections could decide themselves in what way they would try to improve performance. By initiatives at the school level and section level, individuals were encouraged to improve their individual performance as well. Furthermore, to my best knowledge, this is the first study that investigated performance management at Dutch secondary schools.

Practical implications

It should be emphasized that the findings of this study are not generalizable to all Dutch schools. All Dutch schools do have many similarities to the case school however, for example every schools faces increased transparency of performance information via the project „Vensters voor Verantwoording‟. Also, every school has to deal with the Inspection of Education and its requirements. Because of these similarities, it can be expected that to some extent the conclusions will hold for other Dutch schools as well.

A practical implication resulting from this study is that secondary schools should not limit themselves to the formal performance management initiatives required by

governmental institutions. Initiatives of the school provide the school with richer information and more concrete plans about how to improve performance. Relatively easy ways for schools to achieve this are introducing a pupils‟ council and a consultative group of parents. Schools can also conduct some research that adds to the survey of DUO among third year pupils, for example a survey among pupils in other years of education. The increased transparency of performance information via the project „Vensters voor Verantwoording‟ makes this even more important for Dutch schools, because potential pupils and their parents can base their choice of a school on this.

(34)

33

Limitations

Like every study, this research has its limitations. The factor time was an important one. Due to time constraints only a limited number of employees could be interviewed. Interviews with more team leaders and teachers from various sections would have provided the researcher with richer information and probably an even broader view on performance management within the case school. A longer research period, for example a couple of years, would enable the researcher to observe and measure potential effects of new implemented plans stemming from performance management himself.

Another limitation of this study is the fact that it concerns a single case study. Based on the data collected at this single school it is impossible to draw conclusions that contain generalizations applicable to every Dutch secondary school. The factor time was the most important constraint here as well. Also, a single researcher conducted this study. A group of researchers could possibly have conducted a similar research on several schools, which should lead to richer insights and results that can be more generalizable.

A last limitation is the difficulty of defining performance of secondary schools. In this study, smaller exam discrepancies, higher exam grades and a higher satisfaction of pupils were all linked to a better performance of the school. However, while the exams are an end station for the schools, they are just a beginning for the pupils. The question whether they are well prepared for their next study or for their job, or for the future in general, is not answered but should form an important part in determining the performance of secondary schools.

Recommendations for future research

During the interviews and the analysis some interesting findings appeared that do not directly answer the research question, but that do have a link with it. The interviewees all agreed that the structure of the case school had an influence on performance and

performance management. For example, pupils following bilingual education have some extra requirements to fulfill, compared to regular pupils. Some pupils of the profile for pupils talented in sports have trouble with the courses other than sports. Teachers mentioned that they had difficulties with the International School: they found it hard to prepare well for both types of schools. It could be investigated what the role of the structure of a school is on its performance management. Furthermore, all interviewees agreed that changes in the top management team have a great influence on performance and the management of

performance. Future research can investigate the role of managerial changes or the level of stability in managing performance.

Future research can also focus on performance management at secondary schools in countries without or with less external pressure to improve performance. It can be

(35)

34

initiatives to improve their performance. In this way, the influence of external pressures on secondary schools can be assigned.

The last recommendation for future research is to conduct research at several schools with the lowest level of performance and at several schools with the highest level of

(36)

35

References

Armstrong, M., Baron, A. (1998). Performance Management Handbook. London, UK, IPM.

Atkinson, A.A., Waterhouse, J.H., Wells, R.B.. (1997). A Stakeholder Approach to Strategic Performance Measurement. Sloan Management Review. 38 (3), p25-37.

Bandura, A. (1995). Self-efficacy in changing societies. Cambridge, UK: Cambridge University Press.

Bititci, U.S., Carrie, A.S., McDevitt, L.. (1997). Integrated performance measurement systems: an audit and development guide. The TQM Magazine. 9 (1), p46-53.

Bititci, U.S., Garengo, P., Dörfler, V., Nudurupati, S.. (2012). Performance Measurement: Challenges for Tomorrow. International Journal of Management Reviews. 14 (-), p305–327.

Bogt, H.J. ter. (2004). Politicians in Search of Performance Information?- Survey Research on Dutch Aldermen's Use of Performance Information. Financial Accountability &

Management. 20 (3), p221-252.

Bogt, H.J. ter, Scapens, R.W.. (2012). Performance Management in Universities: Effects of the Transition to More Quantitative Measurement Systems. European Accounting Review. 21 (3), p451-497.

Bouckaert, G., Halligan, J. (2006). Performance and Performance Management. In B.G. Peters & J. Pierre (eds.), Handbook of Public Policy (p443-459). London, SAGE Publication Ltd.

Bouckaert, G., Halligan, J. (2008). Managing Performance: International Comparisons. Abingdon, UK: Routledge.

Bouckaert, G., Peters, B.G.. (2002). Performance Measurement and Management: The

Achilles‟ Heel in Administrative Modernization. Public Performance & Management Review. 25 (4), p359-362.

Broadbent, J.. (1995). The values of accounting and education: Some implications of the creation of visibilities and invisibilities. Advances in Public Interest Accounting. 6 (-), p69-98.

Bromwich, M.. (1990). The case for strategic management accounting: The role of accounting information for strategy in competitive markets. Accounting, Organizations & Society. 15 (1), p27-46.

Buchner, T.W.. (2007). Performance management theory: A look from the performer's perspective with implications for HRD. Human Resource Development International. 10 (1), p59-73.

Coens, T., Jenkins, M. (2000). Abolishing Performance Appraisals: Why They Backfire and What to Do Instead. San Francisco, US: Berrett-Koehler Publishers.

PERFORMANCE MANAGEMENT AT DUTCH SECONDARY SCHOOLS