To be stable or not to be stable, that is the question!

(1)

Environmental dynamism, stability and organisational performance. The long term.

Master Thesis

BA(hons) Daniël Reinders

25-6-2020

Wordcount: 10580

Supervisor: Dr. P.E.A. van den Bekerom

Second reader: Dr. J. Schalk

Leiden University Public Administration

To be stable or not to be stable,

that is the question!

(2)

Introduction

Which organisation does not care about its performance? For a private company performance can refer to the profit it makes and for a public organisation performance can refer to the number of satisfied customers for example. Both are aspects of organisational performance. General theories about public organisational performance exist within management literature on performance. These are generally based on two models of organisational performance: the 3 e’s and the IOO model (Andrews, Boyne and Walker, 2010). Both these models focus on the costs, the quality and the efficiency and can be applied to both public and private

organisations. Although these models are not without criticism (see Boyne 2002 for example), they are helpful to get a quick image of what is meant with organisational performance.

Performance can be influenced by several factors like the market demand for the goods or service that an organisation delivers or by the satisfaction of the workers within the organisation. The latter can be influenced via management (Meier and O’Toole, 1999), whereas the first is more difficult to deal with. To theorise the role of management in

organisational performance, Kenneth Meier and Laurence O’Toole (1999) developed a model. This model proposes that organisational performance is influenced by past performance, environmental shocks and three aspects of management (buffering against environmental shocks, exploiting the environment for resources and contributing to organisational stability).

Environmental shocks are changes in the environment of the organisation that affect its performance. According to Meier and O’Toole (1999), “shocks can come from a variety of forces in the environment” (p.512). Changes in the market demand can be an example of suck a shock. These shocks can be broken down into two categories: crises and dynamism. A crisis often has short-term consequences which need immediate action, but dynamism has

consequences that lead to structural problems on the long term (Van den Bekerom, Schmidt and Broekema, 2018).

Environmental dynamism and what public managers can do to cope with it has been studied by several scholars. The 1999 Meier and O’Toole model is an example of this. They tested their model in 2001 using data from the Texas Education Agency and found that network management had an influence on performance (Meier and O’Toole, 2001). In 2009, with the same data, they found that managers can diminish the negative effects of changes in the external environment by maintaining structural stability (Meier and O’Toole, 2009).

In an article of the same year by Kenneth Meier and Georg Boyne it is concluded that public managers can mitigate the negative effects of environmental shocks on the

(4)

performance of public organisations by maintaining structural (in this case personnel) stability (Meier and Boyne, 2009). In a study of the 2004 migration crisis in the United Kingdom and its impact, it was found that a strong central administration and a vibrant community

organisational life helps to diminish the impact of environmental shocks. (Andrews et al, 2013). A strong central administration can be seen as an example of structural stability. So, emphasising stability appears to be a good way to deal with environmental dynamism as public manager.

The question is, if this is also beneficial on the long term. It would be logical, considering contingency theory, that on the long-term stability has a negative impact on organisational performance. If an organisation is stable, it is less likely to adapt. As shown by Meier and Boyne (2009) for example, stability is beneficial to performance, but on the long run, stability may prevent changes that are needed to keep the fit with the environment. If the environment changes, but the organisation does not, according to contingency theory, this will have a negative influence on performance. The influence of stability emphasising stability or change has been studied by Boyne and Meier (2009), but only over an eight-year period. This thesis tends to find out in what way an emphasis on stability by public managers to cope with environmental dynamism influences organisational performance on the long term. To

examine this, the same database of the Texas Education Agency, used by Boyne and Meier (2009) and Meier and O’Toole (2009), will be used with similar variables as Boyne and Meier (2009) used, but this time over a period twice as large as Boyne and Meier’s: 16 years.

In the first part of this thesis the theoretical framework will be presented with the operationalisation of the concepts used. After that the method will be presented followed by the findings. Lastly the conclusion is drawn.

Theoretical framework

In this section important theories and theoretical findings related to the concepts

environmental dynamism, organisational performance and stability will be discussed, in order to create an understanding of what is meant with them in this thesis.

Performance

Organisational performance can be broadly defined as the extent to which an organisation achieves its goals, but specific dimensions of performance have been theorised by scholars.

(5)

Indicators used for measuring performance are generally based on two models of

performance: the 3e’s and the IOO model (Andrews, Boyne and Walker, 2010). The three e’s model draws a clear line form economy, the cost of procuring specific service inputs of given quality, to efficiency to effectiveness (Boyne, 2002). The IOO model has these three

indicators as well but makes some aspects, that are not really clear in the three 3 e’s model, more explicit (Boyne, 2002). The model stands for input, output and outcomes. Input includes expenditure and is similar to economy in the 3e’s model (Walker, Boyne and Brewer, 2010). Outputs are the quantity and the quality of the service. Outcomes refers to the effectiveness of the 3e’s model, but includes impact equity and fairness (walker, Boyne and Brewer, 2010). In this model, inputs, lead to outputs, which lead to outcomes. With efficiency, the input output relationship is meant. The efficiency in this model can be measured by the cost per unit of outcome (Boyne, 2002).

In his 2002 article George Boyne sums up some problems with these models. Both lay a strong emphasis on input measures of performance and less on output measures (Boyne, 2002). Other critiques of Boyne (2002) are that too little attention is given to responsiveness, for example customer satisfaction, that the relationship between internal and external is out of balance and that democratic outcomes lack. Although Boyne’s (2002) article has 141 citations in the social sciences citation index, other scholars still use the 3e’s method as well (SSC, n.d.). Liu, Cheng, Mingers and Meng (2010), for example, published an article on measuring performance via the 3e’s method in 2010, eight years after Boyne’s (2002) article. It probably depends on the kind of output that one wants to measure as scholar. If one is only interested in efficiency, for example, the 3e’s model is sufficient for that. But, if one is interested in, for example, democratic outcomes as well, the 3e’s model is not sufficient. So, it depends on what the researcher wants to measure and, of course, on the available data.

Given the complexity of the goals of public organisations, a more accurate

measurement is achieved when the extra factors of Boyne’s model are taken into account as performance indicators for measuring the performance of a public organisation too. Boyne’s (2002) model isolates five conceptual categories. The first category is output, which can be measured in the quantity and quality of the services (Andrews, Boyne and Walker, 2010). The second category is efficiency, which can be measured as cost per unit of output. The third category consists of service outcomes, which means the achievement of formal objectives (Andrews Boyne and Walker, 2010). The fourth category is responsiveness, with measures of satisfaction of direct service users (Andrews, Boyne and Walker, 2010). The fifth category consists of democratic outcomes, which have to do with participation, accountability and

(6)

probity (Andrews, Boyne and Walker, 2010). A public organisation that scores high on all these performance indicators, can be regarded as having a high organisational performance.

Although Boyne (2002) has good arguments for his critiques on the incompleteness of the way in which performance is understood, for pragmatic reasons the focus of this thesis will be mainly on output performance. This does not mean that the other four categories are less important, but output is one of the easiest categories to measure. As will be explained in more detail in the method section, the Texas Education Agency has a rich database with longitudinal information about the performance of school districts. A limitation of this database, however, is that it doesn’t allow measuring each performance dimension of Boyne (2002). Therefore, following Boyne and Meier (2009), the focus will be on output

performance mainly, with a small overlap with service outcomes.

Modelling the impact of Public Management on Performance

A very influential model for related to management and organisational performance was designed by Kenneth Meier and Laurence O’Toole (1999). Their model is devised to explain how management matters the performance of public organisations (Meier and O’Toole, 1999). Their model, in the form of a formula for organisational performance consists of five variables, which can be subdivided into more variables. In their 1999 publication, and in later articles, they make several of these subdivisions, but the essence of the model stays the same. The dependent variable is the current organisational performance. This performance is influenced by multiple independent variables: Stability, past performance, managerial activity, environmental shocks and an error variable (Meier and O’Toole, 1999). These variables together explain, according to Meier and O’Toole (1999), why public management has an impact on organisational performance.

The basic model looks like this: Ot=ß Ot-1+ ε (Meier and O’Toole, 1999). O is the outcome, in this case organisational performance. ß is the amount of stability and ε are the environmental shocks to the system. Throughout the article, types and results of management are added. The first addition to the model is the type of organisational structures, of which Meier and O’Toole (1999) identify two: hierarchies and networks. Hierarchical organisations are characterised by stability. They can be seen as “a stable set of relations in which the positions are arrayed in a pattern of formally superior-subordinate authority links” (Meier and O’Toole, 1999, p.508). Networks, on the other hand, can be characterised by flexibility. They “are not well established but are in formation or flux due either to the establishment of a

(7)

relatively new program or shift or perturbation in the environment in which an existing program has operated” (Meier and O’Toole, 1999, p.508). The organisational structure is a management decision and if the manager, for example a lower manager, cannot change the structure rigorously, he or she, can influence it a bit by, for example, adhering to the devised hierarchy in the cooperation with his subordinates, or cooperate in a more network like structure.

The second addition to the model is buffer. In 1967, Thompson gave a classic

depiction of core managerial functions, in which he included protecting organisations against disruptions. When reading about managerial buffering, the term slack arises quickly.

According to Thompson (1967) and Galbraith (1973), slack can serve as a buffer which helps organisations to absorb and survive the effects of shocks form the environment. Slack

therefore is conceptualised as “resources that can, if needed, be mobilized as inputs for the technical core during turbulent times” (O’Toole and Meier, 2012, p.186). Slack is a form of buffering, but what is buffering exactly? A very clear definition is formulated by O’Toole and Meier (2012): “we refer to any of these influences that reduce the impacts of environmental forces of organizational or performance results as buffers, and we refer to the dynamic of reducing such influences as buffering” (O’Toole and Meier, 2012, p219). So, buffering is the act of reducing the impact of environmental shocks, whereas buffers are resources that an organisation has to absorb environmental shocks. In the model buffers are accounted for by using the reciprocal of hierarchy as the factor that discounts any environmental shocks (Meier and O’Toole, 1999). Like the organisational structure, creating a buffer is as well a

managerial decision and adds to stability because it allows the organisation to continue its work when a shock occurs.

The third and final addition to the model is the division between internal and external management. The complete Meier and O’Toole (2009) model looks like this: Ot=ß1(S+M1)Ot-1+ß2(Xt/S)(M3/M4)+ εt

Where,

O is some measure of outcome, S is a measure of stability,

M denotes management, which can be divided into three parts

M1 management’s contribution to organizational stability through additions to hierarchy/structure as well as regular operations,

M3 management’s efforts to exploit the environment of the organisation, M4 management’s effort to buffer the unit from environmental shocks,

(8)

X is a vector of environmental forces, ε is an error term,

the other subscripts denote time periods, and

β1 and β2 are estimable parameters (at times M3/M4 is defined as external management or M2).

The alert reader will have noticed that the earlier factor hierarchy in the model is now referred to as stability. This is because the version displayed above is a later version of the Meier and O’Toole model, in which they changed the term. The underlying concept, however, did not change. A hierarchy is characterised as stable and a network structure as unstable. Therefore, an organisation that is close to a hierarchy has a very stable organisational structure. So, actually the stability of the organisational structure is meant with the hierarchy of the

organisation. Stability in this context can be defined as the extent to which the organisational structure is stable.

So, how does this model work? The idea is that current performance is determined by five factors: past performance, stability, internal management, external management and environmental forces or shocks. If nothing changes, it is likely that an organisation that performed well in the past will continue to do so in the present. If the organisation is stable there is less uncertainty, which has a positive influence on performance. Every employee knows what to do. Internal management can contribute to the performance of employees (Vermeeren, Kuipers and Steijn, 2014). External management for example, influences

performance by creating useful connections that can create more efficient collaborations with other organisations (O’Toole and Meier, 2011). If something changes in the environment, for example one of the organisations with which the organisation cooperates goes bankrupt, this is problematic for the organisation and negatively influences its performance. The impact of this shock can be reduced via the buffer of the organisation. If the buffer is big enough the shock can be observed, and if not, the impact can be reduced.

Environmental shocks: crises vs dynamism

What are these shocks? According to Meier and O’Toole (1999), “shocks can come from a variety of forces in the environment” (p. 8). A public organisation is not an island. It has other elements around it which form the organisational environment. The environment of an

organisation can be defined as the set of elements outside the organisations boundaries that can potentially influence the functioning of the organisation (Daft, 2010). These, for example

(9)

political, economic and ecological elements are constantly subject to change (Emery and Trist, 1965; Aldrich, 2008).

These changes can be divided in two categories: dynamism and crises. Gregory Dess and Donald Beard (1984), following an older concept of Howard Aldrich, define

environmental dynamism as the stability-instability of, and the turbulence in, the environment. However, not all changes in the environment are dynamism. The

unpredictability of changes is important for dynamism (Dess and Beard, 1984). A change in demand for ice-cream in winter, for example, is probably easy to predict and therefore not an example of dynamism. A rival company starting to sell ice-cream, which is not predictable, here can be called a form of dynamism. Another important aspect of dynamism is the

interconnectedness between environmental elements (Dess and Beard, 1984). If, for example, the price in paper increases, this can have a negative influence on the performance of

universities because they need to spend more budget on buying paper for student exams. This is an example of an environmental change that is not directly connected to the performance of universities, but still affects it via the interconnectedness of the environment of universities.

Crises influence organisations as well but are different from dynamism. “An

organizational crisis is a low-probability, high-impact event that threatens the viability of the organization and is characterized by ambiguity of cause, effect, and means of resolution, as well as by a belief that decisions must be made swiftly” (Pearson and Clair, 1998, p.60). A crisis, as opposed to dynamism, can occur inside the organisation as well. It can be caused by a bribed employee for example. Another difference with dynamism is that a crisis is a high-impact event, whereas dynamism be caused by several events due to the interconnectedness (Dess and Beard, 1984). As well the effect is different. A crisis is identified as a threat and asks for a direct response, whereas dynamism can be less clear and just cause uncertainty on the course of action (Dess and Beard, 1984). Thus, a crisis is a single event causing a lot of direct damage, whereas dynamism refers to a change in the stability, or instability, of and the turbulence in the environment. While both crises as dynamism qualify as environmental shocks, the focus of this thesis will be on environmental dynamism.

Stability

What is meant with stability? In their 2012 book O’Toole and Meier define stability simply as “constancy in the design, functioning and direction of an administrative system over time” (p. 135). They distinguish five dimensions of stability: structural stability, mission stability,

(10)

production or technology stability, procedural stability and personnel stability (O’Toole and Meier, 2012). Structural stability refers to organisational features like size, formalisation, span of control etc. Mission stability refers to the consistency of the goals, production to the means that are used whereas procedural stability refers to the stability in the procedures that are used. Personnel stability refers to the employees within the organisation.

Personnel stability can be split into two categories: vertical and horizontal stability. Vertical stability is “constancy in the overall assigned hierarchical roles of the organization” (Boyne and Meier, 2009). Organisations with a high level of vertical stability are constant in the number of employees that have a certain task, for example the staff for catering or the number of secretaries. Horizontal stability is stability of specialisations within these groups (Boyne and Meier, 2009). So, for example, the percentage of catering staff that does the dishes. Simply put, vertical stability is the stability of roles in the organisations and horizontal stability is the stability within these roles.

Following Boyne and Meier (2009) the focus of this thesis will be mainly on

personnel stability. This is for the same reasons as the focus on output performance. The data in the database of the Texas Education Agency that can be used to measure stability is very rich on personnel stability. However, changes in personnel stability also reflect changes in structural stability. If, for example, the number of directors diminishes, not only the personnel stability diminishes, but the structural stability as well because there are less directors, which changes the level of formalisation.

From model to practice

Does the model work in practice as well? A study from 2013 by Andrews, Boyne, O’Toole, Meier and Walker shows that the model works in a more crisis like situation, the migrant increase of 2004. After the EU had been enlarged that year, many high-level workers from Eastern Europe migrated to the UK. This immigration “was significantly associated with worse performance, but governments with higher levels of administrative capacity were able to mitigate this effect, as were those with high levels of community capacity within their jurisdictions” (Andres, Boyne, O’Toole, Meier and Walker, 2013, p. 191). This study shows that a buffer, in this case a high level of administrative capacity, helps to diminish the negative effect of a crisis.

In other countries aspects of the model were tested as well. A study by Van den Bekerom, Torenvlied and Akkerman (2016) shows, based on 546 Dutch primary schools, that

(11)

environmental dynamism, measured as changes in the percentage of pupils, negatively affects school performance. They found as well that internally oriented networking activities, for example team involvement, attenuate the negative effect of environmental dynamism in school performance rather than externally oriented networking activities. This study shows that the internal management of the Meier and O’Toole model (2009) has an influence on organisational performance. The model is as well tested by some South Korean studies. For example, Young Han Chun and Miyeon Song (2017) test the Meier and O’Toole model on schools in Korea. They find that “the main proposition of the Meier and O’Toole model, that the quantity and the quality of managerial efforts make a significant difference in

organisational performance, seems to be a valid observation in Korean schools” (Chung and Song, 2017, p. 1059).

But, most of the studies that test the model use data form the United States, especially from the Texas Education Agency. Meier and O’Toole do this in 2003 and find “empirical support for key elements of the network-management portion of the model” (Meier and O’Toole, 2003, p. 689). In 2006 Meier O’Toole and Yi Lu find, based on data form the Texas Education Agency and survey results from superintendents at the Texas schools, that

managerial quality and personnel stability contribute positively to performance. In 2009, Meier and O’Toole use the same tools to conclude that, when faced with environmental shocks “decisions about internal resource allocation and personnel management can be shown to protect core production while sacrificing more peripheral activities and capital investment” (Meier and O’Toole, 2009, p. 485). The idea that emphasising stability is the best way to respond to environmental shocks is tested by Boyne and Meier in 2009. They use the Texas Education Agency database over an eight-year period to test how changes in student numbers, funding and student composition affect performance and how stability mediates this impact. In other words, they test if an emphasis on stability as a response to environmental dynamism diminishes the impact of on organisational performance. They find that “turbulence has a negative effect on performance, and that this is compounded by interorganizational change” (Boyne and Meier, 2009, p. 799). In 2012 they find again that personnel stability is positively and significantly related to organisational performance.

To be stable or not to be stable

It is interesting that Boyne and Meier (2009) and Meier and O’Toole (2012) find this positive influence of stability on organisational performance. This is interesting because it is

(12)

questionable if the influence of an emphasis on stability as a reaction to environmental dynamism positively influences organisational performance as well on the long term.

Contingency theory would argue against this. This theory emerged from earlier research into the area of organisational theory (Woods, 2009). Contingency theory suggests that the activity and structure of complex organisations is influenced by contextual variables like technology and environment (Waterhouse and Tiessen, 1987). Richard Schott (1981) gives a clear explanation of the contingency theory “the best way to organize depends on the nature of the environment to which the organization must relate” (Scott, 1981, p. 114). According to Aldrich (2008), it is hard to culminate the results of contingency research because there is no consensus on the theoretical framework. Nevertheless, the idea of contingency theory is quite simple. If there is a change in the environment, for example in the demand, the organisation is not contingent anymore because it delivers products that, in this case, nobody has a demand for. This suggests that it is important for organisational performance to be contingent with the organisational environment. In line with this it would be logical that the best response to environmental dynamism is to adapt to it.

However, this is in contrast with the findings of Boyne and Meier (2009) that emphasising stability is the best way to diminish the impact of environmental dynamism on organisational performance. If an organisation is stable it performs well (Boyne and Meier, 2009). If nothing changes, however, an organisation faces the risk that necessary innovations that keep the organisation contingent with its environment are not implemented. This would lead to a worse performance on the long term. Therefore, this thesis tends to find out in what way an emphasis on stability by public managers to cope with environmental dynamism influences organisational performance on the long term. The hypotheses are:

H0: Emphasising stability to cope with environmental dynamism has a significant positive influence on organisational performance.

H1: Emphasising stability to cope with environmental dynamism does not positively influence organisational performance on the long term.

Method

Research design

What is the best way to test these hypotheses? Because the research is about the long term, doing an experiment is not possible for this thesis. Although it would provide a very clear

(13)

insight into the influence of an emphasis on stability in reaction to environmental dynamism on organisational performance, this option has to be ruled out due to practical limitations. Therefore, this thesis will use another approach, namely the across-case comparison. This method compares cases that are comparable, but have differences related to the area of the hypotheses. This allow the researcher to test the hypotheses without conduction an

experiment.

For a deductive explanatory research like this, it is best to have many cases with many observations to detect differences over time. As previous research made clear, on the short term an emphasis on stability as a reaction to environmental dynamism has a positive influence on organisational performance. If this is the case on the long term only becomes visible after time has passed. By using many cases, small differences become easier visible than with only a few cases. Therefore, by using a large-n design, these small differences that emerge over time can be detected. Having many observations is important for the validity of the research. Via the observations the development over time can be researched. If there only is a small number of observations it is not clearly visible how the performance developed over time. Therefore, with a too small number of observations, conclusions may not be valid.

Data collection

For a research design like this, a large quantitative dataset with comparable cases over a longer period of time is needed. An excellent dataset for this is the database of the Texas Education Agency on Texas school districts. The Texas Education Agency oversees primary and secondary public education in Texas (Texas Education Agency A, n.d.). The agency has a database with data from more than 1200 school districts over a period of 16 years. Each district is overseen by a school board that is elected by the citizens of the community (Texas Education Agency B, n.d.). The board designs the general policies, budgets and hires the superintendent, who is a trained professional. The richness of the data in this database is quite unique, in the Netherlands for example, a database like this does not exist.

This is a perfect setting to test the influence of an emphasis on stability by public managers as reaction to environmental dynamism on organisational performance. Schools themselves are public organisations, but they are on top of that comparable to other public organisations with few hierarchical layers and a decentralised modus operandi. The educational staff are highly educated professionals, but are, like many other public organisations, bound to rules and regulations. The superintended is like many public

(14)

managers limited by budgets and general policies decided by an elected body, in this case the education board. But one of the most valuable properties of the Texas Education Agency data is provides much data over a long period of time. The database has already proven itself as a reliable database. As already discussed in the theory section, Meier and O’Toole (2012) and Boyne and Meier (2009) use it for their research, but it has been used by other scholars as well, for example John Bohte (2004) and Linda Skrla, James Scheurich, Joseph Johnson and James Koschoreck (2001). Another advantage is that, because it is the same database as Boyne and Meier (2009) used, the findings are easier comparable with their study.

Unit of observation

The Texas Education Agency provides data on four levels: at state level, alt region level, at district level and at campus level (Texas Education Agency C, n.d.). Within the state there are 20 regions, a bit more than 1200 school districts and around 8700 campuses (Texas Education Agency C and D). This research will focus on the district level. This choice has been made for three reasons. First, much data on campus level, which would provide most cases, is relatively incomplete. Second, districts are better comparable cases because one superintendent makes decisions for all the campuses in a district. Therefore, choices regarding stability can be compared clearer. Of course, heads of campuses have some influence on the execution of these decisions, but the position of the superintendent as “manager” is in this case more interesting. Third, Boyne and Meier (2009) do their research as well on district level. Therefore, by using the district level, the results are better comparable.

Operationalisation

To test the hypothesis, three variables have to be measured: environmental dynamism,

stability and the depended variable: organisational performance. For the operationalisation the mostly the same variables are used as in the study by Boyne and Meier (2009), but over a timespan that is twice as large as Boyne and Meier’s (2009). The same operationalisation is used for two reasons. First it has proven itself to be a good way to measure these variables. The article by Boyne and Meier is today still used by many other scholars like Groeneveld, Bakker and Schmidt (2019) and Meah (2019). If Boyne and Meier (2009) would have used a bad way to measure their concepts, the findings would not be referenced to today still. Second, by using similar measure, the findings are again easier to compare.

(15)

Environmental dynamism

How can environmental dynamism be measured best? Following Boyne and Meier (2009), tow clear changes coming from the environment are changes in budget and changes in the number and backgrounds of students. Schools cannot decide themselves how much money they receive and what students apply for the schools. Different students with different backgrounds require a different form of attention, which takes energy from the schools. As well a sudden increase or decrease in the number of students can impact the performance. Both are unpredictable changes that influence organisational performance. They are a sudden change in the stability or instability of the environment. Therefore, these are good indicators for measuring environmental dynamism. These indicators can be measured by taking the data on the number of students enrolled, the percentage of Black, Hispanic and economically disadvantaged students and the total revenue. In order to create a dynamism measure, the approach of Boyne and Meier (2009) is used. A same approach is used by Rutherford and Van der Voet (2019) to measure changes in the environment. The variables are first logged and then lagged. These logged numbers are than regressed with their lagged counterparts. The residuals of this regression become a percentage of changes, with which dynamism can be measured. The higher the number, the more dynamic the environment.

Stability

As already mentioned in the theory part, personnel stability can be divided into two

categories: Vertical and horizontal stability. Vertical stability can be measured by looking at changes in the percentage of employees in different professional categories within school districts. The categories are teachers, central administration staff, support staff, school-level administration and auxiliary personnel. An advantage of these measures is that they, beside personnel stability, also reflect the structural stability. If the percentages of these categories change, not only the personnel stability changes, but as well the hierarchical, or structural stability changes. Horizontal stability, as mentioned in the theory part, is stability of

specialisations within these vertical groups. Horizontal stability can be measured by looking at changes in the percentage of teachers hired for a specific group and subject. These are the percentage of teachers for regular education, compensatory education, special education, bilingual education, career and technology and gifted and talented education.

Following Boyne and Meier (2009) an Euclidean distance measure for both stability measures is created to compare how similar one year’s stability is to its previous year. This is

(16)

lagged percentages form their original percentages. The third step is squaring these outcomes. The fourth step is to take the square root of the sum of these numbers and subtracting this root from hundred. In this way a perfectly stable organisation has a score of hundred and a non-stable organisation a score of 0.

There is a problem with some of the measures. According to the Texas Education Agency’s glossary, the measures are percentages of employees within these categories (TEA E, n.d.). However, for some years the percentages are much higher than 100 for some

categories. It is very unlikely that the percentage of auxiliary staff in 2003 is 23,3 in one district and in the same district in 2004 221,5 percent. It looks like the comma is placed one step to the right too much. If the comma is moved one place to the left, these numbers are more similar to the previous year’s and do not exceed 100, which is impossible for

percentages of employees. Therefore, in the years where the percentages of employees exceed hundred, these variables are divided by ten. The numbers themselves are not changed, only the comma is moved.

Dependent variable performance

How can performance be measured? As already mentioned in the theory chapter, measuring performance of public organisations is not easy. Boyne (2002) gave five conceptual

categories for measuring performance: output, efficiency, service outcomes, responsiveness and democratic outcomes. However, measuring all these categories is very difficult.

Measuring efficiency as a measure of performance is difficult, because it is not directly visible wat one student costs. As well, it is questionable whether making students cost less is in line with the democratic outcomes and responsiveness. Responsiveness is about the fit with the environment, which is as well difficult to measure. Service outcomes, the achievement of formal objectives is in this case related to the output because the objective of schools,

producing students with a certain amount of knowledge, is a form of output. Therefore, in this case, performance can be measured best by looking at two variables as dependent variable for measuring output performance: the percentage of students that scores a pass on the

TAKS/STAAR test, and the percentage that passes the SAT/ACT test. The TAKS, Texas Assessment of Knowledge and Skills, test tests the reading, writing, mathematics and social studies skills of all students in Texas. The test was replaced starting in 2012 by the STAAR. The STAAR “is designed to measure to what extent a student has learned, understood, and is able to apply the concepts and skills expected at each grade level” (Texas Education Agency F, n.d., p. 20). This is a clear form of output performance. The SAT and ACT equivalent are

(17)

given to students who want to go to college, and the tests are required by almost all United States institutions of higher education (Boyne and Meier, 2009). Both the TAKS/STAAR and SAT/ACT pass percentages are good measures for measuring output performance.

Control variables

In order to reduce the chance of causal inference, it is important to control for other factors that can influence the TAKS/STAAR and SAT/ACT pass percentages. As Boyne and Meier (2009) show, the complexity of the task and the resources that are available theoretically and empirically influence performance.

Resources can be both financial and human. Experienced teachers are likely to teach better and therefore achieve better performance for the school. Teacher experience can be measured via the teacher experience data form the database. Non-certified teachers, on the other hand, are likely to achieve worse performance due to an education deficit. Boyne and Meier (2009) control for class size as well. Because in the later data this variable is only available per grade and no average for schools is available, a similar variable is used: the teacher student ratio. The idea behind class size as control variable is that students in small classes get more attention and therefore perform better. The same, however, applies to the teacher student ratio. If there are more teachers, the students receive more attention and are therefore likely to perform better. Money is likely to play a role as well. Teachers who

perform better, can ask for a higher salary. It is likely, therefore, that teacher salary positively influences performance. This can be controlled for by including average teacher salary into the regression. The financial aid from the state is as well likely to influence the performance. With more money schools can hire more and better teachers.

The complexity of the task of schools is connected with the background of the students. Homogeneous student populations with upper -or middle-class backgrounds are likely to perform well in school, independent of the school they go to (Burtless, 1996). Therefore, student backgrounds should be included as control variables. This can be done by including the percentage of Black, Hispanic and economically disadvantaged students.

Validity

Does this research design measure what it is intended for? By using a quantitative approach with a large dataset with data on 1200 school districts over a period of 16 years, differences in the performance over time can be detected. The biggest danger in using this approach is, that

(18)

there is a colliding or confounding variable that is not accounted for. The chance that this occurs can never be ruled out completely, but by accounting for the complexity of the task and the resources a good attempt is made. This is strengthened by the similarity with the design of Boyne and Meier (2009). If Boyne and Meier (2009) would have used a bad design, it is unlikely that their findings would still be referenced to today as already mentioned

before.

Analysis strategy

What to do with all these data? An appropriate way to analyse this data is doing a longitudinal multilevel analysis. With multilevel analysis, variance between cases can be observed at different levels (Hox, 2015). In this case variance between districts, and variance over time. This is important because it is likely that the observations are not completely independent. It is likely that the performance of districts over time is stronger correlated than the relation between performance of districts. Standard statistical tests usually rely heavily on the

assumption that the observations are independent and therefore produce results with standard errors that are much too small (Hox, 2015). Therefore, by using a multilevel approach, this problem can be avoided. That a multilevel approach is a good way to test developments over time becomes clear when looking at research design from other studies that use data over a longer time period, for example Van den Bekerom, Torenvlied and Akkerman (2016).

Results

In this section of the thesis the results of the regression analyses will be presented and discussed.

Should a multi-level approach be used?

The first regression to be done is the regression between time and the TAKS/STAAR and SAT/ATC pass percentages. These results in table 1 show that on average both the percentage that passes the SAT/ACT and TAKS/STAAR increases per year. For the SAT/ACT the percentages increase with 0,22% per year on average and for the TAKS/STAAR with 0,96%. This increase is visible when a scatterplot with regression line is made, as has been done in graph 1 below.

(19)

Table 1. Multilevel Regression Analysis of SAT/ACT and TAKS/STAAR pass percentages SAT/ACT TAKS/STAAR Occasion 0.218***(0.013) 0.956*** (0.011) Constant 17.18*** (0.370) 60.52*** (0.397) Variance District 131.6*** (5.935) 184.2*** (7.627) Variance Residual 49.10*** (0.599) 40.60*** (0.480) Observations 14573 15621 Observations District 13461 14351 Observations Year 1112 1270 Deviance 101948.768 107174.564

Standard errors in parentheses

* p < 0.05, ** p < 0.01, *** p < 0.001

Graph 1. Descriptive statistics of TAKS/STAAR and SAT/ACT pass percentages over time.

But more important, with this regression the intraclass correlation coefficient can be calculated. The intraclass correlation coefficient “indicates the proportion of the variance explained by the grouping structure of the population” (Hox, 2015, p.15). In other words, the intraclass correlation coefficient shows how much of the variance in the data occurs, in this case, between districts. It is likely that there is more difference in performance between

(20)

districts than there is between the performance of a district over time. The intraclass

correlation coefficient quantifies this. This is important because it determines what approach should be used. If the ICC (intraclass correlation coefficient) is 0, it means that all the variation in the data is due to variance between districts and not years. If the ICC is 1, it means that all the variation is due to difference between years and not districts. If the ICC is 0 or 1 it means that there is no use for a multilevel model. So, what is the ICC for the

TAKS/STAAR and SAT/ATC success rates?

Table 2. Intraclass correlation of SAT/ACT and TAKS/STAAR pass percentages and

districts.

SAT/ACT TAKS/STAAR

District 0.728 0.819

Table 2 shows that for the SAT/ACT about 73% of the variance occurs on district level and for the TAKS/STAAR about 82%. This means that 27% respectively 18% of the variance is due to differences between years. Therefore, a multilevel approach is very suitable for regressions with this data.

Environmental dynamism and performance

The second regression to be done is the regression between environmental dynamism and performance. Table 3 shows the results of this regression. It is surprising to see, that environmental dynamism has a statistically significant positive influence on the

TAKS/STAAR and SAT/ACT success percentages. For every 0,195 increase in dynamism for the SAT/ACT and 0,166 for the TAKS/STAAR the percentages increase by 1.

However, when the interaction between dynamism and time is taken into account, the coefficient of dynamism changes. For every 0,097 increase in dynamism, the SAT/ACT is increased by 1 and for every 0,032 decrease in dynamism, the TAKS/STAAR increases by 1. Both influences are statistically significant. So, the dynamism is statistically significant positively related to the SAT/ACT pass percentages and statistically significant negatively related to the TAKS/STAAR pass percentages over the time period of 16 years.

(21)

Table 3. Multilevel Regression Analysis of SAT/ACT and TAKS/STAAR pass

percentages: influence of dynamism.

SAT/ACT TAKS/STAAR SAT/ACT TAKS/STAAR

Occasion 0.320*** 0.701*** 0.324*** 0.700*** (0.024) (0.020) (0.024) (0.020) Black percentage -0.047** -0.308*** -0.052** -0.307*** (0.016) (0.013) (0.016) (0.013) Economically disadvantaged -0.200*** -0.0424*** -0.206*** -0.041*** (0.011) (0.008) (0.011) (0.008) Hispanic percentage -0.098*** -0.162*** -0.098*** -0.161*** (0.010) (0.010) (0.010) (0.010) Teachers without Degree 0.001 -0.090*** -0.002 -0.089*** (0.023) (0.013) (0.023) (0.013) Teacher Salary 0.000119*** 0.000462*** 0.000107*** 0.000465*** (0.0000262) (0.0000207) (0.0000262) (0.0000207) Teacher Experience 0.305*** 0.046 0.317*** 0.043 (0.042) (0.033) (0.042) (0.033) Teacher Student Ratio 0.212*** -0.311*** 0.195*** -0.306*** (0.049) (0.033) (0.049) (0.033) State Percentage -0.032*** 0.012** -0.031*** 0.012** (0.005) (0.005) (0.005) (0.005) Dynamism 0.195*** 0.166*** 0.016 0.223*** (0.059) (0.047) (0.069) (0.054) Occasion # Dynamism 0.097*** -0.032* (0.019) (0.016) Constant 22.35*** 58.37*** 23.24*** 58.10*** (1.289) (1.036) (1.297) (1.044) Variance District 59.90*** 102.7*** 58.47*** 103.1*** (3.186) (4.537) (3.118) (4.557) Variance Residual 48.94*** 38.55*** 48.93*** 38.53*** (0.604) (0.458) (0.604) (0.458) Observations 14573 15621 14573 15621 Observations District 13461 14351 13461 14351 Observations Year 1112 1270 1112 1270 Deviance 101081.45 105713.396 101054.776 105709.19 Likelihood-ratio test 867.32 1461.15 893.99 1465.36

(22)

Stability and performance

The third regression to be done is the regression between stability and performance. Table 4 below shows the results of this regression.

percentages: influence of stability.

Black percentage -0.079*** -0.340*** -0.079*** -0.343*** (0.016) (0.014) (0.016) (0.014) Economically disadvantaged -0.171*** 0.0000377 -0.171*** 0.000929 (0.010) (0.00846) (0.010) (0.00848) Hispanic percentage -0.097*** -0.110*** -0.096*** -0.110*** (0.010) (0.010) (0.010) (0.010) Teachers without Degree 0.016 -0.061*** 0.015 -0.069*** (0.023) (0.014) (0.023) (0.014) Teacher Salary 0.000394*** 0.000997*** 0.000393*** 0.00100*** (0.0000166) (0.0000141) (0.0000167) (0.0000141) Teacher Experience 0.196*** -0.129*** 0.198*** -0.128*** (0.041) (0.034) (0.041) (0.034) Teacher Student Ratio 0.222*** -0.279*** 0.219*** -0.295*** (0.049) (0.034) (0.049) (0.034) State Percentage -0.034*** 0.007 -0.034*** 0.008 (0.005) (0.005) (0.005) (0.005) Vertical Stability 0.015 0.107*** (0.012) (0.009) Horizontal Stability 0.019 0.072*** (0.013) (0.009) Constant 11.70*** 29.21*** 11.36*** 32.42*** (1.510) (1.191) (1.583) (1.167) Variance District 58.74*** 113.1*** 58.82*** 114.5*** (3.095) (5.162) (3.097) (5.218) Variance Residual 49.70*** 41.25*** 49.69*** 41.42*** (0.613) (0.492) (0.613) (0.494) Observations 14573 15621 14573 15621 Observations District 13460 14351 13460 14349 Observations Year 1112 1270 1112 1270

(23)

Table 4. Continued.

Deviance 101270.646 106804.424 101270.16 106879.327

Likelihood-ratio

test 678.12 370.12 678.61 295.17

* p < 0.05, ** p < 0.01, *** p < 0.001

It can be observed in the table that both horizontal and vertical stability have a statistically significant positive influence the TAKS/STAAR pass rates and a statistically insignificant positive influence on the SAT/ACT pass rates. However, when the interaction with time is included, different results appear as can be seen in table 5.

percentages: interaction between time and stability.

Black percentage -0.047** -0.293*** -0.042** -0.305*** (0.016) (0.013) (0.016) (0.013) Economically disadvantaged -0.198*** -0.047*** -0.197*** -0.041*** (0.011) (0.008) (0.011) (0.008) Hispanic percentage -0.097*** -0.155*** -0.099*** -0.159*** (0.010) (0.010) (0.010) (0.010) Teachers without Degree 0.002 -0.070*** -0.0008 -0.081*** (0.023) (0.013) (0.023) (0.013) Teacher Salary 0.000117*** 0.000428*** 0.000122*** 0.000450*** (0.0000263) (0.0000206) (0.0000262) (0.0000206) Teacher Experience 0.307*** 0.042 0.302*** 0.045 (0.042) (0.033) (0.042) (0.033) Teacher Student Ratio 0.217*** -0.283*** 0.200*** -0.295*** (0.049) (0.033) (0.049) (0.033) State Percentage -0.032*** 0.013** -0.032*** 0.013** (0.005) (0.005) (0.005) (0.005) Occasion 0.267 2.647*** -0.553* 2.350*** (0.201) (0.144) (0.252) (0.162) Vertical Stability 0.010 0.233*** (0.020) (0.013) Occasion # Vertical Stability 0.000574 -0.0206*** (0.00209) (0.0015)

(24)

Table 5. Continued. Horizontal Stability -0.055* 0.148*** (0.022) (0.013) Occasion # Horizontal Stability 0.00919*** -0.0175*** (0.00264) (0.00169) Constant 21.25*** 37.41*** 27.41*** 44.54*** (2.164) (1.544) (2.406) (1.574) Variance District 60.27*** 98.95*** 60.64*** 102.0*** (3.210) (4.383) (3.229) (4.508) Variance Residual 48.95*** 37.87*** 48.89*** 38.25*** (0.605) (0.450) (0.604) (0.454) Observations 14573 15621 14573 15621 Observations District 13460 14351 13460 14349 Observations Year 1112 1270 1112 1270 Deviance 10108.81 105409.56 101080.09 105591.582 Likelihood-ratio test 857.96 1764.99 868.68 1582.96

* p < 0.05, ** p < 0.01, *** p < 0.001

It is observable here that there is a very small statistically insignificant positive influence of vertical stability on the SAT/ACT passes when the interaction between vertical stability and time is included. For horizontal stability there is a statistically significant positive influence, but not a strong one. For every 0,00919 increase in horizontal stability the SAT/ACT pass percentages increase by 1. Both influences are not very strong when looking at the

coefficients. The TAKS/STAAR pass percentages are negatively influenced by both vertical and horizontal stability when the interaction with time is included. For every 0,0206 decrease in vertical stability and every 0,0157 decrease in horizontal stability the TAKS/STAAR pass percentages increase by 1. This means that the SAT/ACT pass percentages are positively influenced by horizontal stability, although not very strong and not by vertical stability. The TAKS/STAAR scores are negatively influenced by both horizontal and vertical stability and stronger than the SAT/ACT pass percentages are influenced.

But how does the interaction between stability and the SAT/ACT and TAKS/STAAR pass percentages develop over time? Is the coefficient for year 1 higher than the coefficient for year 16? To visualise this, the margins of the previous regression are stored and used to

(25)

create the contour plots in graphs in graph 2 and 3 below. Graph 2 shows that the influence of the interaction between vertical stability and time on the SAT/ACT and the TAKS/STAAR pass percentages becomes smaller as time increases. So, the coefficient decreases over time. This means that the negative influence of vertical stability on the TAKS/STAAR scores becomes stronger as time increases and the positive influence on the SAT/ACT decreases strongly over time. That this interpretation is correct becomes clear when the same regression is performed over a smaller amount of time. Table 6 shows the same regression for the SAT/ACT scores, but now only over a 12-year period. In this table the coefficient of the interaction is 0,00114, whereas the coefficient over the 16-year period is 0,000547.

Table 6. Multilevel Regression Analysis of SAT/ACT pass percentages: interaction

between vertical stability and time for 2003-2015.

SAT/ACT

Black percentage -0.068*** (0.016)

Economically disadvantaged -0.166*** (0.010)

Hispanic percentage -0.096*** (0.010)

Teachers without Degree -0.035 (0.021)

Teacher Salary 0.0002*** (0.00003)

Teacher Experience 0.242*** (0.040)

Teacher Student Ratio 0.198*** (0.047)

State Percentage -0.020*** (0.00484)

Occasion -0.077 (0.225)

Vertical Stability 0.031 (0.0167)

Occasion # Vertical Stability 0.00114 (0.00235)

Constant 17.68*** (2.008) Variance District 59.04*** (3.022) Variance Residual 32.27*** (0.446) Observations 11798 Observations District 10715 Observations Year 1083 Deviance 77658.012

(26)

Graph 2. Contour plots of the margins from the regression between the SAT/ACT and

(27)

Graph 3. Contour plots of the margins from the regression between the SAT/ACT and

(28)

Graph 3 is at first sight not clear for the SAT/ACT pass percentages, but after becomes readable after time. The influence of the interaction between horizontal stability and time on the SAT/ACT pass percentages decreases for the first four years (2003-2007). The top left corner seems to imply that the influence increases, but actually it is a decrease. The red school districts, which diminish in the left below, become dark green. That is why the amount of dark green increases. From occasion 7 (2009-2018) the influence increases again. Here the same initial confusion can appear, but the principle is the same. The dark green becomes smaller and the other colours bigger because there are fewer school districts in dark green. The changes, however, only happen to school districts with a high or a low score on stability. Schools that are between 50 and 60 do not really experience changes in the influence of the interaction over time. These findings, however, do not apply to the TAKS/STAAR scores. Here an almost linear development is visible. At first is the influence not very strong, but gradually it becomes stronger. This means that the influence of horizontal stability becomes increasingly negative on the TAKS/STAAR scores over time and for the SAT/ACT is

increasingly negative for the first four years but becomes increasingly positive after those first four years.

Environmental dynamism, stability and performance

The third regression to be done is the three-way interaction between dynamism, stability and the SAT/ACT and TAKS/STAAR scores. The results of this regression are listed in table 7 below.

percentages: interaction between dynamism and stability.

Black percentage -0.084*** -0.342*** -0.084*** -0.345*** (0.016) (0.014) (0.016) (0.014) Economically disadvantaged -0.173*** -0.002 -0.174*** -0.001 (0.010) (0.009) (0.010) (0.008) Hispanic percentage -0.097*** -0.110*** -0.097*** -0.110*** (0.010) (0.010) (0.010) (0.010) Teachers without Degree 0.016 -0.060*** 0.015 -0.068*** (0.023) (0.014) (0.023) (0.014) Teacher Salary 0.0004*** 0.001*** 0.0004*** 0.001***

(29)

Table 7. Continued. (0.00002) (0.00001) (0.00002) (0.00001) Teacher Experience 0.194*** -0.130*** 0.196*** -0.130*** (0.041) (0.034) (0.041) (0.034) Teacher Student Ratio 0.218*** -0.282*** 0.214*** -0.298*** (0.049) (0.034) (0.049) (0.034) State Percentage -0.034*** 0.007 -0.033*** 0.008 (0.005) (0.005) (0.005) (0.005) Dynamism -0.438 0.139 -0.700 1.082* (0.536) (0.380) (0.722) (0.465) Vertical Stability 0.013 0.106*** (0.012) (0.010) Dynamism # Vertical Stability 0.008 0.0005 (0.006) (0.004) Horizontal Stability 0.016 0.072*** (0.013) (0.009) Dynamism # Horizontal Stability 0.010 -0.011 (0.008) (0.005) Constant 12.26*** 29.64*** 11.98*** 32.75*** (1.529) (1.197) (1.597) (1.175) Variance District 58.17*** 112.8*** 58.18*** 114.2*** (3.067) (5.151) (3.066) (5.206) Variance Residual 49.68*** 41.22*** 49.68*** 41.38*** (0.613) (0.491) (0.613) (0.493) Observations 14573 15621 14573 15621 Observations District 13460 14351 13460 14349 Observations Year 1112 1270 1112 1270 Deviance 101257.152 106790.552 101256.81 106861.902 Likelihood-ratio test 691.62 383.99 691.96 312.64

* p < 0.05, ** p < 0.01, *** p < 0.001

This table shows that the interaction between dynamism and stability has a statistically

(30)

But how is this if the interaction with time is included in the regression? The results of this regression are visible in table 8.

percentages: interaction between dynamism, stability and time.

Black percentage -0.052** -0.294*** -0.052** -0.306*** (0.016) (0.013) (0.016) (0.013) Economically disadvantaged -0.206*** -0.047*** -0.205*** -0.041*** (0.011) (0.008) (0.011) (0.008) Hispanic percentage -0.098*** -0.155*** -0.098*** -0.158*** (0.010) (0.010) (0.010) (0.010) Teachers without Degree -0.0005 -0.070*** -0.003 -0.081*** (0.023) (0.013) (0.023) (0.013) Teacher Salary 0.0001*** 0.0004*** 0.0001*** 0.0005*** (0.00003) (0.00002) (0.00003) (0.00002) Teacher Experience 0.315*** 0.039 0.311*** 0.042 (0.042) (0.033) (0.042) (0.033) Teacher Student Ratio 0.198*** -0.280*** 0.180*** -0.295*** (0.049) (0.033) (0.049) (0.033) State Percentage -0.031*** 0.013** -0.031*** 0.013** (0.005) (0.005) (0.005) (0.005) Dynamism -0.469 0.307 0.339 0.841 (0.697) (0.469) (0.919) (0.578) Vertical Stability 0.010 0.230*** (0.019) (0.013) Dynamism # Vertical Stability 0.005 -0.002 (0.008) (0.005) Occasion 0.245 2.633*** -0.532* 2.301*** (0.207) (0.146) (0.258) (0.164) Dynamism # Occasion 0.357 -0.156 -0.065 -0.001 (0.293) (0.191) (0.324) (0.211) Vertical Stability # Occasion 0.0009 -0.020*** (0.002) (0.002)

(31)

Table 8. Continued. Dynamism # Vertical Stability # Occasion -0.00276 0.00144 (0.00311) (0.00205) Horizontal Stability -0.056* 0.145*** (0.022) (0.013) Dynamism # Horizontal Stability -0.003 -0.008 (0.011) (0.007) Horizontal Stability # Occasion 0.009*** -0.017*** (0.003) (0.002) Dynamism # Horizontal Stability # Occasion 0.00172 -0.000198 (0.00344) (0.00225) Constant 22.35*** 37.64*** 28.63*** 44.85*** (2.204) (1.554) (2.423) (1.586) Variance district 58.20*** 99.04*** 58.65*** 102.0*** (3.110) (4.392) (3.133) (4.514) Variance Residual 48.94*** 37.84*** 48.87*** 38.22*** (0.604) (0.450) (0.604) (0.454) Observations 14573 15621 14573 15621 Observations District 13460 14351 13460 14349 Observations Year 1112 1270 1112 1270 Deviance 101052.346 105401.566 101042.094 105578.998 Likelihood-ratio test 896.42 1772.98 906.67 1595.55

* p < 0.05, ** p < 0.01, *** p < 0.001

The influence of this three-way interaction on the SAT/ACT pass percentages is different form the influence on the TAKS/STAAR pass percentages. The TAKS/STAAR pass

percentages are negatively influenced by the interaction between dynamism, vertical stability and time. The SAT/ACT percentages are positively influenced by this interaction. The opposite applies to horizontal stability. The TAKS/STAAR percentages are positively influenced by the interaction with horizontal stability and the SAT/ACT percentages negatively. None of these influences, however, are statistically significant.

(32)

Discussion

How do these findings relate to Boyne and Meier’s (2009)? The first diversion arises already with the first measure, the influence of environmental dynamism on the SAT/ACT and TAKS/STAAR pass percentages. Boyne and Meier (2009) find that both the SAT/ACT and the TAKS/STAAR are negatively influenced by performance. The results here, however, showed a statistically significant positive influence on the SAT/ACT pass percentages over time. The influence on the TAKS/STAAR pass percentages is, in line with Boyne and Meier (2009) negative, but only in interaction with time.

The second findings differ as well from Boyne and Meier’s (2009). Boyne and Meier (2009) find a positive influence of vertical stability in interaction with dynamism on both the SAT/ACT and, in their case TAAS (the state test before the TAKS)/TAKS tests. The effect on the SAT/ACT, however, is not significant anymore after 3 years. According to Boyne and Meier (2009) this is because the ACT/SAT has a lower priority in comparison to the

TAAS/TAKS and therefore necessary measures are not taken soon enough and eventually overcome the positive effect of vertical stability (Boyne and Meier, 2009). The findings in this thesis, however, show a negative impact of vertical stability on the TAKS/STAAR tests over time. The influence on the SAT/ACT is insignificantly positive and, as the contour plot shows decreases every year. If this trend continuous, the SAT/ACT pass percentages will be negatively influenced by vertical stability in the future. The interaction with dynamism and vertical stability was negative for the TAKS/STAAR and positive for the SAT/ACT, but not statistically significant.

Boyne and Meier (2009) find a positive influence of horizontal stability as well in interaction with environmental dynamism on the TAAS/TAKS and SAT/ACT tests. The findings in this thesis showed a statistically positive impact of horizontal stability on the SAT/ACT tests and a negative impact on the TAKS/STAAR scores. The positive impact on the SAT/ACT scores is only statistically significant in interaction with time. The negative on the TAKS/STAAR scores increases in a linear line like way over time for the TAKS/STAAR scores. For the SAT/ACT the influence is negative for the first four years and positive after those years. The development of the impact on the SAT/ACT is more similar to a parabola and not to a linear line because of this. It would be very interesting to know how this impact is going to develop in the future. It can continue like a positive linear graph, but it, if the patter from these findings repeats itself it in the future, the influence becomes neither a linear line

(33)

nor a parabola but more like a sine wave, but that is something for future research. The interaction between environmental dynamism and horizontal stability was, like vertical stability, negative for the TAKS/STAAR and positive for the SAT/ACT, but neither are statistically significant. The same applies to the tree-way interaction with dynamism. So, where Boyne and Meier (2009) found a positive influence of the interaction between environmental dynamism and stability, no influence was found here.

Conclusion

What is the effect of an emphasis on stability by public managers to cope with environmental dynamism on the long term? According to previous research, environmental dynamism negatively influences organisational performance. Several studies concluded this based on empirical findings. A public manager, faced with environmental dynamism, has two options: try to adapt the organisation to the changes in the environment or emphasise stability to maintain performance. Most studies support the idea that emphasising the stability is the best way for a public manager to respond to environmental dynamism. This is in contrast with contingency theory, which suggest that an organisation has to innovate when the environment changes to keep the fit with the environment.

To test to what extent an emphasis on stability to cope with environmental dynamism influences organisational performance on the long term, this thesis set up a longitudinal multi-level large n-study with data form the Texas Education Agency over a period of 16 years. Surprisingly, three indicators reported a positive relationship between environmental dynamism and organisational performance, one as well in interaction with time. The fourth indicator, however, became negative over time. As for stability, one performance indicator showed a statistically significant negative influence of stability in interaction with time. The other performance indicator is, based on its development, likely to become negative in future times as well for one performance indicators. So, one performance indicator showed a positive relationship between environmental dynamism and organisational performance over time, one vertical stability is negatively related to organisational performance over time and the other is likely to become as well in the future. For horizontal stability one performance indicator showed a positive relationship with organisational performance over time and the other a negative. No interactions between stability and environmental dynamism had a statistically significant influence on organisational performance.

This means that the H0 hypothesis, that emphasising stability to cope with

(34)

can be rejected for the long term, based on this study and the H1 hypothesis, that emphasising stability to cope with environmental dynamism positively influences organisational

performance on the long term, can be accepted. An emphasis on stability by public managers to cope with environmental dynamism has no statistically significant effect on organisational performance on the long term. Moreover, vertical stability seems to have a negative influence on performance over time. The influence of horizontal stability is not clear.

Theoretical implications

These conclusions imply that a small alteration is needed to the Meier and O’Toole (1999) model of managerial impact on organisational performance. Stability should not be excluded from the model, because when the organisation is stable it performs as was shown by previous research, rather a factor related to innovation, should be added to the model. If there is no innovation but a lot of stability this will likely have a positive influence on the short term because previous research showed a positive relationship on the short term, whereas innovation probably has a negative influence on performance on the short term because it changes the stability. On the long term, however this is the opposite way. So, there is a negative interaction between stability and innovation. If stability is high, innovation is low. This is positive for the outcome on the short term, but negative on the long term. When innovation is high, stability is low. This is negative for the outcome on the short term, but positive on the long term. Therefore, both factors should be included in the model in interaction with each other. The model then could look like this:

Ot=ß1((I-S)+M1)Ot-1+ß2(Xt/(I-S))(M3/M4)+ εt Where,

O is some measure of outcome, S is a measure of stability, I is a measure of innovation

M denotes management, which can be divided into three parts

M1 management’s contribution to organizational stability through additions to hierarchy/structure as well as regular operations or management’s contribution to innovation,

M3 management’s efforts to exploit the environment of the organisation, M4 management’s effort to buffer the unit from environmental shocks, X is a vector of environmental forces,

(35)

ε is an error term,

the other subscripts denote time periods, and

β1 and β2 are estimable parameters (at times M3/M4 is defined as external management or M2).

This is, however, just a theoretical proposition which should be tested empirically.

Practical implications

In practice the findings of this thesis imply that, based on these findings, public managers, when faced with environmental dynamism, should consider whether they value performance on the long term or on the short term. Accordingly, they should either emphasis stability or innovation.

Limitations

Of course, this study is not without its limitations. It is almost unique that data over a 16 years period can be used, but another research over an even wider timespan should be conducted when possible to compare the findings. Because, it cannot be predicted with a 100% certainty how the data develops. Maybe the data does not develop linearly, like most measures showed, but like a sin wave over a very long time period. These 16 years could accidently just be a period of growth of the sine wave. Therefore, although 16 years is a long time period within public administration, even a longer timespan would be very interesting for future research.

But there are more limitations. These have to do with geographical location,

organisation type and limitations in performance measurement. This study was conducted by using data from public school districts in Texas and therefore, these findings apply to public school districts in Texas. These school districts are an example of a public organisation, but maybe other public institutions are differently influenced by environmental dynamism and stability over time. In order to draw more general conclusions similar researches should be conducted in different countries and on different public organisations to see if similar results arise. Another limitation is that this study focuses mainly on output performance. A topic for future research would be how stability and environmental dynamism influence other

To be stable or not to be stable, that is the question!

Environmental dynamism, stability and organisational performance. The long term.

Master Thesis

BA(hons) Daniël Reinders

25-6-2020

Wordcount: 10580

Supervisor: Dr. P.E.A. van den Bekerom

Second reader: Dr. J. Schalk

Leiden University Public Administration

To be stable or not to be stable,

that is the question!

Table of contents

Introduction

Theoretical framework

Method

Results

Conclusion