Enriching knowledge in the Dutch education sector through
XBRL: An information consumers' perception
An evaluation of current states in the Dutch education sector Submitted In Partial Fulfillment For The Degree Of Master In Science
Zehra Abbas 11947470
MASTER INFORMATION STUDIES Information Systems FACULTY OF SCIENCE UNIVERSITY OF AMSTERDAM
5 July 2019
1st examiner: 2nd examiner:
IR. Otte-Pieter Banga Faculty of Science, University of Amsterdam
Dr. A.W. Abcouwer
Faculty of Science, University of Amsterdam
KEYWORDS: Data-driven Decision Making, Data Analytics, Curiosity, Customer Value Proposition, Resilience, XBRL, public-private relations.
ABSTRACT
This study examines the perception of school councils, the Inspectorate of education, local councils and educational specialists, referred as information consumers towards analysis of financial reporting data in the context of knowledge forming and decision making. Outcomes suggest that only the Inspectorate currently perceives the data as useful for risk detection and although there is curiosity from some other consumers, the lack of data analytics skills and the lack of context in the data itself holds back further development. Therefore, additional steps will need to be taken in order to transform XBRL data into a valuable information source. 1. INTRODUCTION
While Investing financial assets in education improves the quality of education significantly, solely pumping money in education is not the answer to ensure the quality of education (Baker, 2017). The Government of education in the Netherlands, DUO, provides school councils a lumpsum budget in which councils are free to decide how to spend their budget. It is expected that institutions and councils deliberately decide how to spend these public assets. Scientific researches can help evaluate the relation between investments in educational settings (e.g. digital blackboards and E-learning methods), with classroom performances (Mathis, W. J.,2017). While such researches can help institutions with attaining valuable investments, these are limitedly conducted and there is no one size that fits all, leaving school councils to make decisions based on their experiences and guts. School councils in the Netherlands annually generate financial statements in order to meet transparency requirements and provide public asset usage justification. Besides the financial statements, the councils are also mandated to provide a strategic justification report of their strategic decisions, governance and performance of the past year. This strategic justification report is free of format and contains a description of the councils' strategic
plan to tackle issues and ensure the pledged quality of education. This information is then checked by the Inspectorate of Education and if needed further inspected. The school councils are all mandated to provide their financial statements through a central XBRL based digital portal. This portal, costing approximately 5 million (DUO, 2017), has been implemented in 2015 and is used to gather data in a structured and standardized way. All the data is thereby automatically tagged with the internationally recognized taxonomy along with the taxonomy of some education-specific terms. Once the data is gathered, it is made available to the public and parties can request for specific reports. Analyzing data of the past performance of the organization from the financial statement can help analyze trends and correspondingly identify potential threats to troubleshoot. The data standard XBRL has been implemented in financial reporting in many countries including the Netherlands. Literature (Goyal, A et al., 2017, Lester, W. F., 2007); Jones, A., & Willis, M., 2003) claims that the XBRL format standardizes the definitions of the data which makes it easier to compare it with other organizations. From the literature, one major benefit of XBRL can be derived: As the data is tagged with metadata from the international taxonomy, the quality in terms of credibility has increased. Thus, for reporting and information retrieval, the format can conclusively bring value in form of benchmarking. However, to create value out of this data, users need to be aware of the possibilities and see the value proposition of such a database. In order to evaluate the value proposition of XBRL, an empirical qualitative research is conducted on the information need of several stakeholders, their awareness of possible information sources and the current steps they take for responsible strategic decision making and knowledge forming. First, a literature review is conducted to create more understanding about data driven knowledge forming and data driven decision making, user acceptance and traits exploratory behavior. With the findings from the literature, secondly, a strategy is shaped in order to evaluate the user value proposition of XBRL. Thirdly, the results of this research are discussed which will lead to the answer of the question:
"To what extent can XBRL based reporting facilitate the knowledge forming and decision-making process of different stakeholders in the Dutch education sector?".
Finally, we evaluate the results by discussing the perceived usefulness, perceived usability and curiosity of the stakeholders to look into financial data and the roles each participant is willing to take for innovating the current process of strategic planning and public asset usage justification.
2. RELATED WORK
As stated in the introduction, the value proposition of XBRL for information consumers is mostly the standardization and the possibility to conduct benchmarking. In order to find the factors that influence the value proposition of these gains for a user, this section will give insight in related literature. The related work section is divided into three sections. The first section describes how data analytics can benefit an organization when applied for decision making and what defines an organization that is data driven. The second section gives insight in the elements that are important to evaluate the user need and value proposition. The third section examines the triggers of innovation and measures of curiosity that play a vital role in one's interest to conduct financial data analytics according to theories on data driven decision making.
Perceived usefulness, ease of use and intentions
While XBRL may offer many advantages, in order to get insight in the value of information provision towards consumers, various theories from psychology and information science can be consulted. The theory of reasoned action of Fishbein and Ajzen (1975) is still a popular theory to rationalize intention and planned behavior. Their model, later extended with an additional factor in 1995 by Taylor and Todd, suggests three factors to explain behavior including; attitude of the environment, subjective norms and perceived behavioral control. The factor attitude suggests the expectations that the environment has from the individual; subjective norms represent the expectations and judgements the individual has towards the subject; and finally, perceived behavioral control defines the perceived ease in which the intended behavior can be performed. In order to understand the users' perspective, it is essential to identify what is important to them and what motivates them to use the system or product (Hendry, 2008). Davis (1985) proposed that system use is a response that can be explained or predicted by observing user motivation and the capabilities of the system. He thereby suggested that the users' motivation can be explained by the ease of use of the system and the perceived usefulness. Davis (1989) defined perceived ease of use as “the degree to which a user believes that using a particular aid would reduce or be free of effort”. He further defined perceived usefulness as “Perceived usefulness is defined as the prospective user's subjective probability that using a specific application system will increase his or her job performance within an organizational context”. Venkatesh (2003) reviewed this model and added that the perception of the user on the system also plays an important role and therefore reformed the factors to perceived ease of use, perceived usefulness and attitude towards the system. In particular, the “Perceived Usefulness” scale is measured using the following criteria: (1) work more quickly, (2) improve job performance, (3) increase productivity, (4) enhance effectiveness, (5) makes it easier to do my job, and (6) useful. The “Perceived Ease of Use” scale is composed of: (1) easy to learn, (2) controllable, (3) interaction is clear and understandable, (4) flexible to interact with, (5) easy to become skillful, and (6) easy to use. Venkatesh (2003) added performance expectancy to usefulness and effort expectancy to usability. Furthermore, literature from marketing suggests that value proposition can be evaluated by exploring the whole process a customer goes through including the pre-purchase, the purchase and the post-purchase. This value propositioning through experience will
according to many including Lemon and Verhoef (2016) give insight in the living context in which the customer operates. As the customer integrates the product into their life's, the fit of the product into the environment of the customer is just as important. Chandler and Lusch (2015) further confirm in the service delivery context that an overview of the customers' current process can help discover the value proposition.
Data driven decision-making
Ongoing research is being done on how data can generate value for organizations and advance the decision-making process. In scientific practice, the data to knowledge graph is often represented as a pyramid (Weggeman, 1998; Shao et al, 2017) representing that data feeds information which in turn facilitates knowledge and wisdom forming. McAfee and Brynjolfsson (2012) found that companies that characterized themselves as data driven, performed significantly better on financial and operational measures. Seddon et al (2017) propose the BASM model that presents how data analytics contributes to creating business value. Their research shows that an organization needs to have resources in terms of capable staffs, readily available data as well as analytical software in order to gain new insights from data on the short term. They furthermore suggest that the benefits from analytics on the long term depend on four factors including; analytic leadership, enterprise wide analytics orientation, well-chosen targets and the extent to which evidence-based decision-making is embedded in the DNA of the organization. Grossman (2018) introduces an Analytic Processes Maturity Model (APMM) in which the analytic maturity of an organization is put on a scale of five levels from the first level, the organization is able to build a report to level five in which the organization bases its action on analytics. The six criteria Grossman (2018) identifies are respectively; organization is building analytic models; is deploying analytic models; is managing and operating analytic infrastructure; is protecting analytic assets through appropriate policies and procedures; is operating an analytic governance structure; and is identifying analytic opportunities to make decisions, and allocate resources based upon an analytic strategy. It is thereby, like in Seddon's criteria, vital that the user has knowledge of data analytics or has conscripted data analysts to accomplish the task.
Innovation and curiosity
While public organizations can always recover from losses, they and are not inherently triggered to try new combinations. However, expectations from the environment can encourage to try new combinations (Borins, 2001). An organization can be considered as a complex system that has to deal with unpredictable situations and developments. The lifecycle phases of an organization can be placed in ‘the adaptive cycle of resilience’ of Abcouwer & Smit (2015). Their model consists of four phases: equilibrium, crisis, new combinations and entrepreneurship. In this cyclic development an organization is situated in an equilibrium, where the state of mind is certain. External factors and environmental developments can bring an organization out of this balance and confront the organization with new challenges in which the organization shifts to a crisis. A crisis state can be defined as a state in which the organization is forced to take action and manage the situation in order to keep its position since traditional procedures will not anymore serve as problem solvers. According to Abcouwer and Smit (2015), the next phase, new combinations, is a fundamental phase in which the organization needs to develop (a combination of) new solutions to adapt to the crisis. According to Abcouwer and Smit (2015) a combination of good management, innovation and new solutions is key in order to enable the organization to increase its resilience. Being adaptive to new situations, leads to the entrepreneurship phase, where fast growth and development predominate, allowing the organization to learn and standardize. Once the organization is settled with the changes, the phase of equilibrium is reached again, this time with freshly acquired
knowledge and skills. In order to remain resilient and successful, an organization has to be adaptive to future uncertainties. Organizational innovative ability is closely connected with creativity and can be defined as “the creation of a valuable, useful new product, service, idea, procedure, or process by individuals working together in a complex social system”. Thereby, Gino, F. (2018) indicates that curiosity is closely linked with innovation. There is an understanding that curiosity’s immediate function is to seek out, explore, and immerse oneself in situations with potential for new information and/or experiences. Kashdan et al. (2018) argue that curiosity, has two components: stretching and embracing. Stretching relates to actively exploring new information or experiences, maintaining concentration, and regulating attention toward an interest or goal. Embracing stands for the willingness to be engaged in novel, uncertain, or unpredictable experiences, some of which could be considered unpleasant (Kashdan & Silvia, 2009; Kashdan et al., 2009). Later, Kashdan et al (2018) expanded their curiosity scale and distinguished the following dimensions of curiosity: (1) Joyous Exploration – in which there is a desire to seek out new knowledge and finding delight in learning and growing. (2) Deprivation sensitivity, indicating a feeling in which not having the information can be frustrating or embarrassing. (3) Stress Tolerance; this measure is similar to the previously described embracing indicating the willingness to embrace confusion and face obscure events. (4) Social curiosity is a measure indicating the urge of wanting to know about other people thoughts and whereabouts. Lastly, (5) Thrill Seeking is a measure which indicates the willingness to take physical, social, and financial risks in order to acquire new experiences. Kashdan et al (2018) thereby distinguish the following types of people; The fascinated, that has a high trait of joyous exploration; the problem solver, that has high traits in deprivation sensitivity; the empathizer, that has high social curiosity traits. Finally, there are also avoiders that score low on all dimensions and do not want to embrace insecure situations.
Concluding the theoretical framework, it is firstly found that for new technological systems or in our case, newly available structured XBRL data, it is important that the technology adds value in form of usefulness and is perceived to be easy to use. Thereby the behavior of the individual is influenced by the attitude of the environment, subjective norms and perceived behavioral control. Secondly Seddon et al (2018) and Grossman (2018) deliver criteria to evaluate the data driven decision-making ability of an organization as McAfee and Brynjolfsson (2012) suggest data driven decision-making has significant positive benefits. Thirdly, triggers have been identified for innovation to proceed. From the works of Abcouwer and Smit (2015) it is instituted that organizations are in a constant cycle of trying new combinations to be resilient. Innovative thinking is thereby critical and driven by creativity and curiosity specified as a matter of the embracement and stretching abilities.
3. METHODOLOGY
XBRL offers many possibilities for the transmitter and analyst of the data, in terms of increase in findability and validity. Nevertheless, the information consumer is not defined specifically in the literature as this can differ per case. Overall, for the information consumer, literature claims that XBRL can bring value in the form of standardization which enables benchmarking and quality assurance which increases the data validity. As discussed earlier, DUO has implemented the XBRL portal to gather financial justification data from school councils. The gathered data can be of interest for several parties with varying information needs. The following consumers are considered by DUO as potential information consumers:
• The inspectorate; examine the justified usage of public assets critically by assessing the financial reports. This group is responsible to take action if the numbers in report are below a certain threshold.
• Management and controllers of School councils; have to gather this data and report it through the portal to DUO in order to show transparency of justified usage of public assets. The main concern of School councils is the judgement of the Inspectorate on correct usage but potentially use this data to compare themselves with other councils or conduct predictive analysis.
• Local authorities; use the data to get an overview of the financial situation and educational offer of the educational institutions in their region and furthermore inspect goals and realizations of education in a broader sense.
• Education specialists; This is a group of trend watchers and sector analysts that provide overall sector insights that can be consulted by public and private stakeholders.
In order to evaluate how these parties potentially use this data and what their information need is, this empirical study examines factors that play a role according to the previously described literature. A qualitative research is conducted in which each potential consumer group has been interviewed. Qualitative researches are useful for developing in-depth understanding of a specific situation (Maxwell, 2008). As this research is conducted in the setting of the Dutch education sector, it can be considered a case study in which the added value for an information consumer sector of the many consumers of XBRL data are weighed. In a broader perspective, it will give insight in the way data analytics contributes to knowledge forming. Through semi-structured interviews the current situation of public asset justification in relation to data analysis is contextualized and an in-depth insight is developed. Semi-structured interviews are chosen as the paramount method to evaluate the users' perception on XBRL data as the interviewee is given room to add perception, while a structure is prepared beforehand. A disadvantage according to Irvine, et al. (2013) may be that the interviewer needs to be skilled. However, the interviewer has followed a course on interview guidance in which this concern is covered. To prepare the interviews, background information about the role the participants play in the
education sector is gathered from online sources and interviews with DUO employees. It is found that the inspectorate is the primary user of the information and they can have a say in what data is included in the financial reports. Thereby educational specialists are also partly substituted by the government. With this information in mind, the interviews will be prepared respectively.
From the literature review it has become clear that whether the user perceives anything as a knowledge base can depend on many factors including the perceived usefulness, intention of use and perceived usability as Venkatesh (2003) claimed. Furthermore, the curiosity of the user to innovate and take risks (Abcouwer, 2015; Kashdan, 2018) also play a role as dissatisfaction with the current situation urges innovation. With respect to the theoretical framework, a conceptual framework is generated as displayed in figure 1 in order to evaluate. In order to gather data about the perception of the user in terms of XBRL as knowledge base, three main factors are derived from the literature:
• Perceived Usefulness: As suggested by Chandler and Lusch (2015), the users' information need is evaluated by conducting a task analysis to give insight in how users satisfy their information need and what information is crucial to stay in business. The task analysis facilitated sufficient information to make high level flowcharts. The flowcharts give insight in how participants perceive their tasks and creates an understanding of why certain information sources are vital and preferred to proceed the different tasks. This task analysis is shown in figure 1 as process of decision making. Another factor in figure 1 is data driven behavior as suggested by Seddon (2018) and Grossman (2018) that will be evaluated partly by the user's decision-making process and additional questions about the extent in which data is embedded in decision making. Furthermore, as Taylor and Todd (1995) suggested that planned behavior can be explained by the expectations and responsibility that the person has influences behavior, these components are represented under 'attitude of environment' in figure 1. Lastly, Venkatesh (2003) evaluates usefulness through the following criteria; 1) work more quickly, (2) improve job performance, (3) increase productivity, (4) enhance effectiveness, (5) makes it easier to do my job, and (6) useful. As in our case it is expected that not every user group is currently using the data, these indicators will be evaluated by asking how he expects to benefit from data analytics in terms of better decision making shown in figure 1 as perceived benefits.
• Perceived Usability: Venkatesh (2003) identify perceived usefulness and perceived ease of use as two significant factors for a system to be accepted. The “Perceived Ease of Use” scale is composed of: (1) easy to learn, (2) controllable, (3) interaction is clear and understandable, (4) flexible to interact with, (5) easy to become skillful, and (6) easy to use. As in our case, there is no solid information system but rather publicly available raw data and a visualization, the ease of use will be evaluated by asking how both of these data types are perceived on the above-mentioned criteria. In order to capture the users' perspective on the ease of use, the users are first asked how often they consult the financial data, in both, raw and visualized format. To gather personal opinion on data visualization, understandability, ease to learn and flexibility are evaluated. Then the user is asked on the organizations' expertise regarding data analytics in raw and visual format. • Reasoned action/ Curiosity: This factor evaluates whether the information consumer is curious about the information that is provided. As Kashdan (2018) claims, this depends on whether the user is willing to take risk and invest time and resources in something he does not know the outcome
of and whether the user likes to joyously explore and whether he thinks he is going to miss out something if he doesn’t act. When the user was not able to spontaneously provide a case in which past experiences were useful, he was provided an example to anticipate whether insight in data would be useful. While Kashdan (2018) also indicates the importance of evaluating the individual's curiosity, this was not measured as most interviews are conducted by phone and this method lacks visual cues (Novick, 2008). As the conceptual framework is built from factors from independent studies, the influences and dominancy between the factors is unmeasured. However as this is a qualitative study, the influence per factor is described separately, in which the reader is free to interpret the dominancy and influence between factors and considered for future work.
In order to increase the internal validity of the data, multiple participants per user group were approached. To assure that the participants were capable to deliver relevant insights, participants needed to be employed by the concerned organization and have a position relevant to public asset justification. DUO, my employer, provided a list of 200 (potential) information consumers that had been in contact with DUO regarding public asset usage justification in the last two years. After careful division by the mentioned criteria above, two employees from the Inspectorate of Education, 20 school councils, three local councils and five educational specialists were approached for an on-call or on-location interview. Two employees from the Inspectorate of Education have been contacted and interviewed. One participant is involved in model building while the other inspector has a background in finance. As both participants can give an insight on the view of the Inspectorate as an organization on financial data analysis and innovation these interviews are considered sufficient. From the 20 school councils that were contacted, seven responded positively. According to Guest et al (2006) the more unstructured an interview is, the more interviews are required in which the interviewer must iteratively analyze the answers and decide when sufficient information is collected. They further indicate that saturation of answers of a homogeneous group can occur after approximately twelve interviews. This means that the researcher can start to see patterns in the answers of homogeneous participants. As the interviews were structured and only the homogeneous group of policy-makers and controllers were interviewed, the iterative analysis process showed saturation after seven interviews. In this research, data saturation was used as a criterion to conclude data collection. Thereby, as generalization is not the point of this qualitative study, but rather finding the value proposition of XBRL data for school councils, the results are professed as relevant. In order to gain perspective of the information needs of the local authorities, five local authorities were contacted. One of them responded positively. However as one interview is not enough to relate the answers to all Dutch local authorities, the answers of the local council are not considered externally valid. Nevertheless, as this interview does give in-depth insight about the role of one local authority, it is still considered relevant for the results section. Finally, three educational specialists are interviewed from intermediate parties (an administration office, a representative of universities and a representative of practical colleges called MBO's) that play a vital role in information provision towards the public and educational institutions. As there are four main specialists and the rest of intermediate parties like administration offices have an undefined role, interviews with 2 main parties and one smaller intermediate party is interviewed, this group is of sufficient scope.
Transcripts have been made for all the interviews. The interviews are conducted in the Dutch language as all participants and the
interviewer are native speakers. The transcripts are thereby also in Dutch. However, before coding the transcripts, imperative sentences were recapitulated in English in which the recapitulated sentences were coded with subjects from the conceptual model. While the interview questions were formed per sub-component in the conceptual model, answers matching with multiple sub-components were given multiple codes. Through axial coding, the answers of the participants were matched with the factors detected in the conceptual framework. A detailed insight in the interview questions can be found in Appendix I. A snapshot of the resulting code structure can be found in appendix II. Now the used methodology is described, the results in the next chapter will depict how the information consumers perceived the usefulness, usability of the data along with answers indicating their curiosity. 4. RESULTS
As mentioned in the methodology, three main indicators have been defined to get insight in whether users perceive XBRL data as a knowledge base; usefulness, curiosity and usability. The result of each factor per consumer group is described below.
Usefulness
Overall all participants indicated to see limited value in analysis of the financial reports as this data is dated and contains no context such as the number of students, the location, vision or governance structure of the council. This context can mostly be found in the justification report as plain text.
Process of decision-making
From the interviews, the work processes for decision making of the participants was derived. School councils were asked how they agree on investing in a certain idea. Results show that councils rely on research in the news, notifications from their network, local councils, inspection, indications from parents, students and teachers and furthermore look into whatever is happening in their environment. Primarily the results of the students give an indication of whether pronounced changes are required. School councils indicated to create a 4 to 5 years strategic plan that settles the goals and ambitions. After having set goals, the ideas are communicated, and the budget is portioned per school. The school councils indicated that the lumpsum budget they receive is depended on the number of students. Therefore, a decent estimation of the expected number of students is important to proceed with the strategic ambitions. When this number is certain schools often finalize their budgets and spend this amount during the year. All school councils indicated to annually evaluate the proposed budget and the actual realization. The school councils indicated to be interested how this estimation can be made as close to reality as possible. A visualization of this work process can be found in appendix C.1.
The Inspectorate was asked how they analyze the data once the reports had been submitted. They revealed to have built a risk detection model which automatically detects the school councils that score below a threshold of specifically three indicators: rightful usage, efficacy and continuity. These indicators are further refined into formula's solvable with the data from the standardized financial report in XBRL. The Inspectorate manually extracts this data from the database of DUO and runs their model. The Inspectorate thereby stated to be mainly concentrated on the indicator continuity as the financial statements are dated reports in which the investments have are already been done. In order to ensure that the school council can continue its service. The accountant is accredited to be the first line of control to check for rightful usage and efficacy in which they report if suspicious activity is taking place. From the results of the risk detection model, school councils that score below the threshold are analyzed. It is possible that a typing error in the XBRL portal has caused the odd indication.
Therefore, the paper based financial report which is accredited by the accountant is leading. The Inspectorate indicated to currently use the XBRL data as it is easily accessible and analyzable as it is digital. However, they indicated to still need to rely on the paper-based report as this was accredited and more reliable. In order to fully take the potential of XBRL, it is required that this data is identical to the paper-based report or also accredited by the accountant. Going further in the process, if the odd numbers are not caused by a spelling error, the justification report along with the continuity report are studied to find whether the school has a plan to polish off the odd numbers before it forms into a hazardous risk for continuity. If needed, the Inspectorate visits the council for additional explanation. If after this visit there is still doubt on control over the continuity, the council is asked to submit an action plan. The council will thereby be under observation and if it has not recovered in the next year, it will again be detected by the risk detection model and the process is repeated. A visualization of this process can be found in appendix C.2.
Educational specialist can be better defined as processors of the data, rather than decisionmakers. However, a situated decision they make is what information is derived from the available data and what subjects are chosen for visualization to create awareness about. As previously stated, school councils additionally rely on reports from the specialist to form their strategic plans. Interviews with the educational specialist suggested different sources of ideas including input from school councils, political suggestions or ideas formed by analysis of the available data. Primarily specialist showed to depend on what data is available from several information sources including DUO's published data. Results show that the specialists visualize this data but primarily leave the benchmarking and further analysis for the councils to do. One specialist indicated to plan regular sessions with school councils to evaluate differences found in the data between councils in order to trigger conversations and get to new insights. The specialists thereby agreed that comparing strategic and financial decisions of councils that differ in size and operation area was counterproductive. While one specialist indicated to evaluate and assist councils how the visualizations are taken into consideration for decision making, the other two specialists indicated to evaluate the usage of the dashboards limitedly.
Lastly, one local council was interviewed, and their work process was derived. The local council stated to annually receive a budget from the government to support school councils in their region. In order to receive this subsidy, school councils need to submit a strategic plan and financial plan. Once submitted, these documents are reviewed, and the local council decides whether to accept or reject the proposal. If accepted, the subsidy is provided. The school council thereby has to provide a justification report about the specific subsidy that is different from the justification report that is provided to DUO. The flowchart of this process can be found in appendix C.3.
Data-driven behavior and knowledge of the user
In order to derive how data-driven the participants are, the criteria of Seddon et al (2018) and Grossman (2018) were used. The seven school councils overall showed to have limited knowledge of data analytics available in their organization. 5 out of 7 councils indicated to have dashboards containing the indicators of the Inspectorate for their own organization along with the student count. One council indicated to also model the expenses per student and teachers while the other council was looking into the relationship of the decrease of students on the costs of accommodation. However, these indicators were not always considered to be leading in the decision-making process. School councils clarified to need more context about for example the number of students as this was vital to predict the lumpsum budget they can expect to receive. School councils agreed that the data
from DUO is at least a year old and as the future brings its own challenges, it is limitedly useful to get insight in future risks and challenges. Thereby, practical experiences of past projects offer sufficient knowledge on what works and what doesn’t. Apart from the official justification report data, school councils expressed more positivity towards conducting analysis on real time financial data in which they could find whether they are still on track to keep the costs within the calculated budget. Furthermore, school councils indicated to foresee that a lot of non-financial and not-measurable indicators play a vital role in decision making and these aspects are often leading when forming the strategic plans. News articles and conversations with other councils, parents and teachers serve as a better information source. Therefore, school councils are considered to be on two out of six on the Analytic Processes Maturity Model of Grossman (2018) as they conduct some reporting but have an unclear vision on possibilities and limited data analytics knowledge assets available. Thereby they fulfill on one of the four criteria from the BASM model of Seddon (2018), which is overview enterprise wide analytics orientation.
The Inspectorate showed to facilitate a large part of their initial decision-making process through data analytics with the data driven risk detection model. As will be further explained under the subject curiosity, other than the mandated three indicators, the Inspectorate is not interested in exploring trends and relations in the data. The indicators are believed to set on safe measurements in which councils are still in a safe stadium. As the Inspectorate has a data analytics team for modeling, they are believed to have capable staff that manages the infrastructure and software, and furthermore act on the output of their analytical model. With this risk detection model, the Inspectorate shows to satisfy its mission and vision as the Inspectorate of Education. The Inspectorate is therefore considered to be on five out of six on the APMM model of Grossman (2018). As for the BASM model, the Inspectorate currently fulfills all criteria as the analytics brings substantial value to the business. Interviews with educational specialists indicated that they primarily process data that is published or gathered from school councils into dashboards. One of the interviews showed that the information on the dashboards are partially tailored per need, as the specialists also answer information requests. One of the specialists for example was planning to experiment with data analytics through advanced text analysis of the written parts of the justification report. This shows that the specialists are currently developing their analytic strategy and have developed their analytical infrastructure. While, two of the three specialists are limitedly looking into the practical use of the analytical models and therefore score low on deployment, one of the specialists discusses its visualization deliberately with the information consumer. As per strategy, the specialists are depended on whatever data is available and limitedly seek to find beyond what is already being published. On the APMM model, the specialists on average score five out of six. As for the BASM model, the business value can be defined as creating new insights with the data leading to decisions of school councils. As two out of three do not regularly evaluate this, they are fulfilling three out of four criteria.
The interview with the local council showed that the council was doing a reorganization of the information department, and specified that the sources that are currently consulted for research are cycled meetings with the school councils in which overall development is discussed, external warnings from government or in the news and the justification report of the subsidy that is submitted. From the interviews it was derived that this data is not analyzed statistically but rather checked per feeling. The local council added that for the financial plan that the council provides for requesting a subsidy is checked per fixed measures while the action plan is checked content wise. For the evaluation at the end of the year, the justification is mostly checked per proof of effort. As no traces of advanced data analytics were found for the
educational department of the local council, but measures for evaluating the feasibility of the financial plan, they score one out of six. The local council currently and as no business value is created from the analysis none of the criteria of the BASM model are met. Appendix V shows how the information consumers score per criteria of Seddon et al (2017) and Grossman (2018).
Attitude of environment
During the interviews the subjective norms, perceived expectations of the environment and perceived amount of control were derived. School councils admitted feeling responsible for justified usage of public assets and collect data in order to prove so. School councils agreed that external parties such as the government, Inspectorate or the media can help to make better decisions through information provision and research. The school councils thereby stated that they create separate reports for parents, students and employees as the justification report that is provided to the government contains a lavish amount of financial jargon.
Interviews with the Inspectorate showed that they do not feel responsible to guide the school councils towards strategic decisions as long as there are no problems as the councils need freedom in their expert field. The interviews further showed that the Inspectorate feels responsible to timely analyze data and provide feedback.
Interviews with educational specialists indicated that the specialists feel responsible to provide an enriched representation of the endeavors of their sector and organize the creative thinking process among school councils. Additionally, it was stated that it is the responsibility of the school council to utilize the insights from the visualizations for making sensible strategic decisions. The specialists realized that they serve as one of the information providers for the school councils but at the same time are not the only source of information.
The local council stated to feel limited responsibility to guide the school councils towards better strategic decisions as they felt to have no authority over the policies. The participant also stated that school councils do not like to much interference and are capable to make their own strategic choices. The role of the local council is thereby to provide additional subsidies if required.
Perceived benefits
Participants were asked how they believed to benefit from analytics of the data from the financial justification report. Interviews with school councils suggest that predicting financial situations by using historical data is not perceived useful as every year brings new challenges. However, the councils were positive about benchmarking the data with other councils in order to find differences and use this as a reason to initiate a dialog and through this dialog come to new insights. The data analysis hereby serves as an initial step to gather new insights. Nonetheless, school councils were particular with the type of councils they would look up as this needs to be sector specific and the compared councils need to be of approximately the same size.
When the Inspectorate was asked about the visualization of the data, they indicated to see no great value proposition for themselves as the proportion of costs and budgeting was already known and this basic information does not serve their needs but rather for secularly involved individuals.
The educational specialists showed to actively seek insight in their sector and showed to have trust in the value of benchmarking and learn from others' experience. While several sources of information are consulted, data from the justification report is believed to only be valuable for the accountant and the Inspectorate that can read through the financial jargon. For analytics on the financial expenses of the council, the specialists would all rather look into another form of the data. They further explained to see no value in using this report for any other kind of analytics such as benchmarking.
The local council believed to not benefit from XBRL as they did not consult the financial justification report.
Concluding the results of the factor usefulness, it is found that significant knowledge about finances and data analytics is required in order to make sense of the data. Currently only the Inspectorate and educational specialists possess this knowledge. While school councils are experimenting with dashboards, a lack of insight in the benefits causes no severe practices except for keeping track of the indicators the inspectorate magistrates on.
Curiosity
While evaluating the usefulness of XBRL through its value proposition in the user's activities, it was found that the willingness to experience and looking into substances- that only by chance can generate value - plays a vital role in whether one is engaging in this activity. Reflecting on what is found for attitude of the environment, a slight bewilderment on who initiates and invests in such initiatives is noticed.
Embracing
In order to evaluate how willing, the participants are to spend time and resources in the unknown, participants were asked how they perceived innovation versus risks, what they were currently innovating in and whether they were interested in investing in financial data analytics to potentially help mitigate risks. Answers indicated that school councils were hesitant to invest in the unvalidated. Councils overall preferred stabilized intuitive visualizations of data rather than investing in developing something from scratch. One council indicated to innovate by enabling digital examinations as it saw many opportunities (e.g. data collection teacher satisfaction) that outweighed the financial risks. This project was conducted in association with a well-established digital examination enterprise.
Answers of the Inspectorate of education show little interest in analyzing past results but rather in initiatives to find new ways to analyze incoming reports specifically highlighting the continuity section. The Inspectorate claimed to embrace innovation by finding efficiency to analyze the data earlier in the process and claimed to be interested in machine to machine initiatives in which data is derived directly from administration systems of councils. They further explained that this was one of the early objectives of XBRL that was hindered by cooperation-, standardization- and other complexities.
The educational specialists, as claimed earlier, see potential in benchmarking and data analytics, but not necessarily from the justification report. The specialists were willing to look into unexplored terrain when there is any suspicion on the field. Their initiatives to embrace innovation include dashboards that give insight into the sector and thereby invest time in finding (pivotal) relationships in the data that could possibly support constitutional accounts. Interviews with the specialists further derived the supposition that school councils are too focused on proving that they had spent their lumpsum correctly, that they were afraid to take risks and thus innovate.
The interview with the local council derived that for indications of financial stability, the local council fully leaned on indications from the Inspectorate in which the Inspectorate alarmed the local council if a school council was about to go bankrupt. They furthermore indicated that there was no trigger to look into financial states of a school council. The local council further explained that their justification report about the subsidy was less focused on financial returns but rather on trial and error.
Stretching
In order to find the range in which participants are willing to extend their horizons in terms of acquiring information about their surroundings, participants were asked how they explore information
that possibly leads towards their goals. Interviews with school councils indicated that councils have regular contact with other councils usually from their own region and sector. The meetings occur approx. every 4-8 weeks in which common problems are discussed. All councils also indicated to conduct some kind of benchmark. The councils indicated to be interested in non-financial indicators such as governance strategy, project management, redemption periods, and financial indicators such as costs per employee, Remuneration of the management board, accommodation pricings and annual savings. The goal of conducting benchmarks is thereby to find dissimilarities and use this result to initiate conversations. The councils thereby added that it is senseless to compare large-city schools with schools in smaller cities and villages as their challenges are not alike.
Interviews with the Inspectorate of education indicated that the accountant is trusted to be the first line of control in which after sign-off, only under severe conditions this verdict can be questioned. Thereby the Inspectorate stated to not be interested in conducting benchmarks nationally and internationally. The Inspectorate further explained that there is awareness that the Inspectorate could look into more links within the data, however correlations between certain elements within the financial statements need to be found before any benchmarking is going to be useful. In contrast, the second interview with Inspectorate in which was stated that benchmarking is not the inspections job. The Inspectorate however does claim to stretch its horizons when noticeable occurrences arise. Currently, the Inspectorate is looking into the diagnosed shrink of students as this affects the lumpsum and thus the financial situation of councils. The Inspectorate has created dashboards per region to detect anomalies. It was further explained that the inspiration for research come from exploration of the gathered data and dialogs with councils and specialists.
Interviews with educational specialists found that their main source of information is the published data from DUO, and data that the school councils willingly provide to the specialist. The specialists thereby try to stretch by organizing sessions in which the councils come together and discuss complications but also
The interview with the local council derived that there is a vision the local council stands for (equal opportunities e.g.) and thereby contentedly supports school councils that have ideas regarding this vision. The vision of the local council is thereby established from national intelligence and observations.
Satisfaction with current situation
In order to explore the triggers that lead to engaging in innovation, participants were asked how satisfied they were with the current situation. From the interviews it was derived that school councils were overall satisfied with the information provision. One council added that the organized networks kept them involved in current matters. When comparing answers of specifically two school councils that had identified themselves as either experienced or new, it was noticed that the school council early in their lifecycle and not fully developed in every area, was more open to experiment and trying out data analytics than the council that was stable and steady.
The Inspectorate also showed to be satisfied with the current situation and specified that the risk detection model provides sufficient insight while additional information is easily verifiable through the justification report while school councils are also eager to clear any doubts.
The educational specialists claimed to feel a need to innovate as they see many possibilities to further develop advanced data analytics. However, there is small insight in whether the initiatives serve for better decision making. The specialists indicated that it's up to the school council to actually convert the insights from the
data to ideas for decision making, however the specialists feel responsible to help councils get a realistic and meaningful prospect. The local council showed to be satisfied with the current situation in which school councils provide a separate report to justify the used subsidy. It was thereby mentioned that other local councils in large cities are most certainly more involved in how schools perform and provide guidelines, however the participant did not feel the need to implement this strategy to their local council as well.
Deprivation sensitivity
School councils were asked whether they were doing or were willing to do analysis in trepidation of falling behind or missing out on advances due to lack of information. The answers indicated that the school councils did mostly feel to have sufficient knowledge, experience and contact with other councils to guide them and avoid mistakes. For budgeting it is helpful to find factors that can form a risk in an early stage, in order to reduce the risk that the realization of an idea costs more than projected earlier. While the participants showed to have an interest in the way other councils mitigate the risks and wanted to know the best strategy for this, they were not sure whether data analytics was the answer. It is known through the network that other councils are also not advanced in financial data analytics as there is a common idea that this data only shows a small part of the bigger picture and not all risks are covered analyzing each other's historical data.
Interviews with the Inspectorate of education indicated that the Inspectorate feels responsible to know when a council is or is about to get in complications, when these councils do not show odd behavior, they are not noteworthy to the inspectors. The Inspectorate thereby wants to prevent that great irregularities go unnoticed. A few years ago, for example a college went bankrupt due to great expenses in accommodation that the school council and the Inspectorate had not foreseen. Such events need to be avoided by all costs. From the interview it was indicated that such an even could have been detected earlier with the available data, but it is easier to say so when the outcome is known. Realizing that certain parameters can help predict risks is useful and lessons are learned from the incident.
While there is no competition between education specialists as they operate in different sectors, one specialist claimed to be ahead of others as they supposedly provide an advanced dashboard and regularly initiate meetings between councils. The other specialist indicated to need financial support to experiment as well. The interview with the local council indicated that the local council currently acts on the incoming requests for subsidy rather than pointing what a vision should be. While some local councils do point to visions and thereby collect data for monitoring, the local council was satisfied with its current method and had no fear of missing out.
Social curiosity
During the interviews participants from school councils were asked to what extent they were curious to know about other school councils in order to get insight in their social curiosity. Results show that the councils showed interest in other parties when they were of a comparable size and same sector. Benchmarking costs with the commercial sector was off the table and not considered useful as there was believed to be a weighty difference in ambition and sector specific costs. All school councils in the Netherlands have organized themselves in national sector-association networks in which they come together to discuss common matters and issues every month. There is for example an association of educational specialist specifically for primary education, secondary education etcetera. Thereby these networks also serve as a party to commune with the government agencies. The interviews suggested that school councils try to find schools that have common struggles to form a coalition and conduct benchmarks mostly on regulations level
and partially on financial level. One school indicated that benchmarking is conducted to compare schools of the same council. Thereby the interviewee indicated that he does not see much value in comparing their own financial situation with other councils as no direct link with education quality and the financial investments can be made when all councils have approximately the same budget. Noticeably, the school councils prefer to engage in dialog with the council, over doing advanced analytics on the justification reports as this is believed to offer more in-depth knowledge sharing.
Interviews with the Inspectorate of education indicate that they are not interested in international benchmarking as rules and regulations of a country play a vital role in the financial position of educational institutions. The Netherlands has a lumpsum budget, while many other countries have privatized the education sector. Thereby this information does not seem of interest as no specific goal is bound to it.
The educational specialists attempt to facilitate curiosity between school councils by visualizing data and organizing sessions in which anomalies are discussed. The specialists themselves are curious on what insight the other sector has, which might also be applicable to their own sector.
The interview with the local council indicated that the council benchmarks with other large cities in terms of number of excellent schools. However, this is not a striving measurement. The participant indicated to find the freedom of councils essential and therefore did not wish to interfere or push the schools. Concluding the curiosity of participants when linked with the traits of curiosity of Kashdan (2018), the school councils score high on social curiosity, but do show risk averse behavior when considering their information sources and willingness to engage in uncertain data analytics activities. The school councils are therefore best identified as emphasizers. While the Inspectorate showed to score low on joyous exploration, it showed to be interested in stress tolerating activities such as the attempt for machine to machine justification. Thereby, high deprivation sensitivity was found in which the Inspectorate is categorized as a problem solver. The educational specialists were identified as joyous explorers as it is their job to bring insights to school councils. While traits of stress tolerance and social curiosity can be found, the educational specialist are categorized as the fascinated. The local council that was interviewed showed low traits of joyous exploration in the field of data analysis and admitted acting passively but kept its information up to date through regular meetings with the school councils. The local council is categorized as the avoider. A table of the traits compared per information consumer can be found in appendix V. Usability
During the interviews it was noticed that many participants were unfamiliar with the visualization of the financial statement. Therefore, participants were asked whether they are familiar with the published financial data in both raw and visualized format and if they did, how this was experienced. Overall, while all school councils were aware that the data is published in raw format, only four out of seven school councils were familiar with the visualization of the data. When asked about the usefulness of such a visualization, participants admitted to only have checked it once or twice. The school councils furthermore explained to see limited value in checking the page regularly as the data is not detailed enough or believed that the data despite of the standardization of XBRL, is still was not comparable as the accountant of a council also is believed to have a big hand in how the different costs are spread across the overview. They indicated that the data may come in handy for doing benchmarking against the national average or councils they already know to be similar to. Another council, that was not
familiar with the visualization before, indicated to want to find councils that are comparable through analysis of the data. One participant suggested that the data is currently static, and the visualization would be useful when parameters could be adjusted in which for example it would show the effect of decrease of number of students on indicators such as solvability, which is a vital indicator the Inspectorate magistrates on.
One of the most prominent users of the data is the Inspectorate of Education. The Inspectorate has regulated to have access to the data the moment it is submitted to the XBRL portal. From the interviews it has been derived that despite of this fast connection, the Inspectorate still needs to manually load the data into their own environment and thereby proactively need to check whether any report has been submitted. This, as the portal currently does not support planned structural data fetching. In order to prevent structural loading of all the report at once and reduce the workload, the inspectors would like to have this data load automatically. Once the data is fetched, the Inspectorate has advanced web-intelligence dashboards the data is loaded into and further analyzed. The Inspectorate thereby indicated to be satisfied with the data once it is loaded in their model. All three education specialists admitted downloading the raw data files to make visualization with this data. The specialists all have created dashboards with the data of their specific sector (primary, secondary, higher) in which they have visualized for example the financial indicators along with non-financial data such as number of students. One specialist admitted using multiple information sources, as this specialist was an administrating service, it had access to more current information and thereby preferred to do analysis using that data over the data from the financial report. The specialist thereby added that the data derived from DUO is reliable as its public justification data. The interview with the local council indicated that the participant was not familiar with the visualization of the financial report. As previously described, the local council is not interested in the financial report as the financial support of the local council is often too small to be mentioned in this report.
Concluding the results of the usability, it is found that the trust in quality motivates consumers to see it as an information source as this ensures usability. The education specialists show full trust in this data, while the inspectorate has doubts as the data is not certified. School councils and local councils thereby mostly due to the usefulness, had a lack of experience in using the data at all.
5. CONCLUSION
By looking into usefulness, usability and curiosity the value proposition of XBRL based reporting for the school councils, the Inspectorate of education, educational specialists and the local council is derived. With this information, an answer for the main question; To what extent can XBRL based reporting facilitate the knowledge forming and decision-making process of different stakeholders in the Dutch education sector?" can be formulated. The results indicate that:
▪ The data is used very limitedly by school councils and local councils. This can be linked to the fact that these participants limitedly experience triggers to become data driven. They mostly rely on their network i.e. educational specialists and the Inspectorate as these do respectively have the resources and knowledge to give insight and make the data comprehensive.
▪ Most decision making by school councils and local councils is currently done following intuitions and experience in which XBRL data is not considered due to the lack of context.
▪ While the Inspectorate and educational councils can be perceived as data driven, their social curiosity and
perception on the usefulness of the data is low. The Inspectorate uses strict measurements to detect financial threats through their data model that is facilitated by XBRL data. The Inspectorate is furthermore not excited to look into benchmarking possibilities as they do not perceive this as their responsibility.
▪ The Educational specialists possess data analytics knowledge but like the school councils state that context is required in order to perceive the data sincerely as useful for the knowledge forming and decision-making process. This context, such as governance structure and number of students can partly be found in the justification report in which a link between the two documents can bring better insight.
Therefore, it can be concluded that only the inspectorate is partly facilitated for the knowledge forming and decision-making process by XBRL data in which trust is still lacking as the data is not certified. The rest of the information consumers do limitedly see benefits in investing in its analysis, therefore, do not benefit in terms of knowledge forming and decision making. 6. DISCUSSION & FUTURE WORK
Data are just a batch of numbers unless its analyzed. As Andrew McAfee and Erik Brynjolfsson (2012) claimed, the fact remains that management needs to be aware of the potential of the data before the full potential can be unlocked. The same is true in this case study, as long as none of the consumers is exploring the data to find potential, it will not become a valuable information source. Public asset justification remains to be complex as the measurement that defines respectful usage is not fathomable due to the need of context. While all parties agree that there is not one number that defines justified usage, this is not a reason to not seek efficiency. During the interviews many councils mentioned to separate the public report for non-professional readers such as students and parents and the formal justification that they provide to the government. It is interesting to see how these parent-friendly versions of justification and policy explanation are different from the strategic justification report that is only read when the risk detection model finds anomalies.
As concluded that the XBRL data needs additional context to be useful, value will be created once pieces of text from the strategic justification will be linked with the XBRL data. As the strategic justification reports are perceived to be a more valuable source of information to gain insight in future position and decision making, it is interesting to look how to make the unstructured information in this report more graspable. Myšková, R., & Hájek, P. (2017) have conducted research on the predictability of financial health by using unstructured linguistic data from annual reports by conducting a text analysis. Their positive results direct to a chance in which new ways for justification can be considered. Thereby there are many text classification methods (Deng, X., Li, Y., Weng, J., & Zhang, J.,2019). that can be considered to classify the text in the justification report with XBRL data. The fact that XBRL is a semantic computer-readable format directs to an opportunity that needs to be considered.
Another point is that interorganizational knowledge sharing necessarily involves participation, at various levels, among multiple interested parties. The educational specialists stated to only visualize data that is acquired from external sources such as DUO, school councils and national statistic organizations. It is essential to understand that whatever is measured by the data providers, will be used as an information source that lead to decision making. Therefore, it is essential that the information providers such as government and statistic organizations mindfully examine why certain elements are measured and why other elements are not. Thereby school councils and local councils also need to consider and solidify what information can lead to better knowledge forming
