1
Table of Contents
1. INTRODUCTION ... 2
2. THE CONCEPT DEFINITON: NEWSBOARD ... 4
2.1 The Logic Behind the Interface Concept: AHP ... 4
2.2 Mock-Up Development ... 7
3. RESEARCH METHODS ... 10
3.1 The Case Study: Arçelik A.Ş. ... 10
3.1.1 Information about the Company ... 10
3.1.2 A Dashboard with the Newspaper Metaphor ... 10
3.2 User Study ... 11
3.2.1 The Protocol ... 11
3.3 The Themes of Analysis ... 12
3.4 The Assessment Model: TAMMS ... 12
4. RESEARCH RESULTS ... 13
4.1 The Coding Scheme of the Interviews ... 13
4.2 Answers to the Research Sub-Questions ... 13
4.3 Assessment Based on TAMMS ... 18
5. CONCLUSIONS ... 18
6. FUTURE WORK ... 18
7. REFERENCES ... 19
8. APPENDIX ... 21
8.1 Interview Scripts ... 21
8.2 Codes of the Interviews ... 45
8.3 Criteria to Prioritize the KPIs on Newsboard ... 45
2
ABSTRACT
In this paper, we aim to discover the advantages and disadvantages of an interactive mobile business intelligence interface concept with self-organization and dynamic structuring abilities. It is developed using a newspaper metaphor particularly from the perspective of user acceptance. This research investigates the proposed concept from four main aspects: dynamic structuring, metaphoric design, mobility and narrative visualization. The newspaper metaphor is used to increase the initial familiarity of a user with the interface concept which, in turn, may lead to greater user acceptance. We also suggest that dynamic the structuring ability of the concept is an alternative and novel approach to enhance the potential user acceptance. To enable this particular functionality, a scientific and quantitative KPI prioritization is used. Analytical Hierarchy Process (AHP) is the main method for this purpose and we explain how it can be employed for KPI prioritization on a business intelligence interface. The mobility is becoming an essential component to increase user acceptance for business intelligence. We argue that tablets are the appropriate devices to implement such a concept. Narrative visualization -the combined use of story-telling with data visualization- is a popular phenomenon in information visualization which is an essential component of business intelligence. Here, we also try to embed narrative visualization in the concept to facilitate quicker and deeper user insights which may have a positive impact on user acceptance.
In order to test the potential effects of this concept on user acceptance, a literature review, a case study, a user study, and interviews with business intelligence experts, reporting experts and top level executives were conducted.
The research results confirmed that the four main aspects of the proposed concept have a significant positive impact on the potential user acceptance for business intelligence.
Keywords
Analytic Hierarchy Process, Business Intelligence, Information Visualization, Metaphoric Design, Narrative Visualization, Newspaper, Tablets, User Acceptance, UX Design.
1. INTRODUCTION
The characteristics of the business settings of modern organizations have been rapidly changing recently [1] [2] [3]. Since its inception, information technologies have been a strong influential driver on transforming business operations [4]. Information has become a vitally important resource in the modern world’s knowledge based economy [5]. Lönnqvist & Pirttimäki (2006, p. 32) argue that “the need for timely and effective business information is recognized as essential for organizations not only to succeed, but even to survive”[1]. Therefore, it is critical for organizations to be competent in effectively acquiring, processing and analysing the information in order to enhance decision making processes [6]. According to Marjanovic (2007), the organizations which can effectively do this should be able to gain a competitive advantage since the analysed information can provide an insight about the current situation in the competition, therefore it enables to learn from the past and foresee the future [3]
According to a recent paper, the penetration of active platform users of business intelligence in organizations is only 24 per
cent [7]. However, in order to benefit from the potential positive impacts of business intelligence in an organization, it is highly advantageous to use it as a supportive tool for decision making. The more use is made of business intelligence, the more potential benefit it may bring to an organization. This is referred to as the “productivity paradox” which is defined as lower returns from information technology investments despite advanced technological developments. The underlying reason for the productivity paradox is claimed to be low system use [8]. Therefore, it is of a vital importance to enhance user acceptance of business intelligence systems. In this paper, we aim to design a novel interface concept to enhance user acceptance of a BI system in an organization which, in turn, would enable the organization to benefit from potential values that business intelligence may bring. The concept we propose is designed considering the needs from various aspects. We try to make use of the potential benefits of metaphoric design and narrative visualization and combine the two in a way that they create harmony to enhance user acceptance. In order to maximize the potential benefits of narrative visualization and metaphoric design, we support them by introducing mobility and dynamism.
The Potential of Narrative Visualization:
“I think people have begun to forget how powerful human stories are, exchanging their sense of empathy for a fetishistic fascination with data, network, patterns, and total information… Really, the data is just part of the story. The human stuff is the main stuff, and the data should enrich it.”
Whale Hunt (Segel and Heer, p. 1140) [9]
In order to communicate information to a recipient, a new class of visualizations have emerged which combine interactive graphics with narratives. According to Segel and Heer (2010), storytellers, notably journalists working in online business, have increasingly been using this combination to create richer stories for the users and convey the information more effectively [9]. Gershon and Page (2001), argue that with the use of narratives, information can be communicated in more psychologically-efficient format. This is also one of the main goals of visualization design [10]. Wojtkowski and Wojtkowski (2002), claim that the complexity of the content might make visual story-telling necessary. In conclusion we believe that narrative visualization might become crucial, especially for large data sources, to provide fast and intuitive exploration.
In the current information visualization world, the most sophisticated tools mainly focus on data exploration and analysis rather than story-telling. They provide necessary supplementary elements to support discovering stories in data, but do little to create actual narratives [9]. One of the goals of this study is to elaborate on this gap by studying the need for story-telling functionalities in information visualization tools for the business intelligence domain.
There are seven genres of narrative visualization identified by Segel and Heer (2010): magazine style, annotated chart, partitioned poster, flow chart, comic strip, slide show and video.
3 The question at stake is which genre is best suited for business intelligence? In this paper, our concept is closest to the magazine style narrative visualization.
The Power of Using Metaphors:
Literature findings reveal that the art of using metaphors in user interface design might be a useful approach to create powerful stories of information. However, it is not an easy task to achieve. Creating strong stories of data, which enables powerful communication of information, requires skills beyond technical expertise of engineering and science [10].
According to Dix et al. (1998), using metaphors creates a value by increasing the initial familiarity between a user and computer interface [11]. Apple Computer Inc. mentions the potential power of using metaphors in its Machintosh Human Interface Guidelines 1992. According to Apple Computer Inc., developers can take advantage of users’ existing knowledge of the world by using metaphors in order to convey concepts and features of an application. Users can form a set of expectations about an interface if metaphors, which may reflect concrete and familiar ideas of users about the real world, are used while designing it [12].
Using metaphors in user interface design is a widely used approach for enhancing user acceptance. According to Qian et al. (2011), people create an internal model of systems and using this internal model, they identify the ways to figure out how system is functioning. Therefore, designing interfaces based on the users’ mental model have been a common point for discussion in the literature [13]. Johnson-Laird (1986) argued that the users’ mental models has a crucial and unifying part in representing real life objects, sequences of events, the way the outside world is and psychological actions of daily life[14]. According to Qian et al. (2011), there are two mental models to be identified; a user’s mental model [15] referring to the beliefs of a user about a system, and a designer’s mental model, referring to the conceptualization of the system used [15] [16]. Metaphors are being used to bridge this gap.
Newspaper is an old and widely used concept with which everyone is familiar. In our concept the newspaper metaphor is used to design the interface concept considering this initial familiarity of users with the interface concept which may reduce the required adaption time of a user. It is also an appropriate approach for creating story-telling interface by providing the opportunity to insert explanatory and narrative texts which may boost the richness of the information displayed. Therefore, the use of the newspaper metaphor may support the potential contributions of narrative visualization. In this sense, we hypothesize that the newspaper metaphor is an appropriate concept to enhance user acceptance.
The Need for Dynamism
Using a newspaper metaphor induces a number of expectations for the end-user. In particular, users expect that a newspaper brings the latest up-to-date news and emphasize the important ones. By reading the headlines and taking an initial look, you should have a basic understanding of the most important updates. Therefore, the proposed use of the newspaper metaphor imposes dynamism because every day a newspaper contains different news or has a different order of the sections based on
the current agenda. The interface concept needs to be designed in a way that it orders the sections dynamically based on the importance of each section. This may enable quicker transmission of information to a user by providing an initial impression about current state of affairs. We argue that users have little time to analyse and derive conclusions of data; therefore some part of the analysis task should be transferred to the interface. This, as a result, may lead to higher user acceptance.
The Need for Mobility:
According to Power (2013), in today’s business world, decision making activities are more sensitive to the time variable due to high competition and fast flow of the information [17]. The majority of decisions are delegated to managers who are mobile, on the go, or away from office environments. The conventional methods of providing decision support systems and BI on desktop devices is getting less workable every day. Therefore, the manager’s decision making process needs to be supported without the confines of place or time.
Mobility of the employees, especially decision makers, is necessary since frequent contact with stakeholders is crucial for the performance of organizations. Therefore, they need availability of their BI tools which supports their decision making processes at any time, at any place. Decentralized decision making is getting more important and mobile business intelligence might be one of the enablers [17].
As a consequence of these needs and the potential benefits, many organizations focus their BI strategies on mobile business intelligence and the interest in it has grown rapidly in commercial markets [18].
The Concept: Newsboard
Considering the potential effectiveness of story-telling user interface design, the power of using metaphors, the need for dynamism and the need for mobility in business intelligence, we introduce a concept called the “Newsboard” which is developed within the scope of this research in order to reveal the ways to enhance potential user acceptance. As it is proposed in this paper, the Newsboard is a dashboard concept which is designed using a newspaper metaphor, has dynamic structuring capabilities that prioritizes KPIs -in this case newspaper sections- based on the data coming to the dashboard, and is implemented on tablets since nowadays tablet newspapers are widely used and users have high familiarity with them which would yield greater user acceptance.
The Research Question:
The main goal of this study is to enhance potential user acceptance of business intelligence in organizations by using such a novel interface concept. The main research question which is investigated in this paper is:
How can we create a dynamically-structured dashboard designed using a newspaper metaphor on tablets and does it improve user acceptance in comparison to a static desktop based approach?
The concept will be examined from 4 main aspects: metaphoric design (the use of the newspaper metaphor), mobility (the use of tablet as a medium), dynamic structuring and self-organization
4 capabilities, and story-telling design. A case study and user studies including a mock-up testing and interviews with top level executives and business intelligence professionals are conducted in order to clarify the problem and propose a sound solution.
The following figure illustrates the hypotheses developed to test the impact of each aspect on user acceptance:
Figure 1-1 Hypotheses
The structure of the paper is as follows: problem definition, description of the Newsboard concept and logic behind it, a case study which reveals and confirms the further improvement opportunities on the concept, presentation of the research results, and finally, discussion and recommendations for future research.
2. THE CONCEPT DEFINITON:
NEWSBOARD
The newsboard is an interface concept which is a multi-column layout with rectangular subsections of key performance indicators. The subsections are dynamically ordered using a scientific prioritization method which is defined in the following section. The layout looks like:
HEADLINE: KPIa
Secondary Section: KPIb Third Section: KPIc Fourth Section: KPId Fifth Section: KPIe
2.1 The Logic Behind the Interface Concept:
AHP
As explained in the introduction section, the newspaper metaphor is used to increase the familiarity of users with the concept which may enable them to transfer their experience about how things work to the interface which may result in a higher potential user acceptance. The use of a newspaper
metaphor requires having an order on the interface which is based on the importance of “the news”, or in this case KPIs. The interface should analyse the data coming to it and dynamically structure the sections of “the newspaper”. Literature review is conducted to find the ways to introduce dynamism to the concept. The leading idea to activate dynamism in the interface is to find a quantitative and scientific KPI prioritization method which might be a part of the engine working behind the interface. Currently, most of the organizations use qualitative KPI prioritization methods such as a strategy map and balance scorecards which may obstruct the integration of dynamism to the interface concept designed here.
Shahin and Mahbod (2006) [19] propose a qualitative way to prioritize the KPIs. According to the reference, organizations deal with high number of key performance indicators related to different functions and, because of this, the introduction of a number of new approaches is necessary to enhance the conventional ways of selecting and prioritizing KPIs.
According to Locke and Latham (1990, pp. 252-261) [20], specific and challenging goals lead to a higher performance than moderate or easier goals. Therefore, some KPIs are definitely more important than others in organizations. The KPI prioritization problem should be viewed as a multiple criteria decision making problem for a healthy assessment of the performance.
Shahin and Mahbod (2006), claim that analytical hierarchy process (AHP) may fit this particular purpose. Analytical
5 hierarchy process is referred to as most efficient and widely used method for multiple criteria decision making problems in organizations [21] [22] [23]. It has been widely used especially in the domains of strategic planning of resources in organizations [24], the assessment of strategic alternatives [25], and the justification of manufacturing technologies [26]. AHP is proven to be appropriate in prioritizing variables which are alternatives to one another [27].
Shahin and Mahbod (2006) propose a novel method to prioritize KPIs based on the use of AHP method. They specifically used the “SMART” criteria -which stands for Specific, Measurable, Attainable, Realistic and Time-Sensitive- to assess the KPIs. However, the use of SMART criteria in the reference is for a generic purpose of guiding decision makers, strategic planners or policymakers. Since the aim of the prioritization in this research is different, this generic approach wouldn’t be a perfect match for prioritizing the KPIs because the users of the Newsboard might have different criteria to assess the importance of the KPIs in their everyday agenda. Therefore, we will use different criteria to rank the KPIs and to reveal these criteria is a part of this research. Yet, this does not change the logic of AHP calculation and the proposed engine behind the interface.
2.1.1.1 Analytical Hierarchy Process (AHP)
1) The History of AHP
In the early 70s, Saaty, developed a novel approach to help decision makers overcome complex problems in a simple way [28]. He defined the first complete application of the theory in Sudan, on 103 ranked road, rail, air, port and river transportation projects [29]. After this implementation, in 1980 Saaty published his textbook named “The Analytical Hierarchy Process”. In 1983, he released the PC based software of analytical hierarchy process which is entitled as “Expert Choice” [30]. Expert Choice helped AHP to become popular and practical in the operations research field.
Within the quarter of the century since he published his first papers on AHP, it has become widely used by decision makers to solve complex decision making problems in more than 30 different areas which include strategic planning, resource allocation and policy making. It is being used to rank, assess, select and benchmark various alternatives in decision making problems. It has been used by companies in private and public sectors to tackle complex problems [19]. In the 80s, AHP was used by IBM as an integral part of its Silverlake Project to assess its computer-aided manufacturing process at its Minnesota unit. As a result, in 1990, IBM won the Malcolm Baldrige National Quality Award[31]. For almost three decades, AHP is being taught as a part of the curriculum at wide variety of engineering and business schools all over the world [19].
2) The Working Principle of AHP
There are three basic principles which analytical hierarchy process is based on.
- Decomposition - Comparative Judgement - Synthesis of Priorities
AHP is a measurement theory to overcome intangible, but quantifiable, criteria that is applicable to a wide range of domains, such as conflict resolution and decision making [32].
Since the criteria to be used in Newsboard are possibly also intangible, but it needs to be quantified to enable a quantitative calculation behind the interface, it is an appropriate method to use.
AHP calculations are generally achieved in 3 steps [19]: 1) Structuring the problem into a hierarchical framework
–
The overall goal should be specified in the first step. In our case, the top goal is to prioritize the KPIs into a set of quantifiable weights.
After having the ultimate goal, the criteria should be defined to assess the different alternatives. In this case, the alternatives are the KPIs. The criteria will be investigated and discovered as a part of the research later. However, as previously stated, the criteria do not have an impact on the logic of the calculation. To sum up:
- Top Goal: A Ranking of the KPIs - Alternatives: A set of KPIs to assess
- Criteria: are used to assess and compare the alternatives
Figure 3-1 provides the overview of hierarchy structure for AHP calculation in Newsboard KPI prioritization case.
Figure 2-1 Hierarchy Structure of Newsboard KPI Prioritization (Adjusted from the model of Shahin and Mahbod 2006)
2) Comparative Analysis –
After the hierarchy is formed, ratio priorities for each node of the hierarchy need to be established. This is achieved by performing pairwise comparisons between the child items with respect to a parent node. These comparisons are based on either the contribution or the importance of each child item to the parent item. Thus, this is a top-down process. When the comparisons are done for a single node, a standardized principal eigenvector of the comparison matrix is obtained. By this way, the values add up to one and eventually become the ratio on the measure of the relative importance of each child item. As a result of this whole process, the local weights of each child item, in this case each KPI, are obtained.
3) Aggregation of the Local Weights into a Composite Priority –
In this final step, after obtaining the local weights they are multiplied by the product of all higher level priorities. By this way global weights are calculated which reflect the importance of each child node in the overall hierarchy. The global weights
6 are summed up for a specific alternative to obtain a composite priority which measures the composite impact of all criteria. To enable dynamism and self-organizing structure of the sections on Newsboard, the global weights of the KPIs will not be enough. To provide insight to the users about what is specifically important on a specific day, the global weights, or in other words the relative importances of the KPIs need to be combined by the change in the KPIs. Therefore, the ultimate goal in the process is not to select a specific KPI, but to rate and obtain the rankings of the KPIs. In that way, the headline and subsequent sections will be determined and ordered on Newsboard. To sum up the proposed steps in order to structure the sections of Newsboard:
Step 1: Define and List the Relevant KPIs
The relevant KPIs which would be the candidates to be displayed on Newsboard should be selected in the first step.
Step 2: Define the Criteria to Compare the KPIs
The criteria should be defined which will be used to prioritize the KPIs.
Step 3: Establish AHP Hierarchy Based on the Criteria After having the two levels, the AHP hierarchy should be constructed as illustrated in Figure 3-1.
Step 4: Make Pairwise Comparisons
AHP pairwise comparisons are done using a nine point scale [23]:
Table 2-1: A Nine Point Scale of AHP Calculation (Source: Saaty, 1994)
Step 4.1 Obtaining Pairwise Comparisons of the Criteria
The criteria to assess the KPIs are listed and a comparison matrix is constructed. Then all criteria are compared based on a nine-point scale for AHP analysis.
Step 4.2 Obtaining Normalized Pairwise Comparisons of the Criteria
The normalized values are obtained by dividing each member of each column by the total value of that column Step 4.3 Obtaining Normalized Pairwise Comparisons of the KPIs with Respect to Each Criterion
Similar to Step 4.1, all KPIs are compared to one another with respect to each criterion. Therefore, 5 calculations should be made (for each criterion). Then the values are normalized by dividing each member by the total value of the corresponding column.
Step 5: Calculate the Global Weights
Global weights are obtained by multiplying local weights of each KPI with respect to each criterion with the local weights of each criterion (calculated in Step 4.2). For instance, if we denote the Local Weights of the criteria with the abbreviation LWCn
and the Local Weights of the KPIs with LWKPI and Global
Weight of the KPIs with GWKPI, the formula to calculate the
weight of KPI x looks like as follows
(LWC1 x LWKPIx) + (LWC2 x LWKPIx) + (LWC3 x LWKPIx) + (LWC4 x LWKPIx) …+ (LWCn x LWKPIx) =
GWKPIx
As a result, the general ranking based on all of the criteria is obtained.
Step 6: Multiply Global Weights with Change Ratio of the KPIs Having the priorities of each KPI after step 5, the dynamism on Newsboard will be enabled by multiplying the global weights with the amount of change of the KPIs. If we denote the Final Weight of KPI x with FWKPIx and the change rate of KPI x with
Change(%)KPIx, the formula to obtain final weights which are
used to rank the KPIs (or sections) on Newsboard is given by: FWKPIx = GWKPIx x Change (%)KPIx
Step 7: Obtain the Rankings of the KPIs (Order of the Section on Newsboard)
The new rankings are obtained which are used to construct a daily Newsboard. Assuming that Newsboard displays 5 KPIs,
the rankings would look as follows:
KPIs Global Weight Ranking
KPIa … 1
KPIb … 2
KPIc … 3
KPId … 4
7 And finally the Newsboard structure would be:
HEADLINE: KPIa Secondary Section:
KPIb
Third Section: KPIc Fourth Section: KPId Fifth Section: KPIe
If the change rates were not a part of the calculation, the order would be the same as in Step 5 which reflects the ranking based on global weights only. However, the change (%) variable increases the dynamism of the interface concept. Assuming that the global weights of KPIs remain static over time, the second variable in the calculation, which is the “change rate”, would sustain the dynamism in the interface since the change rates may vary periodically (e.g. daily, weekly, monthly…).
There might be different ways of doing AHP calculation. It might be done on a personal basis which would result in different priorities, thus a different dashboard for every individual. It might also be done in a functional basis which would result in a different dashboard for every functional unit. Or it might be done in a consensus group to agree on the prioritization criteria and pairwise comparisons. If the calculation is done in a consensus group, users will have a “global” version of Newsboard which displays the same information for all users based on the same prioritization criteria. If the calculation is done in a personal or functional basis, users will have different versions of Newsboard according to their own priorities (e.g. CFO edition, CMO edition)
The concept also enables the user to customize the sections of the newspaper manually independent from the ranking obtained as a result of the calculation. This feature is added to provide user more freedom which may enhance the user acceptance.
2.2 Mock-Up Development
In order to test the concept on the potential users and receive feedback from the experts, a mock-up was needed. By performing a mock-up experiment, the interviewees will have a better understanding of the concept which may lead to more reliable research results. Because of the stated reasons about the need for mobility in the introduction, the concept is to be implemented and displayed on a tablet.
The mock-up was developed using SAP BusinessObjects Dashboards 4.0 environment.
For mock-up development, 5 example KPIs were chosen as a result of consultation with Deniz Günay (Business Intelligence Specialist at Arçelik A.Ş.). Those are: net consolidated sales, gross domestic sales, retail stocks, purchasing rate from Low-Cost-Countries and market share.
The following steps illustrate the steps in the calculation and the construction of the Newsboard mock-up presented in the following section. As for the mock-up we don’t have access to the statistics required to define the weights of the various KPIs we use random numbers for this illustration of the method.
Step 1: Define and List the Relevant KPIs
The relevant KPIs which would be the candidates to be displayed on Newsboard should be selected in the first step. If we take Arçelik A.Ş. as an example, this is done by the Corporate Performance Development Department.
Step 2: Define the Criteria to Compare the KPIs
The criteria should be defined which will be used to prioritize the KPIs. Since the criteria to prioritize the KPIs are unknown at the time of mock-up development, 5 generic criteria from Criteria 1 to Criteria 5 were used.
Step 3: Establish AHP Hierarchy Based on the Criteria After having the two levels, the AHP hierarchy should be constructed as illustrated in Figure 2-1.
Step 4: Make Pairwise Comparisons
AHP pairwise comparisons are done using a nine point scale from “equal importance” to “extreme importance” [33].
Step 4.1 Obtaining Pairwise Comparisons of the Criteria The criteria to assess the KPIs are listed and a comparison matrix is constructed. Then each criterion is compared with respect to another based on a nine-point scale for AHP analysis.
Criteria 1 Criteria 2 Criteria 3 Criteria 4 Criteria 5
Criteria 1 1.00 3.00 2.00 1.00 1.00 Criteria 2 0.33 1.00 1.00 0.33 0.50 Criteria 3 0.50 1.00 1.00 3.00 2.00 Criteria 4 1.00 3.00 0.33 1.00 2.00 Criteria 5 1.00 2.00 0.50 0.50 1.00 TOTAL 3.83 10.00 4.83 5.83 6.50
Table 2-2: Pairwise Comparisons of the Criteria to be used to prioritize the KPIs
Step 4.2 Obtaining Normalized Pairwise Comparisons of the Criteria
The normalized values are obtained by dividing each member of each column by the total value of that column:
Criteria 1 Criteria 2 Criteria 3 Criteria 4 Criteria 5 TOTAL
Criteria 1 0.26 0.30 0.41 0.17 0.15 1.30
Criteria 2 0.09 0.10 0.21 0.06 0.08 0.53
Criteria 3 0.13 0.10 0.21 0.51 0.31 1.26
Criteria 4 0.26 0.30 0.07 0.17 0.31 1.11
Criteria 5 0.26 0.20 0.10 0.09 0.15 0.80
Table 2-3: Normalized Pairwise Comparisons of the Criteria
Step 4.3 Obtaining Normalized Pairwise Comparisons of the KPIs with Respect to Each Criterion
Similar to Step 4.1, all KPIs are compared to one another with respect to each criterion. Therefore, 5 calculations should be made (for each criterion). Then the values are normalized by dividing each member of each column by the total value of that column.
8
Retail Stocks Gross Domestic Sales
Net Consolidated Sales
Purchasing Rate from LCCs
Market Share TOTAL
Criteria 1:
Retail Stocks 0.10 0.06 0.14 0.10 0.17
0.56
Gross Domestic Sales 0.48 0.28 0.21 0.48 0.39
1.84
Net Consolidated Sales 0.29 0.56 0.42 0.29 0.28
1.84
Purchasing Rate from LCCs 0.10 0.06 0.14 0.10 0.11
0.50
Market Share 0.03 0.04 0.08 0.05 0.06
0.26
Criteria 2:
Retail Stocks 0.08 0.03 0.13 0.09 0.08
0.42
Gross Domestic Sales 0.42 0.16 0.20 0.06 0.42
1.25
Net Consolidated Sales 0.25 0.31 0.40 0.56 0.25
1.77
Purchasing Rate from LCCs 0.17 0.47 0.13 0.19 0.17
1.13
Market Share 0.08 0.03 0.13 0.09 0.08
0.42
Criteria 3:
Retail Stocks 0.40 0.47 0.58 0.17 0.20
1.82
Gross Domestic Sales 0.13 0.16 0.10 0.42 0.47
1.27
Net Consolidated Sales 0.13 0.32 0.19 0.25 0.20
1.09
Purchasing Rate from LCCs 0.20 0.03 0.06 0.08 0.07
0.45
Market Share 0.13 0.02 0.06 0.08 0.07
0.37
Criteria 4:
Retail Stocks 0.25 0.28 0.40 0.38 0.16
1.47
Gross Domestic Sales 0.05 0.06 0.07 0.04 0.06
0.27
Net Consolidated Sales 0.08 0.11 0.13 0.19 0.16
0.68
Purchasing Rate from LCCs 0.12 0.28 0.13 0.19 0.31
1.04
Market Share 0.50 0.28 0.27 0.19 0.31
1.55
Criteria 5:
Retail Stocks 0.07 0.07 0.07 0.07 0.10
0.38
Gross Domestic Sales 0.36 0.35 0.33 0.43 0.30
1.76
Net Consolidated Sales 0.36 0.35 0.33 0.29 0.30
1.62
Purchasing Rate from LCCs 0.14 0.12 0.16 0.14 0.20
0.77
Market Share 0.07 0.12 0.11 0.07 0.10
0.47
Table 2-4: Normalized Pairwise Comparisons of the KPIs with Respect to the Each Criterion
Step 5: Calculate the Global Weights
Global weights are obtained by multiplying local weights of each KPI with respect to each criterion with the local weights of each criterion (calculated on Step 4.2). The values in Table 3-3 and Table 3-4 are used for this calculation. For instance, the calculation of global weight for retail stocks is given by:
(0.56 x 1.3) + (0.42 x 0.53) + (1.82 x 1.26) + (1.47 x 1.11) + (0.38 x 0.8) = 5.1700
As a result, the general ranking based on all of the criteria is obtained:
KPIs Global Weight Ranking
Net Consolidated Sales
6.7574
1
Gross Domestic Sales
6.3771
2
Retail Stocks
5.1700
3
Purchasing Rate from LCCs
3.5760
4
Market Share
3.1195
5
Table 2-5 Global Weights and Initial Rankings of KPIs Those values and rankings will stay static overtime unless the priorities of the users change and calculation is done again. Step 6: Multiply Global Weights with Change Ratio of the KPIs Multiplying the global weights by the amount of change of the KPIs. Consider the following scenario:
- Net Consolidated Sales increased by 9 percent compared to previous year.
- Gross Domestic Sales increased by 11 percent compared to previous year.
9 - Air Conditioner Retail Stocks are accumulating by 8
percent.
- Purchasing Rate from Low Cost Countries have increased by 8 percent compared to previous year. - Market Share in UK has increased by 7 percent
compared to previous year.
In order to have the aggregate weights used to rank KPIs on the Newsboard, these change rates are multiplied by the global weights obtained by AHP calculation.
FWKPIx = GWKPIx x Change (%)KPIx
Step 7: Obtain the Rankings of the KPIs (Order of the Section on Newsboard)
The new rankings are obtained which are used to construct Newsboard:
KPIs Global Weight Change(%) Final Weight Final Ranking Net Consolidated Sales 6.7574 9% 0.6082 2
Gross Domestic Sales 6.3771 11% 0.7015 1
Retail Stocks 5.1700 8% 0.4136 3
Purchasing Rate from LCCs 3.5760 8% 0.2861 4
Market Share 3.1195 7% 0.2184 5 Table 2-1 Final Rankings of the KPIs
According to these change percentages, Newsboard will be automatically constructed in the following way:
Figure 2-2 The Newsboard Mock-up (v.1) .
Since gross domestic sales changed more than net consolidated sales, the aggregate weight of gross domestic sales happened to be higher than the aggregate weight of net consolidated sales.
However, net consolidated sales have a higher priority in general. If change (%) of gross domestic sales is lower than net consolidated sales, the net consolidated sales section becomes the headline by getting the highest final ranking.
An Alternative Scenario
Assume that the pairwise comparisons of the criteria change in the following way:
Criteria 1 Criteria 2 Criteria 3 Criteria 4 Criteria 5
Criteria 1 1.00 0.11 0.17 0.20 0.17 Criteria 2 9.00 1.00 2.00 9.00 1.00 Criteria 3 6.00 0.50 1.00 1.00 0.50 Criteria 4 5.00 0.11 1.00 1.00 0.50 Criteria 5 6.00 1.00 2.00 2.00 1.00 TOTAL 27.00 2.72 6.17 13.20 3.17
The normalized values would be:
Criteria 1 Criteria 2 Criteria 3 Criteria 4 Criteria 5 TOTAL
Criteria 1 0.04 0.04 0.03 0.02 0.05 0.17
Criteria 2 0.33 0.37 0.32 0.68 0.32 2.02
Criteria 3 0.22 0.18 0.16 0.08 0.16 0.80
Criteria 4 0.19 0.04 0.16 0.08 0.16 0.62
Criteria 5 0.22 0.37 0.32 0.15 0.32 1.38
If we assume that pairwise comparisons of the KPIs with respect to each criterion remain the same, we would obtain the following global weights:
KPIs Global Weight Ranking
Retail Stocks
3.8495
3
Gross Domestic Sales
6.4745
2
Net Consolidated Sales
7.4402
1
Purchasing Rate from
4.4260
4
Market Share
2.8098
5
Consider the following scenario in another day on the Newsboard; - Net Consolidated Sales increased by 12 percent
compared to previous year.
- Gross Domestic Sales increased by 11 percent compared to previous year.
- Air Conditioner Retail Stocks are accumulating by 8 percent.
- Purchasing Rate from Low Cost Countries have increased by 8 percent compared to previous year. - Market Share in UK has increased by 7 percent
compared to previous year.
The final weights and rankings would be as follows:
KPIs Global Weight Change (%) Final Weight Final Ranking
Net Consolidated Sales 6.7574 12 0.8109 1
Gross Domestic Sales 6.3771 11 0.7015 2
Retail Stocks 5.1700 8 0.4136 3
Purchasing Rate from LCCs 3.5760 8 0.2861 4
10 As a result, net consolidated sales would be the headline and the final Newsboard would be:
Figure 2-3 The Newsboard Mock-Up (An Alternative Scenario) The user study was conducted using the mock-up illustrated in Figure 2-2 and Figure 2-3.
3. RESEARCH METHODS
In order to answer the main research question introduced in the first section of this paper, a case study and a user study were conducted based on interviews with business intelligence experts and C level employees.
3.1 The Case Study: Arçelik A.Ş.
This research has been conducted in a collaboration with Arçelik A.Ş. based on a previous implementation of a business intelligence dashboard using the newspaper metaphor. The paper takes this interface as a starting point for the research. Before defining the concept, we give some background information about the company.
3.1.1 Information about the Company
The information as taken from the company website1:“Arçelik A.Ş. is a company having operations in durable consumer goods industry with production, marketing and after-sales services, it offers products and services around the world with its 25,000 employees, 14 different production facilities in
1Arçelik A.Ş. Corporate Overview
http://www.arcelikas.com/page/72/Corporate_Overview, retrieved on May 28, 2015
five countries (Turkey, Romania, Russia, China and South Africa), its sales and marketing companies all over the world and its 10 brands (Arçelik, Beko, Grundig, Blomberg, ElektraBregenz, Arctic, Leisure, Flavel, Defy and Altus). The consolidated net sales turnover in 2014 was 12.5 Billion Turkish Liras.”[34]
3.1.2 A Dashboard with the Newspaper Metaphor
In 2010, Arçelik A.Ş. started a new business intelligence programme. BI platform was implemented on SAP BusinessObjects. The first phase of BI programme was to develop executive dashboards for C level employees and directors. In the first year of new business intelligence programme, more than 20 executive dashboards were implemented and they were accessible for targeted user segments. However, usage logs revealed that the number of users per day was far lower than expected. User acceptance rate was thus not as expected. Several solution alternatives were attempted including sending teaser or warning e-mails to the users and directing them to the platform, locating 55’’ touch screens in the rooms of the users which continually display the dashboards. These attempts, as well, couldn’t become successful.In 2012, the MIS department decided to develop an executive dashboard using a newspaper metaphor considering the characteristics and user profile of the target segment. Despite the fact that the new dashboard contained the same information with the old version, usage logs and extremely positive feedback from the users revealed that user acceptance rate is much higher than the previous dashboard. Figure 3-1 shows how the dashboard looks like:
Figure 3-1: ARTI News
However, in the long term, usage rates were decreased. Several reasons were identified by the MIS department for the long term low performance. Those reasons were also supported by the interviews conducted with the users, BI experts, and an ex-manager of the MIS department and director of budget, reporting and analysis:
1) The interface was static. Therefore, it became boring and uninteresting for the users. It was repeating the same headline and subsections every day.2
2
(Interview with Rakibe Musal, Ex-Manager of MIS Department, Appendix 8.1, Lines 810, 840)
11 2) The interface was not available on mobile devices.
Desktop usage has generally been decreased, that applied to this particular dashboard as well.3
3.2 User Study
3.2.1 The Protocol
As explained in the case study, the current dashboard which is still in use at Arçelik A.Ş. was taken as a starting point. The concept was improved on two main additional aspects: dynamic structuring and mobility. We developed a mock-up explained in the previous section to test it on the users which are top level executives of Arçelik and the experts in business intelligence. It is hypothesized that such an interface would enhance the user acceptance in business intelligence.
In order to answer the research question introduced in the introduction section, it is decomposed into the 4 main aspects: dynamic structure, mobility, metaphoric design and insight driven design. The hypotheses are formed based on these categories (Figure 1-1)
Two main research methods were used in this research. A literature Review and semi-structured interviews together with an experimental mock-up were conducted with business intelligence experts, reporting experts and managers, and high level executives.
Mainly qualitative methods were employed because the potential sample size was too low to perform quantitative analysis. It was chosen to gather high quality information directly from people who have an experience with a newspaper formatted dashboard. The interviewees were chosen amongst the C level executives, directors, reporting managers and BI experts from Arçelik A.Ş. In addition, senior BI consultants and high level managers from external BI consultancy companies were included who worked with Arçelik A.Ş. during the implementation of current dashboard. The president of Koç Holding Durable Goods Group which is the parent company of Arçelik A.Ş. contributed to the research, as well. Here is the complete list of 12 interviewees:
NAME JOB TITLE COMPANY
Fatih Kemal Ebiçlioğlu
President of Koç Holding Durable Goods Group
Koç Holding Polat Şen Chief Financial Officer Arçelik A.S. Tülin Karabük Chief Marketing Officer Arçelik A.S. Hüseyin Öner Director of IT Arçelik A.S. Sibel Yazıcı
Kesler
Director of Budget
Reporting and Analysis Arçelik A.S. Hümeyra Özener Manager of Financial and Managerial Reporting Arçelik A.S.
Rakibe Musal Manager of Price Cost
Analysis Arçelik A.S.
Hande Güniçen Manager of Corporate Performance Development Arcelik A.S. 3
(Interview with Hüseyin Öner, Director of Information Technologies, Appendix 8.1, Lines 652,653)
Erkan Altan Managing Partner Metric Technology Halil Bakan Expert BI Consultant Metric Technology Deniz Günay BI Specialist Arçelik A.S Tuğba Atmaca BI Specialist Arçelik A.S.
Table 3-1: List of Interviewees
The 6 steps of the user study and the interview protocol are given by:
1) a brief explanation of the mock-up,
2) gathering the information about the participant, 3) an introductory training about the interface while it is
presented to the participant (explanations are given how to use the interface and how dynamic composition of the newspaper is enabled using AHP model)
4) free exploration of the interface 5) conducting the interview
6) filling in post-interview using 15 Likert scale questions
The mock-up illustrated on Figure 2-3 was presented to the users. Interviewees were asked the following questions:
Background Information:
1) Can you introduce yourself and your position in the organization? a. Age
b. Position/Level in the hierarchy c. Experience
Business Intelligence Literacy:
1) How familiar are you to the concept of business intelligence? For how long?
2) How often do you use the business intelligence platform of your company?
3) How important BI/accessing information about the performance of the company is critical when you are making decisions?
Mobile Usage:
1) Does your company provide mobile business intelligence? 2) Do you use BI on mobile devices? If not why?
3) What are the possible advantages of accessing BI on mobile platforms?
Newspaper Interface:
1) You already know about or use newspaper formatted dashboard on desktop devices. What benefits have it brought to you/your company when you start to use it?
2) What did you like/not like about the newspaper formatted dashboard on tablet?
3) What are the possible advantages of having it on a mobile device? 4) Which organizational level should be targeted with this kind of
interface? This interface is assumed to target the top level hierarchy. Do you think this concept is also applicable for the lower levels in the hierarchy?
5) What criteria should be considered when constructing/ordering the sections of the newspaper? (Which criteria should be used in AHP model to prioritize the sections?)
6) What are the advantages/disadvantages of newspaper formatted dashboard comparing to regular/static dashboards?
7) Should the newspaper look the same for all of the users or should it be customized according to the users? Why?
12
insight to the users. To what extent do you think that this is the case?
9) Newspaper formatted dashboard have much more text than a regular dashboard in order to provide quick insights to the users. Do you think that this is a problem?
10) Is there any functionality you would also like to have on the newspaper formatted dashboard?
11) Any other thoughts/ideas/remarks?
The interview scripts are located in Appendix 8.1.
3.3 The Themes of Analysis
The following research sub-questions will be answered using the methods indicated below the each one.
Sub-Question 1: What criteria should be used to prioritize KPIs
on Newsboard in order to enable dynamic structuring?
- Semi Structured interviews with BI professionals and top level executives.
Sub-Question 2: What are the advantages and disadvantages of
implementing a dynamically structured dashboard designed using the newspaper metaphor as a business intelligence interface on tablets?
- Semi Structured interviews with BI professionals and top level executives.
4 aspects (for topic based analysis; see Figure 1-1) will be examined while analysing interview results
- Dynamic structure
o H0: Having a personalized version of
Newsboard increases user acceptance o H0: The dynamic structuring ability of
Newsboard increases the tendency of users to use Newsboard more often
o H0: The dynamic structuring ability of Newsboard increases user acceptance - Mobility
o H0: Tablet is a suitable device for applying the
newspaper metaphor.
o H0: Provided mobility by tablet increases the
tendency of users to use Newsboard more often
o H0:The use of tablet as a medium (mobility) increases user acceptance
- Metaphoric Design
o H0: The use of newspaper metaphor decreases
time required for user adaptation.
o H0: Newsboard is only suitable for higher
level executives in the organizations.
o H0: The use of newspaper metaphor increases user acceptance
- Insight Driven Design
o H0: Newsboard provides quicker insight about
data than a regular dashboard
o H0: Newsboard provides deeper insight about
data than a regular dashboard
o H0: Insight driven design increases user acceptance
3.4 The Assessment Model: TAMMS
Dillon (2001) defined user acceptance as “the demonstrable willingness within a user group to employ information technology for the tasks it is designed to support” [35]. The challenge of user acceptance is not confined to the domains of business intelligence or mobile services but it is a vital issue for software development in general [36].
There are several methods and models developed to assess user acceptance such as Theory of Reasoned Action (TRA) [37], Theory of Planned Behaviour (TRB) [38]. TRA and TRB described subjective norms, perceived behaviour and attitude as the major determinants of a user’s objective and as a result his/her actual behaviour. In the literature, it is agreed that such determinants actually affect the user acceptance [36]. However, it has been criticized that the factors have week prediction power and reliability, and they are too simplistic [39].
Davis developed Technology Acceptance Model (TAM) [40] in 1986 which is a widely used and well-known acceptance model in the scientific user acceptance research area. It has been studied and improved intensively in the coming years. It has been adapted by different domains including mobile services.
Brockmann et al. (2012) argue that the Technology Acceptance Model for Mobile Services (TAMMS) is the best choice to analyse user acceptance in the mobile business intelligence domain [36]. TAMMS uses the original version of TAM and extends it with various factors which are specific to mobile services domain based on the findings of different field studies [41]. These specific factors are “perceived ease of use”, “trust” and “perceived value” which influences the intention to use a mobile service. Additionally, “taking into use” is influenced by the “perceived ease of adoption” and taking into use is an intermediary step which is required to transform “intention to use” to “actual use”. Figure 4-2 illustrates the TAMMS model based on the modifications on the original TAM model:
Figure 3-2: TAMMS (which are highlighted by thicker edges on the boxes) [36]
The results on the user acceptance of Newsboard will be evaluated based on the TAMMS model.
13
4. RESEARCH RESULTS
4.1 The Coding Scheme of the Interviews
As stated in the introduction, the hypotheses are formed based on the 4 aspects. In order to reveal and understand the patterns in the interviews, the scripts were marked and coded according to the 4 themes. Those are metaphoric design, dynamic structure, mobility and insight driven design. For each topic (theme), a colour was assigned and the relevant text blocks were marked with that colour in the scripts (see Appendix 8.1).Theme (Topic for Analysis) Colour Metaphoric Design
Dynamic Structuring Mobility
Insight Driven Design
Table 4-1 Colours for Coding
In total 157 codes were identified within the scripts. The following chart illustrates the occurrences of each theme within the interview scripts. The numbers may contain a bias based on the questions posed. Yet, we observed that the interviewees showed significant amount of interest to the dynamic structuring abilities. They also asked considerable amount of questions to understand how dynamic structuring works. The number of ideas proposed by the interviewees was also high about dynamic structuring. Interviewees did not discuss much about the mobility mainly because the potential advantages of the provided mobility were very clear and unquestionable. The complete list of the codes which shows the actual occurrences and line numbers can be found in Appendix 8.2.
Figure 4-1 The distribution of the codes in the interviews The proposed answers to the research sub-questions introduced in section 4.3 are presented in the following section.
4.2 Answers to the Research Sub-Questions
Sub-Question 1: What criteria should be used to prioritize KPIs on Newsboard in order to enable dynamic structuring?The criteria to prioritize the KPIs were asked to the interviewees. Shahin and Mahbod (2007) use SMART (Specific, Measurable, Attainable, Relevant, Time Sensitive) to prioritize the KPIs. However, we hypothesized that users may have different criteria to rank their KPIs. Therefore, the interviewees were asked to list
the 5 most important criteria for them. The complete list of the criteria stated by the each of the interviewees is located in Appendix 8.3. The 5 most commonly stated criteria were as follows:
1) Up-to-date (12 Occurrences): KPIs on Newsboard should be chosen among the ones which are updated more frequently.
2) Strategically Relevant (12 Occurrences): KPIs on Newsboard should be relevant to current strategic goals of the company.
3) Actionable (11 Occurrences): KPIs on Newsboard should be able to lead decision makers to take actions. 4) Related to Personal Targets (7 Occurrences): KPIs
related to individual targets might lead users to increase their attention and motivate them to use Newsboard more often.
5) Specific (5 Occurrences): KPIs should be specific to the function which the user is working in.
Almost all of the interviewees mentioned up-to-date, strategically relevant and actionable among the most important criteria to prioritize the KPIs to structure Newsboard. “Up-to-date” was, most of the time, firstly mentioned as criterion. Interviewees think that displaying up-to-date information fits with the newspaper metaphor since one of the most important characteristics of newspapers is that they display up-to-date information to the readers4.
In order to increase the curiosity and the interest in using the Newsboard, interviewees also claim that KPIs displayed on Newsboard should be relevant to personal targets of a user; they should be strategically important, quickly actionable and specific to a function. They think that the use of those criteria may enhance user acceptance5.
Sub-Question 2: What are the advantages and disadvantages of implementing a dynamically structured dashboard designed using the newspaper metaphor as business intelligence interface on tablets?
4 topics of analysis were used in order to answer this question. In addition to the interviews, 15 Likert scale questions were asked to support the qualitative findings of the interviews. The complete list and the answers of each of the interviewees are present in Appendix 8.4. The following table shows the averages of scores for the Likert scale questions (over 5.00):
4
Interviewee 1: Tuğba Atmaca, Business Intelligence Specialist at Arçelik A.Ş., Line 133
Interviewee 5: Hüseyin Öner, Director of IT at Arçelik A.Ş., Line 734 Interviewee 8: Polat Şen, Chief Financial Officer at Arçelik A.Ş., Line 1100
5
Interviewee 3: Hande Güniçen, Manager of Corporate Performance Development at Arçelik A.Ş., Line 469
Interviewee 10: Fatih Ebiçlioğlu, President of Koç Holding Durable Goods Group, Line 1248
14
# Question Average
1 I like the Newsboard. 4.92
2 Newsboard is easy to use 4.75
3 I would use the dynamic Newsboard more often
than regular static dashboard. 4.83
4 I like the dynamic structure of the Newsboard 4.83 5 Newsboard would provide quicker insight than
a regular dashboard. 4.67
6 Newsboard would provide deeper insight than a
regular dashboard. 2.33
7 The dynamic structure of Newsboard makes it
harder to get used to. 1.5
8 The dynamic structure of the Newsboard would
increase my motivation to use it more often. 4.75
9
I would like to have the Newsboard personalized according to my priorities rather than having a global version.
4
10 I would use the Newsboard more often since I
can access it on my mobile device. 5
11 Tablet is a suitable device for using the
Newsboard. 5
12 Newsboard has too much text. 2
13 Newsboard is too complex for me to use. 1.17 14 Newsboard is only suitable for top level
hierarchy. 1.42
15 Newsboard can be used by employees of all
level in the hierarchy. 4.67
Table 4-2 Answers of Likert Scale Questions
4.2.1.1 Dynamic Structure
Dynamic structuring and self-organization functionality of the concept was found as the strongest advantage of Newsboard as indicated by the interviewees:
“Actually, dynamism triggers people. If a report remains static, it usually becomes boring and usage rate decreases. This dynamism might be a solution to keep people alerted. This is a clear advantage” Hümeyra Özener, Manager of Financial and Managerial Reporting at Arçelik A.Ş., Line 250
“This dynamism makes the report more attractive for users. If the report is static, after some point, users become numb. But the dynamism of your concept makes important changes more salient and visible for users. Therefore they may become more open to the information. If you make it personal, they can adopt it easier and use more often” Hande Güniçen, Manager of Corporate Performance Development at Arçelik A.Ş., Line 440
“Dynamism may excite the users. It creates curiosity. This depends on how interesting and
dynamic the content is. If had to think about the trade-off between the dynamism and short adoption time of a user because of a static structure, I would choose the dynamism. It may increase the usage rate therefore decreases the required time for adoption” Tülin Karabük, Chief Marketing Officer at Arçelik A.Ş., Line 1188
“Dynamism is good. You provide user the opportunities to personalize the interface. That would definitely increase the user acceptance. Apart from the self-organization capability of the interface, you also give a freedom to adjust the sections. Customization is very important in user acceptance.”
Erkan Altan, Managing Partner at Metric BI Consultancy, Line 1322.
As the quotes from the interviewees indicate, the proposed dynamic structuring ability of the Newsboard by using AHP calculation and giving the users the freedom to arrange the sections according to their interests have very positive impact on the user acceptance. The users indicate that they may use it more often than a static dashboard since it increases the curiosity and interest. Therefore, the dynamic structuring and self-organization functionality of the Newsboard enhances user acceptance. Answers to the Likert scale questions which relate to the dynamism of the concept also support this:
Moreover, users reject that dynamism may increase the required adoption time. The main reason claimed by the interviewees is that the usage rate may increase since dynamism raises curiosity, therefore as the usage rate increases, the required time for adoption decreases.
In response to the interview question 7 for the newspaper interface, respondents indicated that having a personal version of Newsboard based on the personal prioritization criteria will have a very positive impact on user acceptance. However, they also indicated that having a “global” version may help employees to speak the same “language” in the company. Surprisingly, all of the respondents suggested a hybrid solution to tackle this potential problem. They proposed to have both personal and global versions of the Newsboard by designing two pages for each on the same interface. In this way, enhancing user acceptance by a personal version would not hinder speaking the same business “language” in the company.
“I will suggest you a more hybrid approach. What about having both personal and global versions? Employees should be able to track the important KPIs
15 of the company which are relevant to the company’s
general strategic priorities. For instance I need to keep an eye on daily installation quantities by product group even if this is not exactly my area. Some KPIs are relevant to the others, therefore should be followed by everyone. You can have a two page newspaper. First page may display a personal version such as CFO edition or CMO edition. Second page may contain general KPIs. Users should be able to customize their own sections, as well. In that way, they can accept using the dashboard easier.”
Polat Şen, Chief Financial Officer at Arçelik A.Ş, Line 1049
As a result, the following hypotheses are not rejected meaning that dynamism has a positive impact on user acceptance:
H0: Having a personalized version of Newsboard
increases user acceptance
H0: The dynamic structuring ability of Newsboard
increases the tendency of users to use Newsboard more often
H0: The dynamic structuring ability of Newsboard
increases user acceptance
4.2.1.2 Metaphoric Design
It was hypothesized that the use of newspaper metaphor may have a positive impact on user acceptance since the concept is familiar to the potential users and therefore this decreases required time and effort for adaptation. The findings from the interviews confirmed this hypothesis.
“The use of newspaper metaphor makes interface closer to real life. Therefore it is easier for the users to make the connection. It is easier to get used to it…A newspaper, with its visual characteristics and its structure, is a concept which the users are very familiar with. When they use it, it is clear for them what to expect.”
Tuğba Atmaca, Business Intelligence Specialist at Arçelik A.Ş., Line 11, 33
“When we used this metaphor in the company, it enabled users to get familiar with the interface quicker. It increased the usage rate dramatically compared to the old static interface. It was also an innovative and nice idea. Users liked it very much. It was something new for them. This is a way of transmitting information to users which actually exists in real life. It is easier for them to understand. It has a real positive impact on user acceptance.” Deniz Günay, Business Intelligence Specialist at Arçelik A.Ş., Line 543
“Most of the people have the routine of reading daily newspaper. This concept makes an easy connection to this routine. This is a very strong aspect of your concept. People get used to easier thanks to this metaphor.”
Hümeyra Özener, Manager of Financial and Managerial Reporting at Arçelik A.Ş., Line 371
Supporting the literature review findings about the metaphoric design, the interviews strongly confirmed the potential positive impact on user acceptance. Therefore, the following hypotheses are not rejected:
H0: The use of newspaper metaphor decreases time
required for user adaptation.
H0: The use of newspaper metaphor increases user
acceptance
As indicated in the case study section, when the dashboard with a newspaper metaphor was implemented in Arçelik A.Ş., the target group of users was top executives who need summary information and more unfamiliar with modern dashboards. The hypothesis was that the use of the newspaper metaphor is mainly suitable for top level employees. However, this hypothesis was rejected as a result of interviews. Interviewees claim that top executives are not the only ones who are familiar with the newspaper metaphor. In the fast paced business environment where instant access to the important and prioritized information is vital, users need such interfaces which provide quicker insight about the data. Answers given to the Likert scale questions support that, as well:
Therefore, the following hypothesis is rejected.
H0: Newsboard is only suitable for higher level
executives in the organizations.
All in all, the second aspect of the concept, metaphoric design, has a potential positive impact on user acceptance, as well.
4.2.1.3 Insight Driven Design
It is hypothesized that the dynamic structuring ability and automatic prioritization of the KPIs would provide users quicker insight about the data since the interface is structured in a way that more important data is highlighted. This may have a positive effect on user acceptance. Findings from the interviews support this:
“It is better to make it dynamic. In this way, it directly displays what is important and what is less important. We don’t have much time to spend on reports. We need to see the problem instantly” Fatih Ebiçlioğlu, President of Koç Holding Durable Goods Group, Line 1237
“People have little time to analyse data and derive insights. Everybody wants to get the data like a pill. Users don’t want to spend time with complex reports and they just want to see analysed data and insights derived from it.”
Tuğba Atmaca, Business Intelligence Specialist at Arçelik A.Ş., Line 165
“A dashboard should undertake the analysis task on itself. We should minimize the required time for users to get the business insights from data. That kind of interface may successfully achieve this.”
