RuG Master Thesis
Improving Quattro Performance Measuring
System: towards Strategies Leading to Success
Yazhen Wang S1840940 Groningen, April 2011 Supervisor: dr. J. S. Gusc 2nd supervisor: dr. W. Westerman University of Groningen
Preface
This is a master thesis for the final session of my study of International Business Management with a specialization in International Financial Management in University of Groningen. This thesis present the result of a project I did as an intern in Quattro, an e-marketing solution provider based in Cape Town, South Africa.
During a four-month period from 15 October, 2010 to 15 February, 2011, I joined Quattro as an accounting and financial intern. The focus of my work is on performance management and the main purpose of my project in this company is to improve their performance measuring system. This is a challenging task involving investigations into the company from different aspects. It could not be accomplished without the help and cooperation of everyone in the company. I would like to thank all managers and staff of Quattro, for their provision of documents, cooperation during interviews and contributions of good ideas. Their openness and friendliness make my life and work in Cape Town enjoyable.
Furthermore, I would like to thank my supervisor in University of Groningen, Dr. J. S. Gusc for her advices, support and feedbacks to conduct the project and improve my work. I would also like to thank Mr. P. Honig, the CEO of Quattro as well as my supervisor in the company for giving me the opportunity to do an internship in his company and providing me with lots of help during the project.
Thank you,
Yazhen Wang
Abstract
Table of Content
Preface...1
Abstract ...2
1 Introduction ...1
1.1 Quattro and GraphicMail ...1
1.2 The Market of Permission-based e-Marketing Service ...2
1.3 Goal Statement and Problem Description...3
1.4 Research Scope ...4
1.5 Research Questions ...4
2 Methodology ...5
2.1 Research Design...5
2.2 Research Strategy...6
2.3 Data Collection and Analysis...7
3 Literature Review ...9
3.1 Effective Performance Measuring Systems ...9
3.2 Performance Measuring System Design Approaches...11
3.3 Performance Measuring Models ...13
4 Problems of the Current Quattro Performance Measuring System...16
4.1 The current Performance Measuring System ...16
4.2 Diagnosing the Problems ...16
4.3 Conclusion ...18
5 Improving the Design of the Performance Measuring System...19
5.1 Identifying Strategic Objectives...19
5.1.1 Mission/ vision/ value of Quattro...19
5.1.2 Crucial Success Factors...20
5.1.3 Mapping Organizational Strategic Objectives ...23
5.2 Selecting Organizational Performance Measures...24
5.3 Designing Departmental Performance Measures ...25
5.3.2 Setting Departmental Objectives and Selecting Measures for Departments ...29
6 Conclusions and Discussions ...30
6.1 Conclusions...30
6.2 Limitations ...32
6.3 Potential Difficulties for Implementation and Suggestions for Further Work ...32
Reference...34
1 Introduction
In this chapter, first some general background information about the subject company – Quattro, its products and the market it operates in is provided. Then the goal of this research is stated and the problems of the company is introduced to explain the need of this project. After that the scope of this research is defined and the research questions are presented.
1.1 Quattro and GraphicMail
Quattro is an Application Service Provider creates hosted business solutions through one headquarter, two subsidiaries and 8 franchisees at different geographical locations. The most important product of them is GraphicMail, by which they provide mobile and e-mail marketing solutions such as designing, delivering and tracking graphical e-mail newsletters and managing subscribes list. One new product they're going to launch is Cliku, by which specialized mobile marketing solutions are provided to customers. The company was founded in 2001.1 Now there are 35 employees and a few interns working for this company, including 3 top managers – CEO, CFO and CTO, who are also owners of this company. Other 32 staff work for 6 functions: programming and testing, marketing and communication, sales and support, network administration, design as well as administration/ operation.2 Although it is a small company, Quattro has a high level of internationalization both in markets and organizations. GraphicMail is providing service to 16 countries in 8 languages. And it is now planning to enter a few other countries and to develop products in several new languages.3 The sales channels of Quattro include direct sales, resellers, franchisees, private labels and corporate customers. The
headquarter of the company is in Cape Town, South Africa, where the whole marketing function, network administration function, design function and administration/ operation function, most part of the sales and support function and half of the programmers are hosted. 2 other sales & support persons are located in the US, which is their biggest market and where the server hosting of the company lies.4 In the external environment the company operates, the main stake holders are suppliers from whom the company rents its infrastructures, direct customers, resellers, franchisees, and private-labeled customers. The organizational structure, a description of functions of departments and their role in the company’s core activities are presented latter in Section 5.3 as part of the result from the interviews of the study.
1
www.quattro.co.za
2
Quattro Staff and Contact Information
3 www.graphicmail.com
This company is applying a web 2.0 culture,5 which encourage free and instinct communications through diverse modern communication tools such as Skype, Google Doc, Yammer and social networks. It also tries to create relaxed working environment with lower levels of formalization and centralization, encouraging new ideas from all employees.
1.2 The Market of Permission-based e-Marketing Service
Quattro operates in the permission-based e-marketing service market as an e-mail and mobile marketing service provider. Understanding the characteristics and trends of the market the company operates in will provide some views on what the company needs to be successful in this market.
Permission marketing is a term defined by Godin (1999) to describe the kind of marketing activities that before marketers send communicational massages to customers, they must get permissions from the massage receivers for doing so. Permission-based e-marketing is a kind of marketing done through e-communication tools such as e-mail and mobile massages. Since the emerge of such marketing tools, the application of permission-based e-marketing by the marketers experienced the fastest growth for similar disposing rate with traditional ways of marketing but lower-cost and more effective in customer communication. It also enjoys advantages in getting customer loyalty since the communication is customer permitted (Drèze 2005). In 2007 e-mail marketing overtook direct mail and became the most frequently used vehicle for marketing.6 But e-marketing also has its threats such as strict legal environment because of anti-spam laws and customer’s increasing sensitiveness about internet safety (Drèze 2005).
For e-marketing service providers, several trends of the market must be aware of to keep continuous competitiveness. The first is customer requirement of one integrated platform, with the multiplication of desktop applications, mobile/ iPhone applications, iPad applications, etc. The second is the increasing importance of social networks. Social network supported e-mail marketing and integrating e-mail marketing with social networks will strengthen the need in this market. The third is customer demand of the integration of e-marketing application and other applications such as CRM (Customer Relationship Management), ERP (Enterprise Resource Planning) and social media applications.7 Those trends implicated a need for e-marketing service
5
Quattro and GraphicMail Strategy Process – the Market
providers to strengthen cooperation with other service providers, a need for keeping up-dated of the development of the internet environment, and a need for continuous innovation in search of new applications and features.
1.3 Goal Statement and Problem Description
The purpose of this research is to improve the performance measuring system (PMS) of Quattro so that it can better monitor and guide the company’s activities towards its strategic objectives. And achieving those objectives could finally lead the company to success.
The need of improving the PMS is stimulated by management challenges of Quattro. When the company was founded it had fewer employees, smaller business, narrower product range and served fewer customers. The control of the company process was relatively simple and the managers were always able to monitor the performance of the company because not a lot of information they would need to do that. After years of continuous fast development, Quattro now realize that the management tools they’ve been using to monitor and control the activities of the company is out-dated and not sufficient any more. To solve the problems the management of the company decided to implement better reporting mechanisms . Besides the two existing
Diagnoses of the problems of the company’s PMS will be presented later in Chapter 4 after the definition of a satisfying PMS for Quattro. But the problems appeared are enough to indicate that further improvements need to be done to this system.
With the results of this project, Quattro could have a chance to benefit from an improved PMS. The company also has a chance to benefit from side-products during the diagnosing and improving process, e.g. better understanding of its activates and departmental functions, or clarified hierarchy of strategic objectives. Such side-products might help with the management of the company beyond this project. Although the diagnostic conclusions and solutions are made in the situation of one specific company, they could also provide managerial implications to companies in similar situations and facing similar problems.
1.4 Research Scope
It is proposed that the development of a PMS can be divided into three main phases (Bourne et al. 2000): the design of performance measures; the implementation of performance measures and the use of performance measures. And the design phase can be subdivided into identifying the key objectives to be measured and designing the measures themselves. Accomplishing all three phases is estimated to take more than 6 months and could be up to one year. Limited by the four-month research period, this research only focus on improving the design of the PMS – the process of identifying performance objectives and designing measures. It is also proposed that the development of a PMS involves three levels of design, organizational, departmental and individual (Kaplan and Norton 1996). This
research only focus on the company level and departmental level but not individual level. Quattro is a small company with simple organizational structure and all department are actually small groups with people closely cooperating with each other. Individual work can be very flexible and the so is the control and evaluation.
The end product of this research are improved sets of performance measures of objectives identified at both organizational and departmental level. Some discussion of potential difficulties and barriers to implement the PMS is included as suggestions for the next steps. But how to implement the system by collecting data, sorting information and deciding distributions, as well as how to use the system for allocating problems, improving performance and updating strategy are out of the scope of this research.
1.5 Research Questions
How to improve the design of the performance measuring system of Quattro to guide the company activities towards successful strategies?
To answer this question several sub-questions need to be answered: How to define an effective performance measuring system for Quattro? Is the current system effective? If not what are the problems?
How to define the objectives of the company? What performances need to be measured?
What indicators need to be selected to measure them?
2 Methodology 2.1 Research Design
As stated in the last chapter, the main issue this research attempts to address is to improve the performance measuring system (PMS) of Quattro. To accomplish this goal, three phases of work need to be done. The first phase is to define an effective performance system for Quattro. The second phase is to diagnose the problems of the current PMS by checking whether it fits the effectiveness defined in Phase 1. The third phase is to improve the system by solving problems identified in Phase 2.
In Phase 1, effectiveness of the PMS for Quattro is defined on two kinds of evidences. On one hand, literatures on designing effective performance systems and principles for designing performance measures are reviewed. On the other hand, expectations of users of the PMS - managers and employees of Quattro, are considered. The descriptions of expectations are generated from interviews.
In Phase 2, problems of the current PMS are identified by analyzing information from two aspects and comparison between the current PMS and the effective PMS defied in Phase 1. One source of evidence are the performance reports and related company documents. Another are the
The work in Phase 3 is relatively more complicated than those in the first two phases. To improve the design of the system, the performance measures need to be re-selected at both organizational and departmental level, which can be done by answering two questions: what performances of whom to measure and how to measure them. 4 steps are taken for changing the design of performance measures. The first step is to identify the strategic objectives and their hierarchical relationships. For this step, some preparing work is done by reviewing literatures on approaches and tools used in the designing phase of PMS such as Crucial Success Factors (CSF) and Strategic Map. Following these approaches and applying the tools, the company’s strategies are analyzed. Evidences used in this step include reviewing company documents on strategic management and interviews employees on strategic related and function related topics. The second step is to determine the performance perspectives need to be measured. The perspectives are used for cross-checking with the objectives to see if they covers all important performance measuring aspects. This step is done by reviewing performance measuring (PM) models for small business and service sector in literature. The third step is to design the measures for the objectives determined in Step 1 and 2. Bases of this step are existing measures in current PMS and
employees’ ideas of feasible and best measures of performance generated during interviews. The last step is to develop departmental objectives and design measures for the departments. Bases of this step are interviews, observations and document review on departmental functions and contributions to company main activities
2.2 Research Strategy
This study is a business research, which refers to a “organized, systematic, data-based critical, objective, scientific inquiry into a specific problem, undertaken with the purpose of finding answers or solutions to it” (Sekaran and Bougie 2010). It starts from examining a practical problem in its own specific context, rather than from building a theoretical framework first and then testing it. It does not have clearly defined research hypotheses for testing. Instead, it investigates into problems of a system and tries to construct an improved one. The methods suitable for such research are qualitative methods (Thiétart 2001). The corresponding approach for this type of research should be inductive approach, which usually moves from observations of the specific to general implications and conclusions (Heit 2000).
data to provide a tentative problem solution for identified problems and then implement the solution, to evaluate, define and diagnose the resolving efforts (Sekaran and Bougie 2010). Although limited to the research period, there will not be sufficient time for full implementation of the improvement plans to the PMS and observations to the results, feedbacks from end users could be available at any stage of the research as a way to evaluate and improve the resolving efforts.
2.3 Data Collection and Analysis
Three data collecting method are used in this research: interviews, desk research and observations. The most important data collecting method used in this research is face-to-face interviewing. Interviews are used as a way of cooperative inquiry in this action research. Cooperative inquiry aims at working with people having similar concerns and interests to understand the situation, create ways of looking or learn how to make changes for better practices(Heron and Reason 2001). Thus interviews are done with a thought of working together with people involved and learn from their opinions instead of only asking questions as a pure researcher. This kind of interviews are supposed to provide the most diverse and detailed information for exploratory research on practical issues like this one. And it is suitable because face-to-face communication with most members of the company is possible and relatively convenient in situation of this project. However, for the company members outside Cape face-to-face interview is not possible and due to the difficulty of time arrangement (there are time differences between Cape Town and other geographical locations), telephone interviewing is not a good idea either. The same
questions asked during interviews in Cape Town are sent to those members and their written answers are sent back. The interview type of this research is a semi-structured individual interview (Thiétart 2001). General subject theme areas are defined ahead of time and questions are asked in a way without interventions or limitations to direct the subjects’ respondents but with a structured guide defined in advance (the interview plan is shown in Appendices 1). For the phase of defining effectiveness, a theme topic of expectations to the PMS is designed. For the phase of problem identification, a theme topic of problems/ barriers encountered during work is designed. For the phase of identifying objectives and designing measures, theme topics of
Additional questions are asked to top managers. The respondents selected are all 35 members of Quattro, with 24 done by face-to-face interview and 11 by email. All 24 face-to-face respondents responded and 6 out of 11 email respondents responded.
Another data collecting method used in this research is desk research to secondary data. Desk research is relevant to action research because some opinions of people with similar concerns, including current and past employees and interns, are reflected in secondary data. Getting those opinions reflected and understood is another way of involving all participators in this action research. Relevant secondary data includes company documents, company accounting information and reports form professional organizations. Background information about the company, products and market is collected from company website, inside and outside reports on the market environment. For the phase of problem identification, management reports, the new “KPI combined” and illustrating document of it are studied. For the phase of identifying
objectives and designing measures, documents on organizational structure and responsibilities are reviewed to study the functions and activities of the company; strategy related reports such as SWOT analysis, company road maps and strategic direction descriptions are studied to help with defining strategic objectives; all financial and non-financial reports are reviewed to decide the possibility of measure designing.
Besides, by the nature of action research, observations are also used to collect information on functions and activities of the company. The observations are done from a perspective of a researcher as a participant, through observing tasks performed by departments and employees, and time spent on those tasks from ACE Project and company roadmaps.
3 Literature Review
In the theoretical part of this study, literature reviews are done in three areas. The first are criteria/ requirements for an effective performance measuring system (PMS). As a conclusion of this part of literature review effectiveness of Quattro PMS is defined. The second are approaches and tools used in designing an effective PMS. After this part of literature review the approach and tools for designing Quattro system is determined. The third are the models and frameworks of PM
perspectives. As a conclusion a general framework of PM perspectives for Quattro PMS is formed.
3.1 Effective Performance Measuring Systems
Neely et al. (1995) defined a performance measurement system as a set of metrics used to quantify both the efficiency and effectiveness of actions. An effective PMS enables the managers of an organization to determine whether the activities occurring within a facility are supporting the achievement of objectives linking to the organization’s vision and daily activities and enables members of the organization to see where the organization is and where it is going (Hacker & Brotherton, 1998). An effective PMS must first be able to fulfill these functions. In some researches other criteria/ requirements for designing qualified PMS are discussed.
Globerson (1985) concluded with 8 principles for designing PMS: 1) performance criteria must be chosen from the company’s objectives; 2) performance criteria must make possible the comparison of organizations which are in the same business; 3) the purpose of each performance criterion must be clear; 4) data collection and methods of calculating the performance criterion must be clearly defined; 5) ratio-based performance criteria are preferred to absolute number; 6) performance criteria should be under control of the evaluated organizational unit; 7) performance criteria should be selected through discussions with the people involved (customers, employees, managers); 8) objective performance criteria are preferable to subjective ones.
Kanji (2002) developed 7 requisites for a good performance measurement system: 1) it provides performance from a multi- and interrelated perspective; 2) it is linked to the organization’s values and strategy; 3) it is based on the critical success factors or performance drivers; 4) it is valid, reliable and easy to use; 5) it enables comparisons to be made and progress to be monitored; 6) it is linked to the rewards’ system and encourages the appropriate behaviors; 7) it highlights improvement opportunities and suggests some improvement strategies.
Discussion
Among researches in this area the above three provide thorough information on how an effective PMS should be like and focus on measure design itself instead of the whole PMS process including implementations, using and improving loop. There are some common principles in the three researches, especially in the first two. And Kanji’s (2002) work provide more supplements because it is done more recently. Excluding a few criteria related to the implementation and using stage of the PMS, e.g. having clearly data collection and calculating method, changing of
measures as evolving of circumstances and linking the measures to rewarding system to
encourage appropriate behaviors, the characteristics of an effective PMS can be summarized and interpreted in the circumstances of Quattro as the following:
1) It should be designed basing on strategic objectives of the company.
2) It should provide performance information from multiple and interrelated perspectives. By the expectations of managers of Quattro, those perspectives should be from all important aspects influencing the success of the company.
3) It should include departmental level measures. Those measures should vary from different departments and suit the circumstances of each department. And the measures should be under control of the departments evaluated.
4) The measures should be able to enable the determination of both where the company is and where it need to go. Thus there need to be both indicators of results and indicators of factors determining the results for enabling continuous improvement.
5) It should be simple and easy-to-use. This requires the measures to be easily
6) The measures should be designed through discussing with people involved. The 6 criteria of effectiveness are used for diagnosing the PMS of Quattro in Chapter 4. 3.2 Performance Measuring System Design Approaches
There are various discussions on what procedural approaches to follow for designing a PMS in literature. E.g. Kaplan & Norton (1992, 1996) suggested a process involving a serial of interviews and workshops to link long-term strategies to organizational activities; Neely et al. (1996)
suggested a management process dividing the development of PMS into three main phases; Flapper et al. (1996) also suggested a systematic development method including three main steps. Among those approaches, Otley (1999) proposed that a performance management framework can be built by answering a serial of questions, which makes the process more intuitive and practical. Those questions are: 1)What are the key objectives that are central to the organization’s overall future success, and how does it go about evaluating its achievement for each of these objectives? 2)What strategies and plans has the organization adopted and what are the processes and activities that it has decided will be required for it to successfully implement these? How does it assess and measure the performance of these activities? 3) What level of performance does the organization need to achieve in each of the areas defined in the above two questions, and how does it go about setting appropriate performance targets for them? 4) What rewards will managers (and other employees) gain by achieving these performance targets (or, conversely, what penalties will they suffer by failing to achieve them)? 5)What are the information flows (feedback and feed-forward loops) that are necessary to enable the organization to learn from its experience) and to adapt its current behavior in the light of that experience?
Question 3 to 5 of Otley (1999) are dealing with issues of planning, target setting, rewarding and feedback and learning of the PMS. Only the first two questions are relevant to the design of measures. The two questions suggest that before selecting the performance measures, two aspects of work need to be done: identifying the objectives central to the organization’s future success; and identifying the processes and activities required to achieve its strategic objectives. However, Otley has not provide detailed approaches for identifying the two.
Identify the firm’s strategic objectives using the mission statement as a guide (profitability, market share, quality, cost, flexibility, dependability, and innovation). 3) Develop an
understanding of each functional area’s role in achieving the various strategic objectives. 4) For each functional area, develop global performance measures capable of defining the firm’s overall competitive position to top management. 5) Communicate strategic objectives and performance goal to lower levels in the organization. Establish more specific performance criteria at each level. 6) Assure consistency with strategic objectives among the performance criteria used at each level. 7) Assure the compatibility of performance measures used in all functional areas. 8) Use the performance measurement system to identify competitive position, locate problem areas, assist the firm in updating strategic objectives and making tactical decisions to achieve these objectives, and supply feedback after the decisions are implemented. 9) Periodically re-evaluate the appropriateness of the established performance measurement system in view of the current competitive environment.
Also, In his book about how to develop and implement performance indicators, Parmenter (2007) presented a process as a journey from missions and visions of organizations to performance measures, which provide detailed approach and tools for objective identifying. In this process first the mission/ vision/ value of an organization is defined, reviewed and communicated, before strategies issues towards 6 perspectives being derived from them. Then the strategies issues should be cross-checked with critical success factors (CSF) identified for the organization to develop a hierarchy of different levels of strategic objectives. To be noticed, CSFs, which are defined "events, circumstances, conditions, or activities that require special attention because of their significance" (Dickinson et al. 1984), are used as a tool for strategic development tool in the Parameter (2007) process. It is also mentioned in Medori (1998) as the first stage for developing an integrated PMS framework. Another tool used by Parameter (2007) is strategic map developed by Kaplan & Norton (1992, 1996) and used in balanced scorecard for understanding the
hierarchical relationships among strategic objectives leading to the ultimate goals of the organization.
Discussion
Third, basing on the CSFs identified, strategic objectives are determined and cross-checked with the necessary PM perspectives concluded from literature review. Those strategic objectives are formed into a map showing hierarchical relationships among those factors leading to overall strategic goals. Fourth, main activities/ processes of the company and roles of each department involved in the activities are discovered. Fifth, organizational and departmental level of measures are chosen from the PM perspectives.
3.3 Performance Measuring Models
Different models in literatures of discuss from which perspectives organization should have their performance evaluated, e.g. the pyramid model of Lynch & Cross (1991), the result and
determinant model of Fitzgerald et al. (1991), the 4 perspective balanced scorecard model of Kaplan and Norton (1992, 1996). Hudson et al. (2001) even developed which could evaluate the PM frameworks by examining whether they measure performances from certain 6 perspectives. Because Quattro has its uniqueness being a small- and medium- sized enterprise (SME) and a service provider, reviewing the PM framework developed out of evidence from service industry or SMEs provide more meaningful opinions for this research.
Fitzgerald et al. (1991) developed a PM framework for service industry from case studies to British companies in service industry. In this system they pointed out 6 dimensions of performance: financial performance, competitive performance, quality of service, flexibility, resource utilization and innovation. The measurements on those dimensions can be financial or non-financial, internal or external. The crucial feature of this model is that the six performance factors are typed into two conceptually deferent categories - results and determinants.
Competitive performance and financial performance are results because they reflect the success of the chosen strategy. The other four are determinants for determining competitive and financial performance. This system with theme of measures is presented in Table 1.
Table 1 Performance Measurement Framework for Service Industry (Fitzgerald et al. 1991) Categories Performance Perspectives Themes of Measurements
relative market share and position sales growth
competitiveness
measures of customer base profitability liquidity results financial performance capital structure reliability determinants quality of service
aesthetics/ appearance cleanliness/ tidiness comfort friendliness communication courtesy competence access availability security volume flexibility delivery speed flexibility flexibility
specification flexibility productivity
resource utilization
efficiency
performance of the innovation process innovation
performance of individual innovation
Studying Finish technology SMEs, Laitinen (1999) developed a dynamic PM model. This model is linked to an activity-based cost approach, by which the five stages of factors of the
performance measurement follow the service providing process from production to customer. The five stages of factors are: cost of production factors (financial dimensions), production factors financial dimensions), activities (financial and non-financial dimensions), products (non-financial dimensions) and revenue/ cost of products ((non-financial dimensions). The five factors connected with the internal performance of the company determine external performance in terms of financial performance and competitiveness. Therefore the researcher gives 2 other external factors: competitiveness (financial and non-financial dimensions) and financial performance (financial dimensions). A summary of this model is presented in Table 2.
Table 2 Dynamic Performance Measurement Model of Laitinen (1999) Categories Performance Perspectives Themes of Measurements
profitability liquidity financial performance capital structure growth of revenues external competitiveness
change in market share
cost factors elementary cost structure
capacity utilizations of space capacity utilizations of machinery capacity utilizations of employees condition of space
condition of machinery motivation of employees internal
production factors
time taken by activities cost of activities efficiency of activities
quality of activities product profitability products & customer
profitability customer profitability
customer satisfaction with normal products customer satisfaction with special products resources spent on new product development properties of products
number of new products or variations production environmental effects use of products besides other factors Discussion
The dimensions of the two models are either divided into results/ determinants or external/ internal. For this project it is decided to follow the categorizing of Fitzgerald et al. (1991) because following the determinants-results logic it is easier to link the strategic objectives by strategic mapping.
Thus, the 6 main PM perspectives for Quattro are: two in result category - financial performance and competitiveness; four in determinant category - quality of service, effectiveness of activities, product & customer profitability and innovation.
4 Problems of the Current Quattro Performance Measuring System 4.1 The current Performance Measuring System
The current performance measuring system (PMS) of Quattro is formed by two management reports of financial indicators and one newly developed "KPI combined" report of both financial and non-financial indicators. Altogether, the three reports has 70 performance indicators. The two management reports are presented on Google Spreadsheets for monitoring the financial performances. One called Consolidated Master Framework monthly reports summaries of income, direct cost, expense and net income of the headquarter, the two subsidiaries and the consolidated result. It also reports the monthly sales of all national franchisees in different countries and the corresponding monthly growth. Another called Consolidated Master Budget vs. Actual quarterly compares the percentage differences of income, direct cost, expense, EBIT as percentage of income and net income between budgets and actual results of the three companies as well as consolidated result. It also provides more detailed information on the actual accruals of components of income, direct cost and expense of all three companies of Quattro.
The new "KPI combined" report includes eight categories of KPI sets with each having several sub-sets (see Appendices 2). The indicators are presented on Google Spreadsheet and shared as on-line documents among top managers and people who work on the reporting. It also includes a manual about the contents of the Google spreadsheet and how this spreadsheet is created. In this document, KPI is defined as “a measure of performance, it can help an organization to define and evaluate how successful it is and they can use it to make progress towards the long-term goals”.8
4.2 Diagnosing the Problems
Whether the current PMS is effective and what are the problems can be diagnosed by comparing it to the criteria of effective PMSs developed earlier in Section 3.1 and problems described by organizational members during interviews.
Are the measures designed basing on strategic objectives of the company?
Although the current PMS is based on the strategic descriptions and management intentions of the company, the bigger chance is that it is not closely connected to a specific set of strategic
objectives. In fact, the company has not developed such a set of strategic objectives yet. Some strategy statements can be found scattered in previous research works of the company such as market and SWOT analysis and company strategic directions and road-maps. But due to the managers of the company, it has never had any mission/ vision statements, nor has it analyzed the crucial factors to its future success. By the words of the managers themselves, "everything is still in our minds".
Are the measures providing performance information from multiple perspectives important for success of the company?
6 main perspectives necessary for Quattro performance measurement are selected earlier in Section 3.3. This question can be answered by checking are there indicators measuring
performance from each perspective. All measures in the management reports can be considered as measures from the perspective of financial performance. The number of accounts and sign-up can be considered as measures from the perspective of competitiveness. Number of cancellations and customer churn rate can be considered to reflect performance from the perspective of service quality. Cost of marketing and cost per action (CPA) for trials and sign-ups can be considered as measures for efficiency of marketing, and direct sales made can be considered as measure for efficiency of sales, both belonging to the perspective of efficiency of activities. Number of sends per account, sales per account and sales per sends can be considered measures from the
perspective of product & customer profitability. There is still one perspective without any measures – innovation. Moreover, from the perspective of efficiency of activities, there are only measures for marketing and sales efficiency but not other important activities such as product development. Also, The numbers of measures along each dimension are quite imbalanced. Are departmental measures included and vary between different department? Can the departments evaluated control those measures?
the performance of product development department and only sales as indicators for the
performance of sales & support department. Managers not being able to monitor the performance of main departments with largest number of staff caused significant problem for control of the organization.
Can the indicators tell both performance results and directions for performance improvement? Most indicators in the current PMS are result indicators. For the system to show what the barriers for reaching objectives are and how to improve, first the relationships of the objectives need to shown.
Is the system simple and easy-to-use?
The reports are relatively easy to use in the way that they include both ratio indicators and absolute numbers, they also provide graphs showing trends of and contributions to some indicators which allow comparisons. However, one important reason hindering the system from being simple and easy to use is the number of indicators. There are altogether 70 indicators all at organizational level and supposed to be interpreted and monitored by top management for evaluating the performances of the company. Reviewing all of them everyday and trying to find problems requires too much time so there are indicators that the managers never pay attention to. Some of the indicators are measuring from the same theme and their functions are redundant. Are the measures designed through discussing with people involved?
When the measures are selected for the current PMS, only CEO, CFO, project manager and one software developer from the company are involved. Most of the departmental managers and other staff are not aware of the existence of the system so there is no way that their opinions are considered.
4.3 Conclusion
Dealing with the problems identified by checking the Quattro PMS against 6 criteria of effectiveness, the following improvement need to be made to the design of the system:
perspectives selected in Section 3.3 for the measures to evaluate performance from all important perspectives.
2) The improvement of the design must be participated by people involved, that is, all members of the company. All employees should be given access to the reports. 3) The number of organizational level indicators must be reduced to fewer than 20.
Departmental objectives must be developed and departmental measures designed. The reporting and monitoring of the departmental measures should be under control of the department themselves.
5 Improving the Design of the Performance Measuring System 5.1 Identifying Strategic Objectives
Following the pre-determined process, identifying strategic objectives starts with clarifying mission/ visions/ values of the company, goes through crucial success factor (CSF) identification and end up with a map containing various interrelated strategic objectives from 6 PM
perspectives.
5.1.1 Mission/ vision/ value of Quattro
As a starting point of strategic objectives defining, the mission/ visions/ values of the company need to be clearly defined first. Since the company has no such statements yet, discussions with the owners/ managers of the company are initiated. Concluding from statements of them, the mission of Quattro is to “solve real-world e-marketing problems of SMEs and individuals all over
the world”.
The visions of the company are:
1) Making the products of the company internationally known by millions of people; 2) Being top 10 by revenue and market share in email/mobile marketing market. And the values of the company include:
3) Always being reliable;
4) Providing relaxed working environment to employees;
5) Providing franchisees profitable and customer-winning opportunities. 5.1.2 Crucial Success Factors
CSFs are identified from two sources: reviewing company strategic documents, mainly the SWOT analysis9 and investigations into the aspect influencing the success of the company during interviews.
Outcomes of SWOT analysis
A SWOT analysis has been done by CEO of Quattro last year and the results are shown in Appendices 3. Some strategic indications can be generated from those results of analysis.
The first is that efficiency of sales channels, especially franchising, greatly influence sales growth and global competitiveness. Quattro has multiple sales channels including direct sales,
franchising, licensing and sales through resellers/ agents. This is considered one advantage. But arbitrary selection of franchisees, too weak in many countries and non-effective direct and licensing sales are slowing down the growth and internationalization of the company. Enhancing sales channel management could provide opportunities for further achievements of the company. Thereby the PMS need to provide enough information for monitoring efficiency of sales
channels.
The second one is that appealing products are crucial for the competitiveness of the company. The strength of the products of Quattro is that it is user-friendly with good interface and APIs (application programming interfaces), and applicable internationally. The development of the products is fast and with lower cost. But because of launching features too fast without enough testing, their products are sometimes not reliable enough. Also, there are opportunities to increase the attractiveness of their products from increasing the ability of integration by developing more plug-ins for open sources and other platforms. Thereby the PMS need to provide information not only for monitoring the speed and cost of product development, but also the quality and
properness of functions.
The third one is that the company still need a lot of efforts to increase the customer awareness of their products through marketing. Quattro has a forward thinking marketing team with knowledge of all networking channels. But still too small amount of people know about them. There’s a opportunity to increase the sales and global competitiveness through increasing their awareness towards more clients, resellers and potential franchisees, maybe through more marketing channels. Thereby from the perspective of efficiency of marketing activities, the PMS need to provide information indicating the level or change of customer awareness.
Other strategic indications from the SWOT analysis are: the importance of HR aspect and organizational culture – Quattro has its strength in freedom and global citizenship of employees, lean and mean organization and being open to suggestions and improvements, and it wants to continue benefiting from this advantage; the importance of improving communication – one weakness of the company is lack of understanding between different parts of the company, and too many communication channels are not helping with the problem, instead they sometimes confuse the employees; importance of innovation and learning – internally, the company is able to benefit from realizing weakness and dealing with them, and externally they need to always keep up with technology. The importance of support is also mentioned in this analysis, but only as a general factor, not providing detailed strategic directions.
Interview Results
The indications of the CSFs are from the replies to questions about what are perceived to be important to success of the company/ the departments (Question 10 in Appendices 1), factors influencing successful accomplishment of tasks and difficulties/ barriers for fulfilling
responsibilities (Question 5 in Appendices 1). The following factors shown in Table 3 are the most frequently mentioned factors by respondents.
Table 3 Most Frequently Mentioned Success Factors from Interviews
Factors Times mentioned as
influential factors to task accomplishment
Times mentioned as important factor to company/ department success
Efficient communication 10 18
Quality and speed of development
10 16
Gaining and retaining customer satisfaction
4 15
Increasing sales and market share
- 11
Good management - 5
Innovation - 5
The factor of efficient communication is mentioned 10 times in influential factor/ difficulty/ barrier descriptions and 18 times as an important factor for company/ department success. The problems related to this factor are: people ignoring what’s already decided and written out there makes the previous efforts in vain; insufficient internal or external information about what’s in need and what problems need to be solved cause lack of directions of support, development and design works and make them more time-consuming; misunderstandings cause negative feelings between people in different departments, which is most obvious among members in development and support teams; insufficient communication makes things less organized and everyone starts working basing on their own ideas, which causes conflictions and delays of delivery. This factor is important to the success of the company because most of the activities in the company need cooperation and interactions of people from different departments aiming at the same goal. Efficient communication is the base of team-work. It is also because better understanding of problems need to be solved or outputs need to be delivered make accomplishment of tasks better and faster while too many sources of incoming information distract employees from the most important tasks and cause ignorance.
The factor of quality and speed of development is mentioned 10 times in influential factor/ difficulty/ barrier descriptions and 16 times as an important factor for company/ department success. The difficulties related to quality of development are from both external and internal aspects. Externally, frequent occurring of bugs of products cause troubles for customers and support service, for the company keeps developing and launching new features without enough testing while not making enough efforts to fix the existing problems. Internally, the database and tools developed for designing and customer service are said to be out-of-date, difficult to use, not flexible enough and sometimes not providing accurate information. This factor is important to the success of the company because stable, reliable and user-friendly products are crucial for
customer satisfaction and earning the company more customers. Staying above the standard of industry in quality could make the products of the company more competitive in market. Speed of development work directly influence the respond time of customer service and level of customer satisfaction. And it is indicated by employees with much customer communications that
The factor of gaining and retaining customer satisfaction is mentioned 4 times in influential factor/ difficulty/ barrier descriptions and 15 times as an important factor for company/
department success. The problems related to this factor are that sometimes customer opinions are not taken seriously for improvement of products and service; supports sometimes disappoint customers because too much time is in need to get back to customers and delayed service. As a success factor, being able to gain and retain customer satisfaction makes sure that users and triers do not cancel their accounts so that the company can have loyal customers and better reputation to gain more customers. According to respondents, customers need to be kept happy through: relevance of products and features, good value comparing to price, guidance through products, communicative and responsive support staff ready to help on time when needed, and improved paying system being more reliable and easier to manage customer accounts.
The factor of increasing sales and market share is mentioned 11 times as an important factor for company/ department success. The importance of this factor to success is the most obvious. Not too many people have awareness of the company and its products; increasing revenue lead to financial success and authority in the market.
The factor of human resource management is mentioned 10 times as an important factor for company/ department success. On one hand the company needs sufficient number of staff with proper skills and right attitude to devote to their jobs. On the other hand the management need to maintain staff satisfaction by keeping in touch with employees, providing good security and welfare, relaxed working environment and opportunities for employee development, as well as enough guidance and feedbacks.
The factors of innovation and good management are both mentioned 5 times as important factors for company/ department success. As for innovation, keeping up with the latest trends of industry and keeping new products and features on top of competitors are thought to be necessary. As for good management, proper planning and budgeting, good project management and leadership are thought to be necessary.
5.1.3 Mapping Organizational Strategic Objectives
Figure 1 Quattro Strategic Map
5.2 Selecting Organizational Performance Measures
There are 17 inter-related strategic objectives from 6 perspectives in the strategic map. One indicator considered most representative for each objective is selected. The indicators selected are shown in Table 4.
Table 4 Quattro Organizational Level Strategic Objectives and Indicators
Perspectives Objectives Indicators
Financial performance
Increase revenue Revenue growth Enhance market position Market share Competitiveness
Increase global awareness Growth of website visits Increase profitability of unit
product
Sales per million sends Cost reduction Cost per million sends Product & customer
profitability
Increase unit customer spend Sends per customer account Gain & maintain customer
satisfaction
Customer retention rate
Improve quality of products Number of complains on product familiars
Quality of service
Improve quality of customer support
Average customer rating of support chats
Increase efficiency of marketing
Return on investment (ROI) of Marketing
Use efficient marketing channels
Average cost per action (CPA) Improve availability/ speed of
support
Average turn-around time of helpdesk tickets
Improve quality/ speed of development
Average fixing time of product failure Efficiency of
activities
Enhance management of sales channels
% misalignment of sales between GraphicMail and franchisee accounting Improve relevance of
products/ features
Number of new features and adaptations developed Innovation
Improve customer support/ management system
% misalignment of send credit sold between customer account management system and accounting system
5.3 Designing Departmental Performance Measures
The design of departmental performance measures follows the processes of understanding functions and contributions to main activities of departments; setting objectives to main departments; and select indicators for departmental objectives.
5.3.1 Understanding Functions and Contributions of Departments
By understanding the company about the functions and the role of each part of the organization and how they are involved in main activities, it can be indicated that how the departments can contribute to achieving the company strategic objectives and which performances of them are crucial to better performing the main activities. Information about the formation of the departments, the formal structure of the company and report line can be found in company documents. Detailed functions/ tasks of the departments and how they are involved in main activities are investigated by interviews and observations.
As mentioned in Section 1.1, Quattro has only 35 employees and a simple and flat organizational structure. An organizational chart with 6 departments under the leadership of 3 top managers are shown in their company document.10 But this structure does not exactly fit the reality because recently a project manager has been promoted from the department of accounting and
administration and he now has more power and responsibilities. An actual organization structure is shown in Figure 2. The detailed functions of the departments and the actual number of staff
working in those departments are summarized in Appendices 4 from managers' and employees' descriptions of their tasks.
Figure 2 Organizational Structure of Quattro
However, from interviews it can be observed that this structurally simple organization formed a relatively complicated network with a lot of necessary internal communications among
organizational parts and external communication with outsiders for keeping the business running (see Figure 3).
From the organizational network and information flows as well as the function descriptions and the time employees spent on their tasks implied by interviews and observation, it can be concluded that there are 4 main activities in Quattro: product development, customer service, marketing and sales. First, as can be seen from Figure 3, beside management team, there are three centers of communication: development, sales and support, and marketing and
communication. Also, there are relatively more people working in the three departments, whose main responsibilities and most of working time spent on tasks can be summarized as product development, customer service, marketing and sales separately. Moreover, most work of departments other than the three can be seen as supports for the work of the 4 activities, e.g. the department of design spend much of their time designing marketing materials for marketing and visual design for product development; while the department of network administration provide hardware support for customer support and backup support for product development. Besides, most of management attentions also pay to the 4 activities. Therefore, level of success of the 4 activities – product development, customer service, marketing and sales directly influence the level of success of the company.
Each of the 4 activities needs the contributions and cooperation of several departments besides the coordination and overseeing of project manager and top managers. The departments involved in the three activities and their contributions are shown in Table 5, 6, 7 and 8.
Table 5 Departments Involved in Product Development and Their Contributions Departments involved Contributions
programming for new products, applications and features testing, de-bugging and modifications
Development
involved in realistically assessment
communicating customer and franchisee feedbacks for further development
Sales and support
testing products/ features and report bugs to development team Design designing visual part of products, applications and features Network administration backup support
Accounting and administration
managing development project
coming up with new ideas for products and features Management
assigning tasks and setting timelines
Table 6 Departments Involved in Customer Service and Their Contributions Departments involved Contributions
assisting customer with product use Sales and support
customer account management
providing site-based support - webinars, FAQs, videos, etc. informing customer about new products, features and changes informing support staff about new development
answering questions new features from support staff and customers Development
solving customer problems, fixing bugs surveys for customer feedback
Marketing and
communication providing market trends and statistics to support staff
answering questions and solving problems from support staff or customers
Network administration
handling abuses
Design designing visual part of site-based support and back-in interface taking care of debit orders
Accounting and
administration invoicing customers
Table 7 Departments Involved in Marketing and Their Contributions Departments involved Contributions
SEO Advertising
initiating and managing promotions press release - blogs, social medias, etc marketing research
Marketing and communication
surveys
designing website Design
designing marketing materials cooperate with promotions Sales and support
involved in promotions
handling on-site issues for marketing programming marketing features Development
answering questions about promotions coming up with new marketing ideas Management
identifying new markets, new customers and franchisees Table 8 Departments Involved in Sales and Their Contribution
Departments involved Contributions sales
replying sales enquiries settling send credits sold Sales and support
handling on-site issues for marketing
initiating and manage promotions to increase sales Marketing and communication
support communications with sales channels identifying sales channels
Management
5.3.2 Setting Departmental Objectives and Selecting Measures for Departments
Knowing the functions and their contributions to main activities in the organization, it is possible to determine how the departments can contribute to the realization of organizational level objectives by setting and achieving their own departmental objectives. Fewer than 20 indicators are selected for each departments. The objectives settings and measures selected for the
objectives are shown in Table 9. Their measures are selected basing on interview investigations to departmental members about what best stand for their performance and to managers what they are willing to know about the departmental performances.
Table 9 Departmental Level Objectives and Indicators Organizational level
objectives
Departmental objectives Indicators Marketing & communication
New sign-ups from each marketing channel Return on investment (ROI) of each channel Use efficient
marketing channels
Monitor efficiency of various marketing
channels Cost per action (CPA) of each marketing channel
Number of customer accounts Enhance market
position
Increase customer base
Growth of customer accounts Attract more potential
customers to try products
Number of new trials Increase global
awareness
Attract more visitors to company website
Number of visits to website Marketing cost
Cost reduction Control marketing cost
% variations between marketing cost and budgeting
Monitor cost of each marketing channel
Spent on each marketing channel Sales & support
Sales Increase sales
Sales growth Increase revenue
Increase customer base Number of customer accounts Number of support phone calls Number of helpdesk tickets Gain & maintain
customer satisfaction
Solving customer problems
Number of service chats Improve quality of
customer support
Increase readiness and friendliness to customers
Number of customer complains Number of new webinars Number of new tutorials Support customer through
various ways
Number of new FAQs and Helps Average respond time on ACE tasks Improve availability/
speed of support
Shorten support
Improve quality of products
Enhance testing and reduce number of bugs
Number of bug detected Increase reliability of
products
Fix existing bugs Number of bug fixed Improve availability/
speed of support
Fasten response to development-related customer support
Average turn-around time of support ACE task
Improve quality/ speed of development
Accomplish development tasks in due time
Rate finishing ACE tasks on time Number of adaptations made Increase adaptiveness of
products Number of applications developed Improve relevance of
products/ features
Keep up with up-dating product features
Number of new features launched Improve customer
support/ management system
Increase reliability of SiteAdmin system
misalignment of send credit sold between customer account management system and accounting system
Accounting & administration
Amount of receivables aging 30/60/90 days Bad debt ratio
Increase revenue Enhance credit
management and reduce
bad debt Average collection period Monitoring profitability of
each product type
Sales made by each type product Sales made to each customer group Increase profitability
of unit product
Monitoring profitability of
each customer group Sends of each customer group Staff cost per million sends Cost reduction Reduce operating expense
% expense to total expenses Franchisee sales ranking Sales growth of each franchisee Enhance management
of sales channels
Enhancing control over franchisees
Misalignment of send credit sold between customer account management system and accounting system Design Improve relevance of products/ features Increase attractiveness of newsletter templates
Number of new templates designed Average times of new templates used by customers
Network administration
Number of spam complains replied Improve quality of
customer support
Enhance spam
management Number of spam complains per million sends Improve availability/
speed of support
Improve responsiveness to network related support
Average turn-around time of net-work related support tasks
6 Conclusions and Discussions 6.1 Conclusions
measuring system (PMS) of Quattro to better guide the company towards successful achievements of its strategic objectives. Aiming at this purpose, the study followed a three-phase process of defining effectiveness of PMS, identifying the problems of the current system and improving the design of measures. During the process of diagnose and improvement, evidences from literature review, company document study and interviews to organizational members are used. Although effectiveness of an PMS not only depends on its design but also the implementation and continuous evolution of it, without the proper design at the first place the system can never be effective. And the purpose of the design is to fulfill the measuring function by using easy-to-use but representative measures to the right sets of interrelated objectives at different levels of the organization.
6.2 Limitations
One important limitation of this research is that both the current PMS and the improved one are flawed in the inability to monitor effectiveness of communication and human resource
management, which are crucial for the success of the company. Due to the complicated nature of the two activities, the evaluation of their effectiveness need to be done with interviews or surveys, which can only be done once or twice per year. However, the indicators in the PMS are up-dated daily or monthly, which is far more frequently than available data representing the effectiveness of the two activities could be collected. Other projects, such as prospering intranet and bringing people from different departments together, are already initiated by the company in compensation to this one to evaluate and improve communication and staff satisfaction.
Another limitation is from sampling and responding. Because of limited research time and accessibility, interviews are only done to organizational members of Quattro. The opinions of customers and franchisees could also be meaningful for the improvement of the system. For the staff overseas, answering written questions does not provide equally satisfying results to face-to-face interviews and several of them did not reply for unknown reasons.
6.3 Potential Difficulties for Implementation and Suggestions for Further Work
Several potential difficulties and barriers could appear during the process of implementation of the improvement plan to the PMS. Some suggestions are presented in this section for further implementation and improvement of the system.
Second, although it is proposed that everyone in the organization should have access to the PMS, it might take some time before all staff of the company can fully understand and actually use the system. Negative feelings might appear because of the introduction of some new indicators into the performance measurement. E.g. the reporting of number of bugs detected and how much time it takes them to fix might not be welcomed by the staff in development department because it gives them extra pressure of development quality to the current pressure caused mainly by deadlines. And not everyone will be interested in how other departments are doing and what is the overall situation of the company but only caring about doing their own job, although it is important for team-work and company culture development. Communications to staff about the importance and functions of the PMS are suggested.
Third, constant efforts will be in need to keep record for some of the indicators to be reported. This is more obvious when evaluating the efficiency of customer service because some measures, such as service phone calls need to be manually counted and some measures, such as customer complains are subjective to customer opinions. Records of phone calls and customer complains need to be kept for those measures to be reported justly.
Fourth, for franchisee-related performance measuring, Quattro only has limited control because franchisees have their own internet domain in their national language, do their own statistics and accounting. For their performances to be reported, effective communications to national
franchisees and extra regulations in contracts are in need.
Reference
Books and Articles
Bourne M., Mills J., Wilcox M., Neely A. & Platts K. 2000. Designing, implementing and updating performance measurement systems. International Journal of Operation & Production Management, 20(7): 754-771.
Dickinson R., Ferguson C. & Sircar S. 1984. Crucial success factors and small business. American Journal of Small Business, 3: 49-57.
Drèze X. 2005. Lessons from the front line: two key ways in which the internet has changed marketing forever. Applied Stochastic Models in Business & Industry, 21(4/5): 443-448. Flapper S., Fortuin L. & Stoop P. 1996. Towards consistent performance management systems. International Journal of Operations & Production Management, 16(7): 27-37.
Fitzgerald L., Johnston R., Brignall S., Silvestro R. & Voss, C. 1991. Performance Measurement in Service Business, London: CIMA.
Globerson S. 1985. Issues in developing a performance criteria system for an organization, International Journal of Production Research, 23(4): 639-646.
Godin S. 1999. Permission Marketing: Turning Strangers into Friends, and Friends into Customers. New York: Simon & Schuster.
Hacker M & Brotherton P. 1998. Designing and installing effective performance measurement systems. IIE Solutions, 30(8): 18-23.
Heit E. 2000. Properties of inductive reasoning. Psychonomic Bulletin & Review, 7(4): 569-592. Heron J. & Reason P. 2001. The practice of co-operative inquiry: Research “with” rather than “on” people. Handbook of Action Research – Participative inquiry and practice, Reason P & Bradbury H. (Edt.), 2001, London: Sage.
