Decision criteria for purchasing professional services

(1)

1

Decision criteria for purchasing

professional services

What about the public sector?

P.W. (Pim) Hoogers

(2)

2

Decision criteria for purchasing

professional services

What about the public sector?

Master Thesis

Msc Business Administration

Specialization Marketing (Management)

August 2011

P.W. (Pim) Hoogers S1334727 Hilsdijk 86 8051 KE Hattem The Netherlands pim.hoogers@gmail.com (+31) (0)6 22743353 University of Groningen

Faculty of Economics and Business Department of Marketing

Supervisor: drs. J. (Hans) Berger

Co-assessor: prof. dr. J.C. (Janny) Hoekstra

(3)

3

Management Summary

Having a peek in a potential customer’s mind, understanding the purchasing process and knowing which decision criteria they use for selecting a professional services provider can mean the difference between winning an assignment and losing it.

Literature on purchasing professional services is thin, and on the decision criteria for purchasing them even thinner. The studies that have examined the decision criteria for purchasing professional services have almost exclusively used the private sector as subjects, ignoring possible differences with the public sector. Furthermore, they mostly did their research in consulting, architecture and legal services settings and their number of criteria were often limited. This study therefore examined the decision criteria used by the public sector, in an education/training setting and used an extensive set of criteria. Also, unlike in any of the published articles so far, cluster analysis was used to identify clusters.

A mail survey was sent out to 455 public sector heads of departments (covering the complete decentralized government) in The Netherlands, asking them about the decision criteria for selecting a provider from the shortlist the last time they purchased education/training. The response rate was 44 percent.

The results indicated that speaking the professional language of the participants is the most important decision criterion, followed by customization, knowledge of and experience in the client’s industry, using real world experience/anecdotes and prior successful experience with the provider.

Where previous studies found reputation to be of (very) high importance, this study found it to be of only medium importance. A possible reason for the lower importance could be that the public sector has a higher risk threshold than the private sector and therefore engages less in risk reducing activities (e.g. looking for a provider with very good reputation).

(4)

4 When applying the knowledge gained from a professional services study in a particular setting in a different setting than the original, caution is always advised. The same goes for this study. The results from this study, in an education/training setting, can fairly easy be used for other more recently developed professional services such as management consulting and advertisement services. This study does contain decision criteria tailored for the education/training setting which are not useful in other settings. However, the importance of most of the higher ranking criteria like speaking the professional language, customization, knowledge/experience in the client’s industry, prior successful experience with the provider can probably be transferred without much difficulty to the more traditional professional services like accountancy, architecture and legal services, as well the more recently developed professional services. Also, for the criteria covering price, reputation, effectiveness of the training/solution or evidence of successful service delivery (in the past) it is likely that they have a high degree of applicability to the professional services mentioned above.

(5)

5

Preface

In front of you lies the end result of a research into the decision criteria for purchasing professional services in the public sector, my Master Thesis. It is the final part of my long but enjoyable period as a student at the University of Groningen.

After working for almost a year and a half it was not easy getting back into ‘study mode’ and start working on this thesis. Nevertheless, I have enjoyed working on it and am pleased with the final product. Hopefully it is also pleasant to read.

I would like to use this opportunity to thank several people who have contributed to this study. First of all I would like thank the marketing team of the Politieacademie for their support in this study and giving me all the freedom to work on this project.

I want to thank my thesis supervisor Hans Berger for his comments and suggestions while working on this research.

A special thank you goes out to my parents. Thank you for being patient all those years and all your financial support!

Finally, I would like to thank my girlfriend Lidija. We have been through uncertain times together during the last year and a half but things have worked out fine. Thank you for never giving up!

(6)

6

1. Introduction

1.1 Professional services procurement and decision criteria

Within a country like The Netherlands up to 80 percent of the economic transaction volume happens in a business-to-business environment (Lichtenthal & Mummelaneni, 2009). As Western economies became more and more focused on services instead of traditional production, researchers have also partially shifted their attention towards this subject, including professional services.

Already in 1977 Kotler & Conner Jr. (1977) emphasized the importance of distinguishing the marketing of professional services. In their paper they elaborated on several strategies for professional firms to increase marketing effectiveness.

Despite limited attention for B2B marketing in mainstream marketing journals, where only 6,7 percent of the articles concern B2B (LaPlaca & Katrichis, 2009), a vast amount of papers have been written on the subject of marketing professional services since the 1970’s. The topics vary from the role of marketing within UK accounting and law firms (Morgan, 1990) to creating and appropriating value (Wagner et al., 2010). In many cases there is a strong focus on relationship marketing. However, the subject of purchasing professional services has attracted relatively limited academic attention in comparison to the level of research into physical and industrial products (Howden & Pressey, 2008; Lian & Laing, 2007; Reid, 2008; Stock & Zinszer, 1987). Carter & Ellram (2003) quantified this difference in academic attention and found that in 35 years of publishing, only 10 out of the 774 articles in the

Journal of Supply Chain Management were about purchasing services.

Also, professional service firms themselves spend relatively little resources on (customer) knowledge development (Fosstenløkken et al. 2003), which is somewhat ironic as Nätti & Ojasalo (2008) rightly point out, since knowledge is the foundation of professional service firms.

(9)

9 Knowing which criteria purchasers of professional services use is very important within the purchasing process. It provides greater understanding of the customer’s considerations when faced with a purchasing dilemma. With this understanding, companies can (re)design services which meet these criteria or target (potential) customers with services which score high on the important criteria.

Knowledge of the decision criteria can also be exploited in marketing communications. Emphasizing characteristics which are high on the criteria list may persuade (potential) customers to purchase your services and prevents you from communicating issues which are of relative unimportance.

Furthermore, adding unimportant characteristics or features to your services is likely to drive up costs without adding any value for customers.

Up to now only a handful of studies have focused on the decision criteria for purchasing professional services (Saleh & Sarkar, 1973; Stock & Zinszer, 1987; Dawes et al., 1992; Sonmez & Moorhouse, 2010). Out of that handful of studies, only Dawes et al. (1992) included the public sector in their sample.

1.2 Problem statement

As a result of the issues mentioned above the main research question for this study is: ‘Which

decision criteria are used by the public sector for purchasing professional services?’

Going further than any of the published articles so far, this study includes cluster and discriminant analyses to identify clusters of respondents/criteria. Other relations between identification variables and decision criteria will be investigated as well.

(10)

10 This study was conducted in cooperation with the Police Academy of the Netherlands (from now on Politieacademie) to strengthen their knowledge about (potential) customers.

The research provides a valuable contribution to the very scarce academic knowledge about decision criteria used by the public sector for purchasing professional services. Also by using an education and training setting in contrast to accounting, consulting or legal services which are often used for professional services research, this study contributes to the professional service and purchasing literature in general.

Since the research is (incrementally) original and has scientific and practical utility, according to Corley & Gioia (2011) this study can be considered as a theoretical contribution.

1.3 Structure

After the introduction, a brief overview of the Politieacademie will be presented in chapter 2, including its activities and competitive environment. Chapter 3 contains a review of the literature relevant to this study, followed by the methodology of this study in chapter 4. The results and their discussion are covered in chapter 5 and the conclusion makes up chapter 6. Limitations of this study and suggestions for further research can be found in chapter 7.

(11)

11

2. About the Politieacademie

1

The Politieacademie is the central organization responsible for recruitment, selection, education and knowledge development & management for the police profession in the Netherlands.

The Politieacademie is integrated in the Dutch police system and falls under the responsibility of the Ministry of Security and Justice. The President of the Executive Board of the Politieacademie reports directly to the Minister of Security and Justice and the State Secretary of Security and Justice.

The Politieacademie has three main responsibilities: recruitment and selection, education and knowledge management. Each of these responsibilities is a directorate within the larger organization. Supporting tasks and capabilities are centralized in two supporting directorates: Management & Corporate Staff (for policy support, e.g. communication & marketing, control, strategy) and the Corporate Support Services (for operational support, e.g. HR, housing, purchasing).

The Executive Board consists of three people and each directorate is led by a director who is supported by the heads of the individual departments. Figure 2.1 shows a summarized version the organizational chart.

Each of three main responsibilities of the Politieacademie is shortly discussed below:

2.1 Recruitment and Selection

The Politieacademie executes the national recruitment and selection program for all Dutch police forces. This means that everyone who wishes to be a police officer in the Netherlands has to be recruited and selected by the Politieacademie. The selections consist of psychological, physical, language and intelligence tests.

The Politieacademie also conducts assessments for current police officers and employees who wish to grow within the police organization.

1

This complete section is merely intended to improve the overall readability of this Thesis by providing a brief overview of the organization for which this study was conducted. This section is based on information provided by the Politieacademie and all the information can be publicly accessed at www.politieacademie.nl and

(12)

12

2.2 Education

The Politieacademie offers non-scientific professional education and scientific education at Bachelor and Master level (accredited in the United Kingdom, the Netherlands and Belgium). Education is divided in initial (for new police officers) and post-initial (for current police officers) education and is provided by six schools, all with their own specialty (e.g. crisis and risk management, criminal investigation or enforcement) at ten locations across the Netherlands.

2.3 Research, Knowledge & Development

The Politieacademie promotes knowledge development and actively develops knowledge of and for the policing profession. Several knowledge services are offered to the police forces and they produce (applied) research on police and public safety issues. The research topics are organized around thirteen lectureships.

2.4 The Politieacademie in figure and numbers (in 2010)

- 1.705 employees

- Over 15.000 applicants for a function within the Dutch police system - 5.015 standard selections executed

- 1.559 students started an initial education program - 8.138 post-initial education students

(13)

13

2.5 Competitive environment

The Politieacademie has a monopoly position for recruiting, selecting and training new police officers. However, when the initial training ends, so does the monopoly. The Politieacademie then has to compete with (semi) private organizations for providing assessments, training and knowledge services. For the recruitment and selection of civilian functions within the Dutch police system there is also no monopoly position.

This non-monopoly position also allows the Politieacademie to sell their services to other organizations beyond the traditional police market. Obvious potential customers include the decentralized governments (discussed in section 4.1), ministries and independent government bodies. The total education and training market for decentralized governments in the Netherlands is estimated to be worth 187 million Euro’s and the estimation for the central government reaches a similar amount (Binnenlands Bestuur, 2011).

(14)

14

3. Literature review

3.1 Professional services

In order to understand the decision criteria of professional service firms (PSFs) it is useful to define what a professional service firm is. Over the last decades many authors have written about PSFs and the services they provide, however their characteristics and definitions have been very different.

In their 1969 article McDonald & Stromberger recognize accountants/auditors/bookkeepers as professional service providers as well as advertising agencies, business and management consultants, engineering and architectural firms and companies that provide legal services. This is far less than the whole range of advisory activities Wittreich (1966) identified as professional services three years earlier. He considered financial, economic, public relations, advertising, legal, personnel, research and many other advisory activities as professional services.

A more systematic approach was attempted by Gumesson (1978), using characteristics instead of examples. His characteristics of ‘qualified personnel’, ‘advisory nature’, ‘problem solving’, ‘assignment from buyer to seller’ and ‘professional independence of suppliers of other services and goods’ leave space for including more services than McDonald & Stromberger (1969) do and seems to match with Wittreich’s (1966) examples.

This difference in narrow versus wide scope of what professional services are has been reoccurring in literature. Abbot’s (1991) remark that professional services are characterized by the existence of significant entrance barriers in the form of exams and licensing seems to be more in line with the narrow scope of McDonald & Stromberger (1969).

However, Lian & Laing (2007) state that professional services are collectively characterized by a high degree of intangibility, comprise non-standardized offerings and involve a high degree of interaction between the supplying organization and the purchaser.

(15)

15 professional services. These characteristics seem to favor the wider scope of Wittreich (1966) and Gumesson (1978).

A different way of defining professional services was done by Bloom (1984) who describes professional services as what economists sometimes call “credence goods”, where the purchaser has to place great faith in the hands of the PSF. His described lack of attributes with which a buyer can confidently and completely evaluate the service before -or even after- making the purchase decision was already recognized by Wittreich (1966) and is also mentioned by Day & Barksdale Jr. (1994, 2003), Howden & Pressey (2008) and Wuyts et al. (2009). In many cases the service provider has to determine the needs of the customer, further increasing the perceived risk from the buyer’s perspective.

Acknowledging the vast amount of definitions and characteristics used by researchers, Von Nordenflycht (2010) developed a systematic theory about what is and what is not a professional service firm, based on a review of PSF literature. Instead of describing their general characteristics, Von Nordenflycht uses distinct characteristics of PSFs to discriminate them. His theory recognizes three important characteristics: ‘knowledge intensity’ (i.e. human capital intensity), ‘low capital intensity’ and a ‘professionalized workforce’. A PSF does not need to meet all three characteristics in order to be qualified as a PSF and Von Nordenflycht (2010) categorizes them in four different groups based on their shared characteristics. The common characteristic however is always knowledge intensity. This also seems to be the dominant and only widely acknowledged characteristic for defining PSFs.

Von Nordenflycht’s taxonomy and examples can be found in table 3.1. The Politieacademie would be categorized as a neo-PSF in this model.

A major advantage of this systematic approach is the way it allows studies to be assessed for applicability. In their studies Thakor & Kumar (2000) and Howden & Pressey (2008) warn for using all studies on professional services in different professional service industries. What might be true for one category of PSFs might be different for another since their characteristics are different. For example, using the outcomes of a study for accounting services has a high likelihood of not being applicable for biotechnology firms.

(16)

16 and engineering consulting) Malthotra & Morris (2009) found differences with regard to knowledge base, organizational form and structure, hierarchy culture, degree of social closure, jurisdictional boundaries, client interaction and pricing system.

These differences between PSFs in the same category raises serious questions about the degree of generalization of PSF research beyond the (sub-)sector in which it was conducted.

Characteristics Category (with examples) Knowledge

Intensity Low Capital Intensity Professionalized Workforce Technology Developers Biotech R&D labs X Neo-PSFs Consulting Advertising X X Professional Campuses Hospitals X X Classic PSFs (or Regulated PSFs) Law Accounting Architecture X X X

Table 3.1: Von Nordenflycht’s (2010) PSF categorization

Some authors include organizational form to define whether a firm is or is not a PSF. Mintzberg (1979) for example states that when professionals are active in a corporate organized company, one should not speak of a PSF but of a ‘professional bureaucracy’. In this manner the Politieacademie would not be considered a PSF, but since Von Nordenflycht’s (2010) model is used in this study, the Politieacademie does qualify as a PSF. Furthermore, Sonmez & Moorhouse (2010) also use an education/training setting for their PSF study and Nätti & Ojasalo (2008) explicitly recognize education as a professional service.

(17)

17 Although interesting and useful, a discussion about naming these types of organizations is not the aim of this paper. Since ‘PSF’ seems to be the dominant term used in (marketing) literature, the term is also used in this study for the type of organizations mentioned above. With regard to the discussion what is and what is not a PSF, the model in table 3.1 is preferred. It not only provides a systematic approach to what is and is not a PSF, it also allows identifying different type of PSFs .

3.2 Decision criteria

As mentioned in the introduction the subject of purchasing professional services has attracted relatively limited academic attention (Howden & Pressey, 2008; Lian & Laing, 2007; Reid, 2008; Saleh & Sarkar, 1973; Stock & Zinszer, 1987) and as a result little is known about the decision criteria being used or their individual importance.

The few studies that have been conducted on the purchase of professional services, and the decision criteria in particular, were quite different in their approach, number of criteria used, depth of their analysis and research setting.

The earliest work by Saleh & Sarkar (1973) focused on selection of consulting engineers by the public sector in the Province of Saskatchewan, Canada. Their thirteen variables were split up in two groups (competence factors and personality factors) and respondents were asked, amongst other things, to assign relative importance to the criteria on a 5-point scale.

Their criteria can be considered crude and vague. As a result, many their criteria like ‘competence’, ‘cooperation’, ‘civic reputation’ or ‘knowledge of local conditions’, are open to multiple interpretations. Nevertheless their research showed that experience, knowledge of local conditions, technical reputation to be important decision criteria.

(18)

18 A similar problem with limited generalization due to the technical focus of their research (architectural and engineering services) occurred with the study of Day & Barksdale Jr. (1992). Their qualitative research into the process of purchasing professional services yielded 20 variables, which they condensed into four underlying factors, but they did not investigate the importance of each variable. They labeled the four factors as ‘Experience, Expertise, Competence’, ‘Understanding Client’s Needs and Interests’, ‘Interaction, Relationship, Communications’ and ‘Contractual/Administrative Conformance’. These four factors by themselves offer limited help to managers as they contain up to eight variables per factor and is impossible for managers to focus on all 20 variables.

The research of Corcoran & McLean (1998) also does not place importance on the variables under study. They investigated how UK and Australian governments deal with selecting management consultant firms. They asked their 12-bodied sample if they use certain criteria and if they find it easy to use those criteria when selecting a management consultant. They did not ask how important they found the suggested criteria. All respondents answered they use all suggested criteria. Also, their eleven criteria and twelve respondents seem a bit meager for generalization. The limitations of their research are very unfortunate since it is one of the few studies dedicated to the purchase of professional services by the public sector.

Two studies which seem to be free from the limitations of the studies mentioned above are the studies of Stock & Zinszer (1987) and Dawes et al. (1992). They both produced a list of criteria which do not seem to be too specific, have an importance score for each criterion and both have large samples (N= 235 and 253 respectively). Stock & Zinszer (1987) used logistics consulting as object of their study and Dawes et al. (1992) used management consultants. The results of the two studies are quite similar. Prior experience (in the client’s industry), the consultant(s) working on the project and the reputation of the consultant or firm featured in the top 5 of importance in both studies.

Size of the firm, age of the firm, location and whether or not the firm offers a full range of services were found to be relatively unimportant and total costs for the service appeared to be of medium importance in both studies.

(19)

19 online survey amongst purchasers of professional services (marketing and sales training in their case).

In descending order of importance those high level criteria are 1) competence, 2) knowledge and understanding, 3) product, 4) reputation, 5) organizational capability and 6) cost. The first three high level criteria are considered final stage differentiators (making the difference in the final choice moment), the last three as pre-qualifiers (important for making it on to the shortlist or not).

The practical use of these high level criteria however is limited. They do seem to match with important criteria from earlier research (Dawes et al., 1992; Saleh & Sarkar, 1973; Stock & Zinszer, 1987) but since the authors do not further specify how they define, for example, ‘competence’ or ‘organizational capability’ these high level criteria remain vague and open to different interpretations. Also their relation to the 37 other criteria remains unclear.

The uncondensed 37 criteria (the complete list can be found in Appendix I) on the other hand provide useful insights in to the decision criteria of professional service buyers. Because they use considerably more criteria than previous studies they can be more specific and they provide a full description of each criterion. This provides more clarity and prevents (in most cases) the criteria to be open to multiple interpretations.

Sonmez & Moorhouse (2010) found 1) customization of the service/solution, 2) the ability to challenge current thinking, 3) bringing added value and 4) the ability to measure training effectiveness to be most important. Least important were 37) being the cheapest, 36) size of the company measured in revenue, 35) (book) publications in their area of expertise and 34) an indication of their top five clients by revenue.

From the fact that ‘being the cheapest’ was ranked lowest of all the criteria and ‘cost’ was ranked lowest of the high level criteria one might conclude that price does not seem to be of high relevance to buyers of professional services. This is somewhat surprising since earlier studies (Dawes et al., 1992; Saleh & Sarkar, 1973; Stock & Zinszer, 1987) showed costs to be of medium importance.

(20)

20 Wuyts et al. (2009) found that the importance of price depends on the decision making stage. In the stage of consideration (making it on to the short list) price is of no importance. However, when a PSF is on the shortlist and a final choice has to be made price becomes quite important, ranked third after ‘good personal relationships’ and ‘expert image’.

Another interesting viewpoint on the importance of price comes from Fitzsimmons et al. (1998). They propose that the importance of price is related to the importance of the service to the buyer: when the importance of the service is low, vendor selection is driven primarily by cost found that purchasers place greater importance on price under conditions of evaluating less critical services, and on quality for more critical services. The problem however with Fitzsimmons’s et al. (1998) reasoning is that the research by Ostrom & Iacobucci (1995) was conducted amongst consumers, not in a B2B setting. The fact that it has not been tested in a B2B setting does not make the proposition untrue, but one should be careful with using such assumptions.

3.3 Public sector procurement

Not only knowledge about decision criteria for purchasing (professional) services is scarce, knowledge about services procurement in general is limited (Seth, 1996) and this is especially true for the public sector (Murray, 2001; Roodhooft & Van den Abbeele, 2006; Schiele, 2005). As Seth (1996) further points out, literature on this subject is scattered over many disciplines like political sciences, organizational psychology and several sub-sectors of business studies. This makes it even harder to collect and process all available knowledge. Seth’s (1996) review of organizational buying behavior literature also yields few clues about distinct characteristics of public sector purchasing. Naturally, in the fifteen years between his review and this study, articles have been published about public sector procurement. However, most of these articles are about regulation and public tendering, which could be a cause for the limited knowledge about public sector procurement of services.

When an assignment reaches a certain value threshold in the European Union, governments have to use a public tender. These measures have been taken by the European Committee to make sure that government assignments are transparent, competition between companies is fair and all potential suppliers have a level playing field. Much of the academic literature concerning government purchasing available has to do with this public tendering.

(21)

21 steadily lowered over the last decade, giving purchasers less freedom to choose a service provider. However, restrictions only apply to assignments with monetary values over the thresholds. Under the thresholds governments are free to select a provider of their liking. The United States also has a central contract approval processes for government contracts exceeding a certain value threshold. Governments have lowered the threshold below which managers can execute contracts outside this central contract approval processes (Potoski, 2008), as in the European Union.

European Union regulation only allows two selection criteria to be used in tenders: the lowest price or the most “economically advantageous” (Roodhooft & Van den Abbeele, 2006). The latter can be considered as ‘value-for-money’ and is the dominating principle for choosing a provider (Corcoran & McLean, 1998; Loader, 2007). Wimmer’s (2003) claim that competitive tendering on price alone is rarely a suitable mean of selection seems to confirm the dominance of the value-for-money principle. However, she presents no empirical evidence for this claim.

Although Lindskog et al. (2010) state that private and public sector purchasing is in many ways similar, empirical evidence suggests this may only be true for their comparison of the stages in the general purchasing process.

Corcoran & McLean (1998), Lian & Laing (2004) and Roodhooft & Van den Abbeele (2006) have shown the public sector to be more transaction oriented, whereas the private sector has generally followed the path of forming close, enduring relationships with suppliers (for large assignments). Roodhooft & Van den Abbeele (2006) further found that the private sector has a better knowledge of the PSF market (consultancy in their case) as well as a better knowledge of what are acceptable consulting prices. They further conclude that there is a higher variation of market knowledge in the public sector than in the private sector, that their satisfaction levels with service providers are lower and that the private sector puts in general more effort in evaluating the consulting projects than the public sector.

(22)

22 out of three municipalities did not use a standardized method for supplier evaluation. Furthermore, a majority (57 percent) of the municipalities that participated in their survey preferred using local suppliers over non-local suppliers. In practice this often means that in case of equal performance on priceand quality, the local supplier is offered the business.

(23)

23

4. Methodology

4.1 Subjects

This study focuses on the public sector in The Netherlands. Since including all ministries, provinces, municipalities, water boards, independent public bodies, advisory boards, etcetera would be too extensive for the scope of this study and not relevant for the Politieacademie, only provinces, municipalities and water boards were included in the population. Together they are known as ‘the decentralized governments’, referring to their position in relation to the central government. They all have their own span of power and have publicly elected governors. Due to their own administrative independence from the centralized government, different laws and regulations apply to them and they are partially allowed to make their own regulation (within their district).

The target population consists of 12 provinces, 25 water boards and 418 municipalities (total target population is 455).

Because of their relative independence from the central government, all decentralized governments have their own (law) enforcement duties with regard to the specific laws and regulations applicable to their responsibilities within their district. Because of this most decentralized governments have an Enforcement (& Public Supervision) department.

The large majority of the target population is functionally organized and the heads of the departments control most of their own budgets.

Due to the nature of the education the Politieacademie offers (i.e. focused on policing and enforcement), the heads of the enforcement departments are targeted to participate in this study. This choice is supported by Stock & Zinszer (1987) who found (jr. or sr.) functional executives to be the best person to target for surveys. Also, the purchasing literature indicates that purchasing departments are rarely involved in the purchase of professional services (e.g. Mitchell, 1994; Schiele & McCue, 2006; Smeltzer & Ogdon, 2002), making the heads of the enforcement department a logical choice.

(24)

24

4.2 Data collection

A mail survey is used as data collection tool for this research. Because of the exploratory nature of this study, qualitative methods would also have been appropriate (Corcoran & McLean, 1998) but by using a survey the whole population of decentralized governments can be targeted. This would not have been possible with for example interviews as interviewing 455 respondents would be too time consuming and costly. Also, it would be preferred to have an experienced interviewer conducting the interviews (Baarda & De Goede, 2006) which is not available for this study.

A mail survey, a telephone survey and a web survey were considered for the data collection. The telephone method was rejected because of the extensive time it consumes to achieve a reasonable sized sample, let alone to contact the whole target population. Also reaching the respondents at a time convenient for them is thought to be difficult due to their busy agenda’s, resulting in many non-responses. Furthermore, in a written survey the respondents are less likely to give socially desirable answers (Baarda & De Goede, 2006).

The mail survey was chosen over the web survey for this study for several reasons.

First of all, there is no database with all relevant names and email addresses of the targeted subjects. Since (potential) respondents prefer to be invited over the same channel as the actual questionnaire (Manfreda et al., 2008) it would be advised to invited them via email for a web survey, not via regular mail.

The second reason is that web surveys on average yield a (6-11 percent) lower response rate compared to mail surveys (Anseel et al. 2010; Baruch & Holtom, 2008; Malhotra, 2004; Manfreda et al., 2008). Also, no significant improvements in response rates for web surveys were reported for an ‘internet literate’ population compared to the general population.

Achieving high response rates in surveys has always been an issue for researchers and response rates have declined substantially over the last decades (Anseel et al., 2010; Baruch, 1999). Organizational researchers have an even harder time than their consumer-oriented colleagues to achieve adequate response rates, but a positive note is that response rates seem to be stabilizing (Baruch & Holtom, 2008).

In order to maximize the response rate, the following techniques were applied:

(25)

25 Coorper, 1983), it seems odd not to introduce the research and explain its purpose. Also, it can be used to motivate the respondent to participate in the survey.

- The Politieacademie and University of Groningen were explicitly mentioned in the cover letter and their logo’s used on the cover letter and the questionnaire because university and/or government sponsorship were found to have a positive effect on response rates (Anseel et al., 2010; Dennis Jr., 2003; Duncan, 1979; Linsky, 1975) - The cover letter was signed personally. The reported findings on the effectiveness of

personalization in raising response rates have been mixed. Duncan (1979), Yu & Cooper (1983) and Anseel et al. (2010) have reported positively, while Dennis Jr. (2003) questions the positive relationship. Most researchers define personalization as addressing the respondent personally in the cover letter envelope but Anseel et al. (2010) also include a personal signature by the researcher in the cover letter as a form of personalization. Since some studies have reported positively in relation to response rate, the choice was made to personally sign all 455 cover letters. Larger scale surveys of course do not permit this technique.

- A post-paid return envelope was sent with the questionnaire (Dennis Jr., 2003). Although research suggests using a real stamp is more effective (Connon, 2008; Harris & Guffey Jr., 1978; Linsky, 1975) and multiple stamps increase response rate even further (Duffy & Martin, 2001), these techniques were found to be too time consuming.

- As an incentive, respondents were offered an invitation for two persons to attend a seminar with a topic relevant to their occupation, free of charge, if they returned the completed questionnaire.

The effect of offering a non-monetary incentive or premium on response rate is a matter of mixed findings. Linsky (1975) found all studies offering non-monetary incentives to show favorable results, as did Yu & Cooper (1983). Anseel et al. (2010), however, found no positive results in their meta-analytic review.

(26)

26 postcard reminders have been proven to be effective tools for improving response rates (Anseel et al., 2010; Breen et al., 2010; Dennis Jr., 2003; Duncan, 1979; Linsky, 1975; Malhotra, 2004; Roscoe et al. 1975; Yu & Cooper 1983). The follow-up included a modified cover letter (also personally signed), the questionnaire and a post-paid return envelope.

- Respondents were offered a choice with regard to anonymity. They could send back the questionnaire anonymous, but they then forfeited the invitation for free seminar participation and a summary of the research results since their names and the names of their organization are needed to send it to them.

The effect of anonymity in surveys yields mixed results (Linsky, 1975). And although Malhotra (2004) states anonymity should always be promised to respondents, Yu & Cooper (1983) and Dennis Jr. (2003) found anonymity not to have a significant effect on response rates. Since no study found it to have a significant negative effect, respondents were offered a choice.

Techniques not used in this study include pre-notification, monetary incentives, colored paper and personalized addressing.

The general consensus in literature is that pre-notifications significantly help raising response rates (Anseel et al., 2010; Dennis Jr., 2003; Duncan, 1979; Linsky, 1975; Malhotra, 2004; Yu & Cooper, 1983). Although repeat contacts have been shown effective in raising response rates, Dennis Jr. (2003) found pre-notifications to be only marginally effective and Linsky (1975) advises to choose a follow-up letter over pre-notification when one has to choose because of cost reasons. For the latter reason the choice was made for a follow-up letter, not a pre-notification.

Monetary incentives were not used since it is considered inappropriate for an organizational survey. The public sector in particular will be reluctant to accept monetary compensation for participating in a survey and many governments have made guidelines for accepting gifts and/or incentives to avoid any appearance of conflicting interests or bribery. Furthermore, as Anseel et al. (2010) argue, organizational respondents are already being compensated for the time they spend filling out the questionnaire.

(27)

27 response rates is not found (Dennis Jr., 2003; Duncan, 1979; Linsky, 1975). It was therefore not used in this study.

As mentioned earlier, there is no database with all the names of the respondents. Personalized addressing was therefore not an option.

4.3 Questionnaire

4.3.1 Concept questionnaire

The basis for the questionnaire was found in the research of Sonmez & Moorhouse (2010). Their research is not only the most recent to date, it is also the most elaborate, using a total of 43 criteria (37 plus 6). Using a larger amount of criteria allowed them to ask more specific questions uncovering subtle but important differences with other studies. This provides a more precise picture of what purchasers of professional services find important.

The 6 high level criteria were discarded because they were not specified in contrast to the other 37 criteria. Without a description of the criteria, it is unclear what the authors exactly meant. Criteria like ‘product’ or ‘competence’ are open to too many interpretations for inclusion without further description. However, criteria with regard to reputation, costs, and understanding of the customer and their industry are included in the questionnaire as other studies have shown them to be important. The rest of the criteria were carefully examined for clarity of their description and relevance to the target population. This resulted in eliminating the following criteria:

- ‘bring added value’, because of its unclear definition - ‘presentation’, because of its unclear definition

- ‘demo cultural understanding’ (demonstrate understanding cross-cultural challenges), because of its lacking relevance to the target population

- ‘consistency’ (consistency in training materials across countries), because of its lacking relevance to the target population

(28)

28 The criterion ‘lowest price’ was included because of the mixed results in literature with regard to the importance of price. Also, like all governments in ‘Western’ countries, the Dutch government is cutting its budget due to the ongoing financial crisis which started in 2008. The Dutch (central) government wants to cut the budget with 15 billion Euro’s before 2015 and 29 billion Euro’s in the long term (Dutch Ministry of Finance, 2010). This will also have a significant impact on the decentralized governments since they are partially dependent on funds from the central government.

Furthermore, when looking at larger assignments (for which public tendering is used) price is always an issue. Although value-for-money is “not the lowest price” and it involves balancing what is provided against price, the lowest price is one of the few selection criteria allowed by the European Committee. And even though the price criterion is not rated as the dominant principle in the selection process, it nevertheless plays a role if the selection is based on the most “economically advantageous” tender (Roodhooft & Van den Abbeele, 2006).

Because of its importance in public tendering, ‘lowest price’ might also play a role in this setting.

An alternative for eliminating the criteria with unclear definitions was to create new definitions for these criteria. The choice was made not to do this because it has a high risk of causing invalid results. If the new definitions do not match the intended definitions of Sonmez & Moorhouse (2010), this study would measure different criteria but compare their ranking and scores in the end nevertheless.

(29)

29

4.3.2 Testing

As suggested by Malhotra (2004), the questionnaire was tested before it was sent out to the target population. The test group consisted of eight public sector managers and policy makers. Six personal interviews were conducted and two persons commented on the concept questionnaire by email. Members of the test group were asked if the criteria in the concept questionnaire were relevant to them, if they use criteria which were not in the concept questionnaire and if they thought some matters received too much or not enough attention. The input from the test group did not provide motive to exclude any criteria from the concept questionnaire but did lead to the inclusion of four additional criteria which were not used in previous PSF purchasing criteria research. The additional criteria are discussed below.

- Location: four test group members indicated that the location where the education/training is provided is of high importance to them when making a final choice between the providers on the shortlist. If the location is far from their own location, participants can already be tired from traveling before starting the session. This would negatively affect the education/training effectiveness and is therefore a relevant criterion for them. Also, if the education/training takes place during several serried days and is far from the organization’s location, participants would have stay in hotels and dinner arrangements have to be made. This would increase the total costs of the education/training.

One respondent indicated the location was of no importance at all.

The reason that location as a criterion is not found in previous decision criteria research may lie in the diversity of professional services and the settings of previous studies. Many large PSFs, whether it is a management consulting, architecture or accounting firm, go to their client’s office for personal meetings. With education/training this is often not the case and participant in many cases go to the location of the education/training provider.

(30)

30 - Speaks professional language: all of the test group members indicated that it is very important to them that the trainer providing the sessions speaks the professional language of the participants. The dominant reason for this is that it allows the trainer to reach out to the participants and win their trust. The trainer needs to prove that he/she understands the professional position and dilemmas the participants face. This would, according to the test group members, increase the training’s effectiveness.

Many professional service providers often only come in contact with managers or other highly skilled/educated staff from the client’s organization. These people are more likely to understand what the expert is talking about, even though it is not their own area of expertise. They are more likely to possess abstract thinking capabilities.

Front line personnel at the education/training PSF also comes into contact with people who are not managers or highly educated and do not possess well developed abstract thinking capabilities. This could be a reason that the test group members all indicated speaking the professional language is highly important, while previous studies have not signaled this.

- E-learning support: four out of six personal interviews revealed that the use of internet in a supporting role is found to be important. This e-learning option (in support of the personal sessions) allows the participants to access more information (and different ways of absorbing the information) and gives the participant more flexibility in planning and participation.

Possible reasons why this criterion has not been identified by previous research can be found in the timeframe when previous studies have been conducted (the absence or infancy of the internet at the time) and the large difference between the PSFs, even within Von Nordenflycht’s (2010) categories.

(31)

31 The private sector of course also makes budgets for the next year and the end of the current year, but these are often less strict than in the public sector. They can generate extra income more easily than many public sector organizations can because public organizations depend on taxpayers money as main (or only) source of income. In contrast, private sector companies face less predictable demand and therefore often have more flexible budgets.

Since no study has been dedicate to the decision criteria of the public sector, this has not been signaled in the past. Furthermore, many professional services (like legal advice/representation, advertising or management consultancy) are required because of changes in an organization’s environment and are therefore more difficult to plan well in advance.

The test group was also asked to comment on the relevance of the ten lowest ranked criteria of Sonmez & Moorhouse (2010). All test group members indicated that the criteria were not relevant and should therefore not be included in the questionnaire, with the exception of the criterion ‘lowest price’. Several respondents found this criterion to be “interesting” and were curious whether or not their colleagues used this criterion, especially considering the upcoming budget cuts.

4.3.3 Final questionnaire

The final questionnaire consists of 33 criteria. The criteria and their full description can be found in table 5.1. The questionnaire in English can be found inappendix IIand the complete questionnaire in Dutch (as sent out to the respondents, including cover letter) inappendix III.

(32)

32 The questionnaire was originally made in English and then translated into Dutch by a translation agency2 for the target population. It is likely that respondents are more reluctant to fill in a questionnaire in a language other than their native language. They may not want to make an extra effort to understand the questions, be unsure about their level of English and since enforcement is not an international topic they may feel less affective commitment to fill in a questionnaire in English. Sending the questionnaire in English is therefore most likely to negatively affect the response rate.

An English questionnaire is also likely to negatively affect validity. Respondents might not understand the questions correctly and because of that give answers to questions which are not asked.

The translation was adjusted where needed or when a more suitable translation was available.

2

(33)

33

5. Results and discussion

5.1 Response

In total 200 respondents completed the questionnaire and sent it back (of which 25 anonymous), resulting in a response rate of 44 percent for the total population. The response rate amongst water boards was the highest with 72 percent, of the provinces 66,6 percent responded and the response rate for municipalities was 41,4 percent. Surprisingly, only one returned survey was not usable and only a few questionnaires showed missing values. If any answer was missing, it was mainly in the section with identification variables, not the section with decision criteria.

Seventeen more questionnaires (representing 21 organizations due to enforcement collaborations between municipalities) were returned during and after analysis of the data and were therefore not included in this research.

An explanation for the large differences in response rates may be found in the relatively low number of water boards (25) and provinces (12), compared to municipalities (418). The numbers are too small though to confidently state reasons for the differences.

However, one possible reason could be that the enforcement task turned out to be part of a larger department in many smaller municipalities. These departments are then often called ‘public services’ or ‘public area management’. In these cases, the questionnaire may not have reached the right person within the organization (since the envelopes were addressed to the head of the Enforcement department) and was therefore not filled in.

The response rate of 44 percent is in line with response rates of other academic studies. Baruch & Holtom (2008) reported an average response rate of 48,3 percent, with a standard deviation (SD) of 21,6, for published academic studies between 2000 and 2005. However, surveys collecting data from organizations reported an average response rate of only 35,7 percent (SD 18,8). They also stated that response rates should be within 1 SD of their average. Since the response rate of this study is within their norm, no elaborate reasoning for achieved result is needed.

(34)

34

5.2 Decision criteria

The mean scores and standard deviations (SD) for the 33 criteria are shown in table 5.1.

Criterion Full description Mean

rating

SD 1. Speaks

professional language

The trainer delivering the sessions speaks the professional language of the participants

6,17 ,739 2. Customization The ability of the provider to customize the training

to our needs

5,99 ,870 3. Knowledge of

industry

The provider demonstrates a good knowledge of our industry’s/sector’s issues

5,92 ,846 4. Experience in

industry

The provider has experience serving organizations in our industry/sector

5,91 ,881 5. Offer real

experience

The organization and/or the trainer uses real world experience and anecdotes

5,86 ,880 6. Successful

experience

Prior successful experience with the provider 5,77 ,714 7. Return on

investment

Evidence of tangible return on investment 5,66 ,895 8. Challenge thinking The ability of the trainer delivering the sessions to

challenge our thinking

5,61 ,891 9. Effective

training/solution

The training has been used by many people and proven to be effective – a tried and tested solution

5,55 1,028 10. Price The quoted price of the training (per participant) 5,53 1,058 11. Location The training takes place at a location that is easy and

fast to reach

5,43 1,233 12. Training

effectiveness

The ability to measure the training effectiveness 5,43 1,199 13. Personal contact Personal contact with the trainer delivering the

sessions

5,40 1,068 14. Personal

assessment

Personal assessment of the trainer delivering the sessions

5,33 1,050 15. Specific

reputation trainer

The reputation of the trainer delivering the sessions on the specific subject

5,32 ,951 16. Specific

reputation provider

The reputation of the provider on the specific subject 5,28 1,034 17. Flexibility Flexibility to schedule the training sessions around

our agenda

5,26 1,114 18. Latest innovative

methods

The training features the latest and most innovative methods/techniques

5,25 1,278 19. General

reputation provider

(35)

35 20. General

reputation trainer

The general reputation of the trainer delivering the sessions

5,13 1,041 21. Success stories Examples of success stories from other companies 4,80 1,210 22. Demonstrate good

knowledge

The provider demonstrates a good knowledge of our organization

4,75 1,152 23. Planning well in

advance

The possibility to plan education/training well in advance because of our budget cycle

4,75 1,466 24. E-learning

support

The provider uses e-learning support besides personal contact with the trainer delivering the sessions

4,71 1,343

25. Recommendation Recommendation from someone within our organization

4,59 1,255 26. Projects

completed

Evidence of the number of similar projects completed

4,48 1,229 27. Personal

relationship

There is a good personal relationship with the trainer delivering the sessions

4,44 1,482 28. Range The range of trainings offered by the provider 4,43 1,468 29. Train internal

personnel

The opportunity to train internal personnel to deliver the training

4,14 1,566 30. References Having access to current clients for a personal

reference

4,08 1,271 31. Help business

case

The provider helps us to build a business case internally

4,07 1,457 32. Opportunity to

test

The opportunity to participate in a pilot or test session

3,84 1,522 33. Lowest price The quoted price (per participant) was the lowest of

all potential suppliers

3,71 1,401

Table 5.1: Criteria, full description, mean rating and standard deviation

As indicated by all members of the test group, the most important decision criterion for purchasing education/training is that the trainer speaks the professional language of the participants.

(36)

36 ‘Customization’ is ranked as the second most important criterion. This is in line with the research of Sonmez & Moorhouse (2010) where the criterion comprehending customization is ranked the highest, with a mean score of 6,47 out of 7. If not for the added criterion ‘speaks professional language’, it would be the highest ranking criterion in this study as well. It also fits the bigger picture where customers more and more want tailored products/services, especially when purchasing professional services.

The third (‘knowledge of industry’), fourth (‘experience in industry’) and fifth (‘offer real experience’) criteria all cover an aspect of industry expertise or industry experience. Unsurprisingly, the third and fourth criteria have almost identical mean scores and are two of the strongest correlating criteria (see appendix IV). Also, they are the two of the strongest correlating criteria for the criterion ‘speaks professional language’. This makes sense since knowledge of and experience in the client’s industry helps speaking the participant’s language.

The use and high importance of experience as decision criterion is consistent with the existing literature (Bloom, 1984; Corcoran & McLean, 1998; Dawes et al., 1992; Saleh & Sarkar, 1973; Sonmez & Moorhouse, 2010; Stock & Zinszer, 1987) from the private sector and different professional services.

Although it may not come as a surprise to some, there is a remarkable consensus amongst respondents with regard to the importance of successful prior experience with a PSF. Nobody rated this criterion lower than four (neutral) and 96,5 percent of the respondents gave a score of five or higher (somewhat important to very important). Almost 68 percent rated it with a six or higher. This emphasizes the importance of customer retention.

By invalidating the misconception that customer retention is the beatifical solution for improving financial results, as suggested by Reichheld & Sasser Jr. (1990), East et al. (2006) provide a clear review of the customer retention literature. This study, however, does shows customer retention provides a foot-in-the-door to improve revenues. Marketing is not just about acquiring new customers, it is also about cultivating current ones.

(37)

37 place. This makes reputation of only medium importance. All criteria concerning reputation show medium to high correlations (as categorized by Cohen (1988)), varying from ,355 to ,668. All correlations can be found in appendix IV.

The relatively low scores for all reputation criteria are somewhat surprising.

Purchasing professional services is perceived to be a relatively high risk activity (Mitchell et al., 2003). The credence qualities of the services increase outcome uncertainty (performance risk), which would traditionally lead to more risk reducing activities.

Reputation conveys quality information, providing signals in situations marked by information complexity or inadequacy (Fombrun & Shanley, 1990; Shapiro, 1983). As Fombrun & Shanley (1990) suggest, (corporate) reputation therefore serves as a credible market signal when facing imperfect information.

Placing great value on reputations therefore makes sense as a risk reducing activity, yet the public sector seems to go against this logic. It might be that the perceived risk is lower for purchasing education/training services compared to purchasing other professional services. Another explanation is that the public sector has a higher risk threshold and therefore engages less in risk avoiding activities. A non-performing service (and the possible financial consequences as a result) is less of a threat to its existence than it might be to many private organizations. The public sector usually has ‘deeper pockets’, has a fairly secure stream of income and cannot go bankrupt. Support for the latter explanation can be found in the work of Mitchell et al. (2003). They found that local authority employees/managers were far more concerned with their own job security or jeopardized pay increase after purchasing a professional service that did not perform as desired, than that they were concerned with the local authority losing money.

Another group of criteria which is of relative importance to the respondents includes ROI and effectiveness. These criteria are ‘return on investment’ (ranked seventh), ‘effective training/solution‘ (ranked ninth) and ‘training effectiveness’ (ranked twelfth). The criterion ‘return on investment’ shows a medium correlation with ‘training effectiveness’ (,481) and ‘effective training/solution‘ (,477).

(38)

38 Price (per participant) seems to be of slightly higher importance to the respondents in this study than in previous studies where price was usually ranked in the middle (Dawes et al., 1992; Saleh & Sarkar, 1973; Stock & Zinszer, 1987). As in Sonmez & Moorhouse’s (2010) study, being the cheapest was ranked last.

Based on these results, as well as the interviews in the test phase, it can be concluded that price is of importance, but only in relation to other criteria. There needs to be a balance between price and quality, just being the cheapest will not get you very far in this industry. These results regarding price and the lowest price imply similar roles for these criteria as in public tendering, discussed in paragraph 3.3.

As expected, location is of relative importance but does not belong to the top ranking criteria. The other two added criteria, ‘planning well in advance’ and ‘e-learning support’, did not live up to their expected importance based on the test phase. They are both found in the bottom third of table 5.1.

When comparing the results of this study in the public sector to the previous studies in the private sector it seems that the most obvious difference is the lower importance of the criteria covering reputation. Where previous studies found (specific types of) reputation to be (very) important (Dawes et al., 1992; Saleh & Sarkar, 1973; Sonmez & Moorhouse, 2010; Stock & Zinszer, 1987), this study found the reputation criteria to be of only medium importance. Issues like customization, knowledge of and experience in the clients industry and prior successful experience with the provider or individuals do not seem to show differences in importance.

5.3 Clustering

In order to determine whether specific groups of training/criteria combinations could be identified, cluster analyses and discriminant analyses were conducted (as in Cray et al. (1988, 1991)).

(39)

39 Malhotra (2004) and Punj & Stewart (1983) both discuss the importance of variable selection for cluster analysis. Variables that are not related to the final clustering in some manner, i.e. do not differentiate among clusters, cause a serious deterioration of the performance of all clustering methods. This problem is least severe with the K-means procedure. “Shotgun approaches” where everything known about the observations is used as a basis for clustering should be avoided when possible. However, since there is no prior knowledge about the discriminating power of the criteria this approach is used nevertheless. Using for example only the top 20 criteria in table 5.1 (to avoid the “shotgun approach”) is likely to lead to more homogeneity amongst the clusters.

When looking at the agglomeration schedule in appendix V, the hierarchical clustering method offers no concrete evidence for the number of clusters to be used. Two, three, four or even seven clusters could be formed.

Because of the above, K-Means analyses were conducted with two, three, four, five, six and seven clusters. The results can be found in appendix VI. All clusters show sufficiently large distances.

Despite the low number of cases within two clusters in the seven cluster option, it still is an interesting option because it does show the largest difference in the agglomeration schedule (stages 188-189). The two cluster option has the second largest difference between coefficients for the options under consideration, but it allows for only limited profiling after the discriminant analyses. For the same reason the three cluster option is less preferred than the four cluster option. The four cluster option still has a relatively large difference in coefficients, has a sufficient number of cases in all clusters and allows for more detailed profiling.

(40)

40 To further assess whether the four or seven cluster option was best, discriminant analyses for both options were conducted. The first discriminant analysis showed all decision criteria to be significant for both options. For the four cluster option, three significant discriminant functions were generated. Six significant discriminant functions were generated for the seven cluster option. The structure matrix for the seven cluster option did not show a more neatly grouping (by strong correlations) of criteria covering a similar aspect across the functions than the four cluster option. The second discriminant analysis resulted in no significant identification variables for both options.

Decision criteria for purchasing professional services

Decision criteria for purchasing

professional services

What about the public sector?

P.W. (Pim) Hoogers

Decision criteria for purchasing

professional services

What about the public sector?

Master Thesis

Msc Business Administration

Specialization Marketing (Management)

August 2011

Management Summary

Preface

Table of contents

1.

Introduction

2.

About the Politieacademie

3.

Literature review

4.

Methodology

5.

Results and discussion