Tailoring the Tailor’s Tools:
Towards an Efficient Measure for Person-Organization Fit in
Employee Selection Processes
Master’s Thesis by Marc Plasmans (10606408)
Graduate School of Communication Master’s Programme Communication Science
Supervision by Dr James Slevin June 27, 2014
Abstract
This study aims to halt organizations in unnecessarily losing time and resources on flawed selection methods for person-organization (PO) fit and on wrongfully selected applicants with a low PO fit. To achieve this goal, a minimalized and efficient PO fit measure was identified and evaluated; the Matching Values Test (MVT). Unlike previous measures, the MVT provides an efficient technique for measuring need-supplies PO fit indirectly through two multiple-response questions. The psychometric properties of this measure were assessed by distributing an online questionnaire to a random sample of currently employed people at various organizations (N = 253). Results indicate that the MVT’s internal consistency reliability is insufficient, but the measure generally shows acceptable levels of test-retest reliability, convergent and concurrent validity. Yet, on most accounts, the MVT was somewhat inferior to a more elaborate, well-established measure for PO fit. Based on these results and additional ontological assessments, the psychometric properties of the MVT were deemed sufficient for effective implementation in employee selection processes. Yet, several recommendations can be made to improve the measure’s usefulness in this context. Most notably, it is recommended that item groupings are changed to be more clearly indicative of one specific underlying dimension. Future assessments of improved versions of the MVT are required before managers may optimally apply this measure in employee selection processes with a maximally reduced error for predicting PO fit.
Keywords: Matching Values Test, person-organization fit, value congruence, subjective fit, employee selection, recruitment, psychometric
1. Introduction
In the context of employee selection, organizations increasingly take on a tailoring role as they attempt to maximize the fit between employees and the various ‘environmental suits’ of an organization (Kristof-Brown, Zimmerman, & Johnson, 2005). In addition to hiring those with the right knowledge, skills and abilities (KSAs) for a particular job, individuals are now often hired for their compatibility with the organization as a whole (Harris & De Chernatony, 2001; Tsai, Chen, & Chen, 2012; Westerman & Cyr, 2004). The extent to which a person fits an organization’s unique ‘cultural suit’ is generally referred to as person-organization (PO) fit (Kristof-Brown et al., 2005). Hiring individuals with a high PO fit is particularly important for organizations, because it significantly reduces employees’ turnover intention (Kristof, 1996; Moynihan & Pandey, 2008). Numerous labor market studies found that newly hired employees are most likely to turn over (Farber, 1994; Jovanovic, 1979), which will be costly because they depart after the organization made investments into recruitment, selection and training but before returns on these investments are realized in the form of performance (Griffeth & Hom, 2001). In addition, scholars found extensive support for the positive effects of PO fit on employees’ behavior and attitude, such as increased job satisfaction and
organizational commitment (Lauver & Kristof-Brown, 2001; O'Reilly, Chatman, & Caldwell, 1991). In pursuit of these desired effects, many organizations now attempt to adequately select job candidates that will have a high PO fit (Tsai et al., 2012; Kristof, 1996).
As the organizational interest in PO fit selection methods is growing, many
organizations now face a significant challenge with regards to measuring this construct. When organizations decide to measure PO fit, they have to rely on implicit judgments, standardized tests and techniques – selection decision aids (SDAs) – or both (Adkins, Russell, & Werbel, 1994; Hambleton, Kalliath, & Taylor, 2000; Highhouse, 2008). However, the applicability of these methods in employee selection processes is significantly limited. Implicit judgments by
recruiters were found to be of little value for predicting an applicant’s level of PO fit (Adkins et al., 1994; Cable & Judge, 1997). In contrast, academic literature includes a range
empirically validated measures for PO fit, which were found to have good predictive value (De Clercq, Fontaine, & Anseel, 2008; Highhouse, 2008). However, this particular group of SDAs fails to fulfil the practical needs of many organizations due to long response times and multiple outcome values (De Clercq et al., 2008; Edwards, 1994; Ryan & Tippins, 2004). Generally, organizations require more efficient measures – measures that minimize the time and effort required by respondents and the organization that applies it. Using such measures may increase the pool of selectable applicants and contributes to maintaining a positive brand image by providing a more accessible application procedure (Bhatnagar & Srivastava, 2008). Furthermore, PO fit is generally one of many constructs under study and thus space on employee selection questionnaires is often limited (Hambleton et al., 2000; Kristof-Brown et al., 2005). Without an efficient measure, organizations may decide to not measure PO fit at all or apply efficient, commercially developed measures instead (De Clercq et al., 2008;
Hambleton et al., 2000). While many organizations appear to use these commercial measures, the predictive value of such measures is questionable at best. In fact, one of the most widely used measures for PO fit – the commercially developed Omnia Profile® – was found to have a generally insufficient criterion-related validity (Hambleton et al., 2000).
The apparent lack of efficient, empirically validated methods in academic literature to measure PO fit causes at least two problems for organizations in the context of employee selection. First, organizations may be unaware of the flaws of existing PO fit measurement methods and thus waste significant time and resources by applying methods with limited predictive value (Hambleton et al., 2000; Highhouse, 2008). Second, whether these methods are applied or not, without alternative PO fit measures there is a significantly increased risk for organizations of hiring individuals with a low PO fit. As a result, a lot of additional
resources may be required for extensive training and development processes to increase the PO fit of new employees (Cheng & Ho, 2001; Garavan, 2007). Moreover, large amounts of time and money may be lost on individuals that will need to be dismissed early after hiring due to a limited fit in the organization (Adkins et al., 1994; Cable & Judge, 1997; Kristof-Brown, 2000). The present study aims to minimize the use of flawed selection measures of PO fit and to vastly decrease the chance that an organization selects applications with a low PO fit. As such, this study is concerned with identifying and evaluating a new, efficient measure for PO fit; the Matching Values Test (MVT). To determine whether this ‘tailored’ measure can reach similar levels of accuracy as found in longer, empirically validated
measures for PO fit, the present study will evaluate the psychometric properties of the MVT. As such, the following question has guided this study: “To what extent are the psychometric properties of the MVT sufficient for predicting PO fit in the context of employee selection?”.
The psychometric properties of the MVT will be assessed by establishing the measure’s overall reliability and criterion-related validity. By combining the results of this study with existing theory, a critical overall assessment will be made of the MVT’s viability for use in employee selection processes. This assessment results in a set of recommendations on how the measure may be improved for effective use in employee selection processes. The present study’s set of deliverables will advance academic literature on PO fit by providing a first step in filling the literature gap with regards to efficient PO fit measures. Moreover, it extends the small body of literature on the usefulness of PO fit in a selection context, as called for by fit scholars (McCulloch & Turban, 2007; Werbel & Gilliland, 1999). Herewith, this study contributes to increasing both scholars’ and practitioners’ awareness of the flaws of existing measurement methods of PO fit. Practitioners may apply this knowledge to improve their employee selection processes and decrease the risk of hiring individuals with a low PO fit. This way, organizations’ unnecessary loss of time and resources may be avoided.
In the following section of this paper, a clear description of the MVT and its relevance to the present study will first be presented. Subsequently, I will outline this study’s guiding theoretical framework, which will result in the set of tested hypotheses. The applied methods that were used to test these hypotheses will be reported next, followed by the results of the conducted statistical analyses. Then these results will be discussed and I will make
recommendations on how to improve the MVT’s applicability in employee selection
processes. Finally, this paper will be concluded by presenting the practical implications of the findings and by making suggestions for future research.
2. The Matching Values Test
The MVT was developed as part of one of the larger products by Employer Brand Insights –
an organization that develops analytical measures for multinational organizations(Employer Brand Insights, 2014; J. Mensink, personal communication, June 3, 2014). This particular measure was identified for use in the present study, because of its high potential to be successfully applied in employee selection processes as both an efficient and sufficiently accurate measure for PO fit. This potential can be explained by at least two arguments. First, the MVT was developed by a commercial organization out of personal need for an efficient PO fit measure for employee selection processes. As such, the measure is likely to fulfil the needs of other organizations in terms of use and efficiency. Second, in line with recent
recommendations on maximizing the accuracy of PO fit measures (Ostroff, Shin, and Kinicki, 2005), the MVT is based on a well-developed theoretical structure; this structure is provided by the Competing Values Framework (CVF) by Quinn & Rohrbaugh (1981).
Underlying Theoretical Structure. The CVF was originally developed as a way to categorize competing organizational strategy and design theories. According to Quinn and Rohrbaugh (1981), these theories can be grouped along two primary axes. The first axis
contrasts flexibility and change with control and stability, whereas the second axis contrasts internal focus (well-being and development of individuals inside the organization) with external focus (well-being of the organization itself, while emphasizing task
accomplishment). When plotting these two axes, four quadrants occur which each reflect a distinct type of organizational structure. Once it was recognized that organizations typifying the four quadrants have different organizational values, the CVF was adapted to study organizational culture and later PO fit (De Clercq et al., 2008; Quinn & Kimberly, 1984).
While scholars often assign different labels to the four cultural quadrants of the CVF, the present study follows recent research on PO fit (e.g., Leung & Chaturvedi, 2011; Meyer, Hecht, Gill, & Toplonytsky, 2010; Ostroff et al., 2005) by defining the four quadrants as follows: human relations (flexible, internal), open systems (flexible, external), rational goal (control, external), and internal process (control, internal). Specifically, organizations with an emphasis on human relations promote loyalty, trust and teamwork in order to increase
employee engagement and development. Those with an open systems orientation strive toward innovation and growth as they encourage employees to take risks and use their
creativity. Organizations within the rational goal quadrant emphasize efficiency, productivity and accountability as they strive for goal attainment and competitiveness. Finally, those organizations with an internal process orientation typically use bureaucratic control strategies in order to increase overall timeliness and efficiency (Quinn & Rohrbaugh, 1981; Quinn & Kimberly, 1984). Even though these quadrants appear to clearly delineate four distinct
organizational cultures, in reality organizations reflect characteristics from different quadrants of the CVF to varying degrees (Cameron & Quinn, 2006). In fact, Cameron & Quinn (2006) argued that it may even be beneficial for organizations to reflect values from multiple
quadrants. Following this argumentation, the present study will refer to each of these four as a component of organizational culture, rather than being a culture in itself.
Table 1 – Value Groupings of the MVT
Human Relations Open Systems Rational Goal Internal Process Helpfulness Adventure Results-oriented Optimization Equality Creativity Wanting to win Carefulness
Teamwork Innovation Leading Stability
Human Relations/ Open System/ Rational Goal/ Internal Process/ Open System Rational Goal Internal Process Human Relations Flexibility Autonomy Structure Deliberation Freedom Ambition Cost-conscious Sustainability Fun Challenging Craftsmanship Collective
Note. As the original MVT is in Dutch, the values in this table are translated and may therefore not fully reflect the intended meanings; the original list of values in Dutch is presented in table B1 in the appendix.
Psychometric Properties. The MVT can be used to make predictive or current assessments of PO fit by establishing the degree of congruence between two sets of the same values; one set measures preferred organizational values (P-dimension of PO fit) and the other set measures perceived organizational values (O-dimension of PO fit). Whereas preferred organizational values in employee selection processes are always based on the response of the applicant, perceived organizational values may also be based on responses by members of an organization (e.g., managers and supervisors) or the aggregate of their responses. Ideally, however, not both measurements are based on responses of the applicant, because this would allow them to fake their PO fit if so motivated (McCulloch & Turban, 2007). The actual PO fit measurement is done by asking respondents to pick at least two values from a total list of 24 values, which are based on the four underlying dimensions of the CVF as shown in table 1. Hereby, a multiple-response format was used, because it allows consideration of a large variety of values in a time-efficient and inviting manner.
After data acquisition, the MVT offers two ways to calculate PO fit; calculation by match count or difference scores. On the one hand, calculation by match count refers to
directly comparing the answers of both questions of the MVT; the more matching values are found, the higher is the PO fit score. Calculation by difference scores, on the other hand, takes a more traditional approach to calculating PO fit by using the four underlying dimensions of the CVF (Kristof-Brown et al., 2005; Leung & Chaturvedi, 2011). As this is study includes the first psychometric assessment of the MVT, part of this assessment is to compare the suitability of these calculation methods for employee selection processes. In the following section, I will situate the MVT within existing academic literature.
3. Theoretical Framework
Below, the theoretical framework that functions as the literary backbone of this research is presented. This section is divided in four subsections; (1) a fitting selection process, (2) the right applicant fit, (3) PO fit measures and calculations, and (4) evaluating the MVT.
3.1. A Fitting Selection Process
The importance of selection processes has long been debated in academic literature. This debate is concerned with determining whether individual behavior (e.g., job performance) is a function of the person (attributes of an employee), the situation (characteristics of the work setting), or the interaction between both variables (Bowen, Ledford, & Nathan, 1991; Stevens & Ash, 2001). While scholars originally believed that socialization and training practices are key to creating a successful workforce (Mischel, 1968; Mount & Barrick, 1995), now most scholars recognize the importance of both attraction and selection processes for acquiring the best candidates for a particular job or organization as well (Judge & Cable, 1997; Ryan & Tippins, 2004). In fact, employee selection processes appear to be deemed more important than ever as a result of globalization, pressure for speed and innovation, and growing
& Snow, 2001). Despite the increased attention for personal characteristics, the majority of researchers today assumes that both the person and the situation matter and that the
interaction between these variables determines employees’ performance and other behaviors (Stevens & Ash, 2001). While the present study mainly focuses on selection processes, it acknowledges the importance of situational variables by taking an interactionist perspective.
Fitting Selection Methods. Most organizations apply employee selection processes in some way, however, the different methods that are applied within these processes may differ significantly across organizations (Schmidt & Hunter, 1998). When developing employee selection processes, employees such as human resource (HR) managers are challenged to address an organization’s varied needs (Ryan & Tippins, 2004). Hereby, they need to take into account the sometimes competing needs and views of an organization’s varied
stakeholders – hiring managers, applicants, legal departments, labor unions, and other external groups. So, the choice for a particular selection method may be the outcome of interrelated communicative processes by an organization’s internal and external stakeholders (Cornelissen, 2011; Johns, 1993). Moreover, groups may use their respective power to exert additional pressure (Mitchell, Agle, & Wood, 1997). For example, potential applicants may exert pressure on HR managers by not applying for a job, because they deem the selection process to be too elaborate; as this could significantly limit the available talent pool for an organization, the selection process may be adapted. By balancing the respective importance of a wide variety of considerations, the usefulness of specific methods for employee selection processes can be determined (Ryan & Tippins, 2004).
Despite the general belief among scholars that the usefulness of employee selection methods is largely situation-dependent, existing literature also offers suggestions that may be used to assess these methods’ general usefulness. According to Ryan and Tippins (2004), the most important consideration for any selection method is its validity, because a test with low
validity will not result in good hiring decisions and therefore will not be cost-effective over an extended time period. Whereas scholars generally agree with this statement (Pfeffer & Sutton, 2006; Rousseau, 2006; Schmidt & Hunter, 1998), the vast majority of practitioners continues to rely on intuitive selection methods with low validity; in particular, implicit judgments based on unstructured interviews (Highhouse, 2008). The traditional unstructured interview has remained the most widely used selection method for over 100 years, even though many SDAs (e.g., standardized tests, structured interviews, mechanical combination of predictors) have been developed with a significantly higher validity and thus significantly reduce error in predictions of employee behavior (Buckley, Norris, & Wiese, 2000; Schmidt & Hunter, 1988). Highhouse (2008) states that this may be caused by practitioners’ implicit belief that it is possible to achieve near-perfect precision when predicting behaviors through selection methods. As such, practitioners have an inherent resistance to the analytical approach of SDAs, because they fail to view selection as probabilistic and subject to error. There is substantial evidence that ambiguity about the likelihood that unstructured interviews will lead to successful predictions encourages more optimism than a low known probability for success of an SDA (Kuhn, 1997). Thus, despite the crucial importance of validity for selection
methods, practitioners may continue to stubbornly rely on intuition as the main selection method. In line with recent recommendations by scholars (Highhouse, 2008; Ryan & Tippins, 2004), the present study encourages data-based decision making though SDAs.
After choosing the preferred set of SDAs, their overall usefulness within the employee selection process also depends on the way they are applied. Firstly, scholars often recommend the use of multiple SDAs to reduce overall prediction error (Schmidt & Hunter, 1998).
Furthermore, the use of SDAs may decrease workforce diversity, by systematically excluding specific groups of people. As workforce diversity is generally valued by stakeholders of an organization, SDAs should often be applied in such a way that prevents an overly
homogeneous workplace (Sackett, Schmitt, Ellingson, & Kabin, 2001). Also, as the outcomes of SDAs are interpreted by individuals, the probability for errors may be increased. Therefore, there should be clear, standardized guidelines on how to apply each SDA (Highhouse, 2008; Ryan & Tippins, 2004). Finally, SDAs should be used to assess criteria that are relevant for a particular situation (Barrick, Mount, & Judge, 2001; Ryan & Tippins, 2004). The SDA in the present study refers to the MVT, which predicts PO fit. To be able to assess the usefulness of this measure in employee selection processes, I will now elaborate on the criterion it predicts.
3.2. The Right Applicant Fit
SDAs may be used to predict a wide range of selection criteria – employee behaviors. The vast majority of employee selection criteria, however, can be embedded within the broader context of person-environment (PE) fit (Kristof, 1996; Kristof-Brown et al., 2005). This paradigmatic term can be regarded as a “syndrome with many manifestations” (Schneider, 2001, p. 142) and is generally defined as the compatibility between attributes of individuals and their respective (work) environments (Pervin, 1968; Schneider, 1987). When focusing on PO fit, the environmental aspect refers to organizations as the construct addresses the
compatibility between individuals and their environments on an organizational level (Kristof-Brown et al., 2005). Next to PO fit, four other main types of PE fit have been defined; person-vocation (PV) fit, person-job (PJ) fit, person-group (PG) fit and person-supervisor (PS) fit (Kristof-Brown et al., 2005). All PE fit types have been well accepted by scholars and it was found that they were additive as well as differentially important for a wide variety of
outcomes (e.g., Cable & DeRue, 2002; Meir & Melamed, 1986; Saks & Ashforth, 1997). While there is conceptual and empirical support for the distinctiveness of the five
environment types of fit, they may overlap (Kristof, 1996). For example, job characteristics are part of the overall organizational characteristics, which results in an overlap between PJ
and PO fit. Nevertheless, employees may experience varying degrees of fit when comparing the job level with the organizational level (O’Reilly et al., 1991).
Complementary versus Supplementary Fit. The PE fit paradigm comprises two longstanding traditions of research, which equally apply to PO fit (Muchinsky & Monahan, 1987). First, the first research tradition refers to supplementary fit, which occurs when an individual and an environment possess similar or matching characteristics (Cable & Edwards, 2004; Muchinsky & Monahan, 1987). Whereas complementary conceptualizations of fit have dominated the PJ fit literature, supplementary fit has been the focus of other fit types – under which PO fit (Kristof-Brown et al., 2005). In the context of supplementary fit, PO fit could occur when an organization hires an employee that possesses skills that are already widely possessed by its current personnel. By taking both research traditions into account, PO fit can be defined as: “The compatibility between individuals and organizations that occurs when: (a) at least one entity provides what the other needs, or (b) they share similar fundamental
characteristics, or (c) both” (Kristof, 1996, p. 4).
Second, the other research tradition is built around the notion of complementary fit, which occurs when a person’s or an organization’s characteristics provide what the other wants or needs. Thus, complementary fit occurs when a person’s characteristics fill a gap in – or “make whole” – the current environment, or vice versa (Muchinsky & Monahan, 1987, p. 271). Originally, Muchinsky and Monahan (1987) strictly operationalized this type of fit as individuals’ skills meeting environmental needs or demands (demands-abilities fit). However, Kristof (1996) expanded the concept by including when individuals’ needs are met by
environmental supplies (needs-supplies fit). So, from a PO fit perspective, complementary fit occurs either when an organization satisfies individuals' needs or when an individual has the abilities that are required to meet organizational demands. The MVT operates within the needs-supplies complementary tradition of PO fit research.
Content Dimensions of PO fit. Within the two research traditions of fit, a wide variety of content dimensions is used to operationalize PO fit (Kristof-Brown et al., 2005). Even though the decision of which content dimension to use is somewhat dependent on the applied research tradition, scholars have attempted to propose a definitive content dimension for PO fit. As such, reviews of PO fit often contain heated debates about the relative merits of these various dimensions and how they should be measured (e.g., Edwards, 1994; Kristof, 1996; Meglino, Ravlin, & Adkins, 1991). Chatman (1991) argued for values as the basis of PO fit, because they are enduring characteristics of individuals and organizations. Conversely, Ryan and Kristof-Brown (2003) stated that personality traits are more stable, proximal to behaviour, and visible in others’ behavior than values. With the validation of the OCP (O’Reilly et al., 1991), however, value congruence has become widely accepted as the defining
operationalization of PO fit (Kristof, 1996; Verquer, Beehr, & Wagner, 2003). Additional content dimensions to measure PO fit include KSAs, goals, needs and attitudes (Bretz & Judge, 1994; Kristof-Brown et al., 2005; Witt & Nye, 1992). Following the vast majority of research on PO fit (Kristof, 1996; Kristof-Brown et al., 2005; Verquer et al., 2003), the MVT operationalizes PO fit by assessing the value congruence between individuals and
organizations. The MVT defines value congruence as the degree of similarity between an individual’s work value preferences and the cultural value system of an organization (Bretz & Judge, 1994; Chatman, 1991; Leung & Chaturvedi, 2011).
In summary, the MVT specifically predicts needs-supplies complementary PO fit while using value congruence as its content dimension. In several instances, fit scholars referred to this specific type of fit as values-supplies fit as well, since individuals’ preferred values are compared with organizations’ embodied or ‘supplied’ values (Edwards, 1992; Leung & Chaturvedi, 2011; Van Vianen, 2000). Some critics argue, however, that SDAs should assess as many specific PO fit types as possible as this would provide a more
comprehensive measurement of an individual’s overall PO fit (Kristof-Brown et al., 2005; Cable & Edwards, 2004; Ostroff et al., 2005). While the present study may be criticized for the same reasons, a counterargument may be that efficient measures need to prioritize certain types of fit over others in order to remain efficient. More specifically, the MVT also takes a needs-supplies perspective to PO fit over the supplementary fit perspective, since it measures individuals’ work value preferences – not individuals’ personal values (Kristof, 1996;
Edwards, 1991). While a supplementary fit perspective would have been valid as well, the MVT was created from the general belief that personality traits are more implicit and complex, which may make them harder to measure than an individual’s needs (Oliver & Mooradian, 2003; Schwartz, 1992). Thus, a supplementary perspective may have led to decreased efficiency of the measure due to increased complexity of the measured constructs. All in all, these choices may have been necessary to create an efficient measure, but it should be noted that they unavoidably decrease the comprehensiveness of the overall PO fit
assessment by the MVT. In the following section, I will elaborate how SDAs may be used to measure needs-supplies, complementary PO fit.
3.3. PO Fit Measures and Calculations
In order to assess whether the MVT’s psychometric properties are suitable for employee selection processes, it is important to establish what is known about ways to measure and calculate PO fit (Edwards, 1994; Kristof, 1996; Kristof‐Brown et al., 2005).
Direct versus Indirect Measuring Traditions. Two main measuring traditions of PO fit can be identified; indirect and direct measures (Kristof, 1996). First, scholars generally agree that a direct assessment of compatibility can be referred to as perceived fit (French, Rogers, & Cobb, 1974; Kristof‐Brown et al., 2005). Hereby, good PO fit is said to exist as long as it is perceived to exist, regardless of whether an individual has similar characteristics to,
complements, or is complemented by an organization (Kristof, 1996). Second, Kristof (1996) distinguished perceived fit from actual fit, whereby PO fit is indirectly determined through explicit comparisons of separately rated person (P) and organization (O) variables. Measures of actual fit can be divided into subjective and objective fit measures (French et al., 1974). To measure subjective fit, the same person reports on P en O variables as he or she perceives them. So, both perceived and subjective fit are assessed by a single source. Over the years, these two terms have often been used interchangeably or with reversed definitions (e.g., Cable & DeRue, 2002; Judge & Cable, 1997; Verquer et al., 2003). The present study follows the original definitions as posed by French et al. (1974). When measuring objective fit, P and O variables are reported by different sources. For example, organizational values could be measured via a survey among a group of managers and aggregating their scores. This score could then be compared with any individual’s values to determine PO fit. So, here the
organizational environment exists “independently” of an individual’s perception of fit (French et al., 1974, p. 316).
While all three measures of fit (perceived, subjective and objective) have been proven to be viable in determining attitudes and behavior (Cable & Derue, 2002; Leung &
Chaturvedi, 2011), scholars often disagree on which provides the most accurate fit measurement (Kristof, 1996; Kristof-Brown et al., 2005). Many believe that measures of perceived and subjective fit allow a certain degree of consistency bias, because a single source assesses the variables at hand (Edwards, 1991; Endler & Magnussen, 1976; French et al., 1974). According to both self-perception and cognitive dissonance theory, individuals are driven to maintain internally consistent perceptions (Bem, 1967; Festinger, 1957). As such, it is unlikely that an individual reports a low fit with an organization and reports a high level of satisfaction with that organization at the same time; this would cause cognitive dissonance (Festinger, 1957). This may explain why previous studies have found that perceived fit is
most strongly related to behavioural outcomes and objective fit shows the weakest correlations (Cable & Derue, 2002; Leung & Chaturvedi, 2011). Yet, Edwards (1991) denounced direct measures of fit entirely, because of their fundamental shortcomings with regards to consistency bias. Nevertheless, studies on PO fit continue to apply all three measures – separately or combined (Verquer et al., 2003). The MVT may be critiqued for including only one measurement type; indirect fit (e.g., Ostroff et al., 2005). However, the inclusion of additional types could have harmed the efficiency of the MVT.
Profile Similarity Indices (PSIs) versus Polynomial Regression. Aside from the chosen measurement type for assessing fit, the calculation of fit may significantly influence outcome values of PO fit as well. A recent debate in scholarly literature concerns the calculation of fit via indirect measures (Kristof-Brown et al., 2005). Traditionally, PO fit was calculated via PSIs, which may refer to either difference scores or profile correlations. Difference scores base fit measures on the sum of differences between P and O profiles, whereas profile correlations assess similarity in the rank ordering of elements within P and O profiles (Edwards, 1993). Even though the use of PSIs was traditionally prevalent in PO fit research, this calculation method has been frequently criticized with regards to issues of discarded information, conceptual ambiguity, being insensitive to the source of differences, and excessively restrictive constraints (Cronbach, 1958; Edwards, 1993; Nunally, 1962).
The criticism on PSIs was answered by Edwards (1993, 1994, 1996), who suggested to use polynomial regression to calculate the various forms of PE fit. Instead of collapsing P and O measures in a single score that captures ‘fit’, polynomial regression includes P, O, and associated higher-order as predictors. Hereby, fit relationships are presented in
three-dimensional surface plots, since every P, O and dependent variable is unique. Thus, by
examining characteristics of the surface plots it can be determined whether a fit relationship is supported. This often creates a better understanding fit relationships’ complexity than PSIs
(Edwards, 1993; Kristof-Brown et al., 2005). The introduction of polynomial regression has significantly changed PO fit research; the vast majority of recent studies have used this calculation approach (e.g., Edwards & Cable, 2009; Meyer et al., 2010; Van Vianen, 2000). The sudden rise in popularity of this calculation approach, however, does not mean it remains uncriticised. In fact, issues of multicollinearity, dependency on sample size and power, difficulties with complex moderation models and dummy-coded variables, as well as
conceptual concerns have been reported (Kristof, 1996; Leung & Chaturvedi, 2011; Tisak & Smith, 1994). Perhaps the biggest limitation of polynomial regression for the present study is the outcome of three-dimensional plots rather than a single fit score. This makes it hard to effectively apply polynomial regression in the context of employee selection. As such, the MVT follows the minority of recent studies on PO fit (e.g., Leung & Chaturvedi, 2011) by using PSIs and match counts instead.
Previous Tailor’s Tools. There is a wide variety of known measurement measures to determine PO fit. Whereas scholars’ generally agree on the short, three-item measure that may
be used to assess perceived PO fit directly (e.g., Cable & DeRue, 2002; Cable & Judge, 1996; Lauver & Kristof-Brown, 2001), opinions on assessing PO fit indirectly through value
congruence differ significantly (De Clercq et al., 2008). Scholars particularly disagree in terms of the measures’ theoretical foundations and their scaling techniques. First, for
indirectly measuring PO fit through value congruence alone, De Clercq et al. (2008) identified 42 measures in an extensive literature review of existing fit measures of which many were based on different theoretical foundations. Arguably, the three most well-established and widely used empirically validated measures were the 108-item Organizational Culture Profile (OCP), the 48-item Comparative Emphasis Scales (CES) and a 32-item CVF measure
(Meglino et al., 1991; Kalliath, Bluedorn, & Strube, 1999b; O’Reilly et al., 1991). These three well-established measures have been at the center of debates between scholars in terms of the
relative merits of their theoretical foundations (De Clercq et al., 2008). While the Omnia Profile® is not part of this debate, it is worth mentioning as this is the only commercially developed PO fit measure that was empirically evaluated. Much like the MVT, the Omnia Profile® was specifically aimed at meeting the demands of practitioners rather than scholars but it was not considered to be valid due to its failure of significantly predicting job
satisfaction and additional other external correlates (Hambleton et al., 2000).
Second, PO fit measures generally apply one of two scaling techniques; Q-sort or Likert scales. Advocates of the Q-sort technique praise its ability to avert socially desirable responses by forcing participants to rank-order values in an ipsative manner (Cable & Judge, 1997). Critics, however, may state that it fails to account for the possibility that unequal distances between items exist and that it may not accurately depict the value profiles of individuals (Edwards, 1993). In contrast, the main criticism of Likert scales is that this technique is particularly prone to social desirability bias (Kristof, 1996). Yet, these scales – often five-point agreement scales – are easier to administer than Q-sort scaling techniques via a wider range of media and usually take respondents less time (Ten Klooster, Visser, & De Jong, 2008). As a result of their efficiency, Likert scales may be most suitable for employee selection processes. Remarkably, multiple-response measures with dichotomous variables (‘yes’ or ‘no’) as the MVT are not yet reported in PO fit literature. While personality research has often applied two-choice and dichotomous scales in the past (Comrey & Montag, 1982), no instances were found where respondents had to make a carefully weighed decisions in order to ‘pick’ items from a list. Based on measures that are most similar, however, it was found that two-choice personality scales may provide significant problems in terms of reliability; they were generally applied for their ease of use and efficiency as well (Cohen, 1983; Velicer & Stevenson, 1978).
Despite the significant amount of existing PO fit measures, literature on efficient PO fit measures is largely absent except for a limited attempt by Cable and Judge (1996) to shorten the OCP almost two decades ago (Hambleton et al., 2000; Kristof-Brown et al., 2005). This may be the result of the general belief that short measures are inferior to long measures in terms of their psychometric properties (Gosling, Rentfrow, & Swann, 2003). Scholars in similar fields of social research (e.g. personality and leadership), however, have devoted a wide range of studies on this matter. In fact, the overall demand for short measures is generally reported to be growing (Gosling et al., 2003; Rammstedt & John, 2007).
Examples of this trend towards minimal measurement instruments include shortened versions of the Big Five personality measure (e.g., Gosling et al., 2003) and single-item measures for constructs as job satisfaction, ability ratings and cultural identity (Benet-Martínez, Leu, Lee, & Morris, 2002; Robins, Hendin, & Trzesniewski, 2001; Wanous, Reichers, & Hudy, 1997). As predicted by Burisch (1997), many of these minimally short measures show respectable psychometric characteristics, which suggests that a short measure for PO fit may be
sufficiently valid and reliable as well.
3.4. Evaluating the MVT
In the present study, the suitability of the MVT for employee selection processes is assessed by establishing the measure’s overall reliability and criterion-related validity. As the 24 values of the MVT are entirely based the four underlying dimensions of the CVF and
previous CVF measures, it may be expected that the internal consistency of the MVT will be sufficient (Kalliath, Bluedorn & Gillespie 1999a; Ostroff et al., 2005; Quinn & Spreitzer, 1991). Since the MVT measures both preferred and perceived organizational values, it is debatable whether these constructs of preference and perception are unlikely to change over time. However, preferred values are based on one’s personal values and personality (Oliver &
Mooradian, 2003; Schwartz, 1992), while perceived values are based on an organization’s culture (Leung & Chaturvedi, 2011; Schein, 1990). As these personal and organizational values are generally considered as more stable constructs that are only likely to change over extended periods of time (Schein, 1990; Schwartz, 1992), the MVT is expected to have sufficient test-retest reliability. As such, I hypothesized the following:
H1: Based on assessments of internal consistency (H1a) and test-retest reliability (H1b), the MVT is sufficiently reliable.
To determine the MVT’s criterion-related validity, its convergent and concurrent validity will be assessed. First, to assess the measure’s convergent validity, the MVT will be compared to the well-established and widely used 32-item CVF measure (De Clercq et al., 2008; Kalliath et al., 1999b). Because this measure is relatively short in length and it is based on the same theoretical foundation as the MVT, both measures can be effectively compared.
Second, as one of the main purposes of the MVT will be to predict applicants’ behavior by measuring their PO fit, it is important to test the measure’s concurrent validity. Based on findings of previous research on PO fit, three behavioral outcomes were used to test the MVT’s concurrent validity. To start, PO fit was found to have a positive effect on
employees’ overall job satisfaction; employees are more satisfied with their jobs when they fit well with their respective organizations (e.g., Bretz & Judge, 1994; Lauver & Kristof-Brown, 2001; Meglino et al., 1991). Furthermore, it was found that the willingness of employees to recommend the organization they work for increases when there is a high PO fit (e.g., Cable & Judge, 1996). The better a ‘cultural suit’ fits a person, the more likely it is that this person will recommend the tailor to friends; the person becomes a brand ambassador (Elving,
were also found to decrease employees’ intention to leave the organization they work for (e.g., Cable & Judge, 1996; Lauver & Kristof-Brown, 2001; O’Reilly et al., 1991). As a result, it may be expected that organizations with a significant amount of employees with a good PO fit have relatively low voluntary turnover rates (Mitchell, Holtom, Lee, Sablynski, & Erez, 2001). To establish the MVT’s criterion-related validity, I hypothesize the following:
H2: The subjective PO fit scores as measured by the MVT through match count (H2a) and difference scores (H2b) are congruent with the PO fit scores as measured by the CVF measure as measured through difference scores.
H3a: There will be a positive relationship between MVT’s subjective PO fit scores acquired through match count and employees’ job satisfaction.
H3b: There will be a positive relationship between MVT’s subjective PO fit scores acquired through match count and employees’ willingness to recommend their organization. H3c: There will be a negative relationship between MVT’s subjective PO fit scores
acquired through match count and employees’ intention to leave their organization.
The three final hypotheses address the same respective relationships as H3a-H3c, however, H3d-H3f are concerned with MVT’s subjective PO fit scores acquired through difference scores rather than match count. Even though research on comparable measures is significantly scarce, the generally positive expectations for all hypotheses are primarily based on
comparable, efficient personality measures which often acquired sufficient validity and reliability scores (Burisch, 1997; Gosling et al., 2003). In the method section below, I will describe how the previously described hypotheses will be tested in this study.
4. Method
The used methods for finding the extent to which the MVT can be regarded as a reliable and valid and measurement instrument for value congruence in the context of employee selection are outlined below. This section is divided into five subsections; (1) sample, (2) procedure, (3) measures, and (4) statistical analyses.
4.1. Sample
The main study’s sample in this paper consisted of 253 Dutch employees from different organizations. Composing the sample with current employees rather than job seekers may seem counterintuitive as the research problem is embedded in the context of employee selection. However, it is believed that testing the subjective fit rather than the objective fit will still produce a sufficient psychometric assessment of the MVT in the practical context of employee selection, because both are indirect measures of fit and thus highly similar. Yet, using the MVT as an objective fit measure instead may be expected to lead to weaker correlations between PO fit scores and behavioural outcomes, based on the findings of previous research (Cable & Derue, 2002; Leung & Chaturvedi, 2011). This should be taken into account when interpreting the results of the present study.
The employees included in the sample either work for an organization full-time (53%), part-time (20%) or they have an internship or side job (13%). A considerable group of respondents indicated that they recently became unemployed (14%), however, for the
purposes of this study this group was included in the sample as well. Newly unemployed respondents should have sufficient knowledge of their previous organization in order to adequately indicate perceived organization’s values. In terms of gender, the majority of respondents was male (57%). Ages ranged from 19 to 62 with an average age of 39 years (SD = 11.83). Furthermore, organizational tenure ranged from less than 1 year (27%) to over 20
years (6%); values for organizational tenure were well-dispersed over the complete sample. The vast majority of respondents was higher educated (74%).
4.2. Procedure
Data for this study was collected during three different stages; a pilot study, the main study and a retest. At all three stages, the majority of respondents was approached via e-mail. As this study was conducted as part of a graduate internship, the internship organization made its customer database available for data collection. As this organization provides free
supplementary services to a widely used professional online networking platform, it is believed that using the database did not significantly harm the variability of the sample. To avoid customer complaints, the database was filtered for respondents who wanted to receive the organization’s occasional digital newsletters. For the same reason, no follow-up e-mails were sent to increase response rates at any point during this study. Respondents from the database were initially approached via e-mail in the style of the company’s newsletter. Additionally, respondents outside the database were approached via social media and online forums as a way of maximizing sample size and variability (Babbie, 2007). For each stage of this study, a digital questionnaire was used for data collection; all questionnaires were
administered in Dutch. To maximize response rates, a tablet computer was randomly awarded to one respondent of the pilot study and one respondent of the main study and retest. Further information on the procedures for each stage of this study is presented below.
Pilot Study. A pilot study was conducted in two stages to assess whether the MVT could provide measurable data with regards to PO fit. The first stage of the pilot study was conducted to test the initial version of the MVT and establish whether it could provide measurable data with regards to PO fit. In total, 152 randomly selected respondents returned completed questionnaires for the pilot study and four experts reviewed the initial version of
the MVT to establish its face validity – experts were selected based on their background (academic or professional) and their familiarity with theoretical concepts at the basis of this study. Based on the results of this study, 15 values of the MVT were replaced to more adequately measure the four main latent constructs. Furthermore, the wordings of the posed questions were slightly adapted to increase the measure’s validity. Yet, no fundamental changes were made to the measure. In appendix III, an overview of the initially distributed pilot questionnaire and the original MVT can be found.
The second stage of the pilot study was conducted among six additional experts to assess the translational validity of the new version of the MVT. As such, the face validity of this measure was considered to be acceptable. The content validity of the MVT was calculated for each of the measure’s 24 values, using Lynn’s (1986) Content Validity Index (CVI). In table A1 in the appendix, an overview of these values is presented. Fourteen out of 24 items were deemed content valid (mean CVI = .58, p < .05). Yet, the content validity of only two items could be considered convincingly low, namely for ‘optimization’ and ‘sustainability’ which both had a content validity of CVI = .50, p < .05. Based on these results, the content validity of the MVT is insufficient. However, after careful deliberation with the creators of the MVT it was decided to not make any further changes to the measure, mainly because of the exploratory tests’ subjective nature and the measure’s acceptable face validity.
Main Study. Based on the information acquired from the available database, 5976 respondents were approached. Whereas 274 respondents started the questionnaire, only 199 completed questionnaires were returned; a total response rate of just over 3%. While the considerably low response rate would generally be considered as insufficient (Badger & Werret, 2005; Keeter, Miller, Kohut, Groves, & Presser, 2000), it was deemed appropriate for the present study as no direct inferences to the general population are made based on the results. Post-hoc inspection and interviews revealed that the consulted database was poorly
updated and that the digital newspaper format of the invitations may have prevented potential respondents from opening the invitational e-mail. Moreover, it is expected that the decision to not send follow-up mails has also significantly affected the final response rate, as these are generally considered to be of high importance (Babbie, 2007; James & Bolstein, 1990). To strengthen overall validity and reliability claims of this study, a random sample of 54
additional respondents was gathered via social networking sites (e.g., Facebook and Twitter) and online forums.
Retest. In order to assess the test-retest reliability of the MVT, respondents that completed the main study’s questionnaire were asked if they may be approached for a second questionnaire within the next three weeks. Two weeks after initially distributing the main study’s questionnaire, the 202 respondents that expressed their willingness to take part in the second study were approached. This is a generally accepted time interval for retesting and it reflects the procedure of similar studies (Gosling et al., 2003; Waltz, Strickland, & Lenz, 2005). Of the approached respondents, 81 completed questionnaires were returned.
4.3.Measures
Personal Values Preference. Two measures were used to capture employees’ preferred organizational values; the CVF measure and the MTV. In order to measure preferred values with the CVF measure, I used a slightly modified version of the 16-item scale originally developed by Quinn and Spreitzer (1991). The modifications reflect those that were proposed by Kalliath et al. (1999a). This way, the included concepts may more adequately measure the four latent constructs. To measure personal value preferences with the CVF measure,
employees were asked to respond to the statement: “I would like to work for an organization which emphasizes the following values”. For each of the 16 values, responses were recorded on a seven-point scale (ranging from 1 = completely disagree to 7 = completely agree). Using
Likert scales rather than a Q-sort method corresponds with recent studies on PO fit (Kristof-Brown et al., 2005; Leung & Chaturvedi, 2011).
With regards to the MVT, preferred organizational values were measured by one multi-response question, namely: “Which of the following words BEST describe what you look for in your work?”. Hereby, the word ‘best’ was written in capitals to encourage
respondents to make careful considerations in which values they pick. There was no limit for the number of values that could be picked by respondents from the total list of 24 values, however, the minimum was two words in order to acquire a sufficient amount of data per response for calculating PO fit. All items of the CVF measure and MVT were randomized for each respondent to limit response bias (Babbie, 2007).
Perceived Organization’s Values. To measure an organization’s values as perceived by employees, the CVF measure and the MVT were used too. For the CVF measure, the same 16-item measure was used, whereby respondents indicate to what extent the organization they work for reflects each value on a seven-point scale. Employees were asked to respond to the statement: “The following values are representative of my current employer”.
In order to capture perceived organization’s values by employees with the MVT, employees were asked to respond to the question: “According to you, which of the following words fit BEST with your current employer?”. Here, ‘according to you’ was added to clearly instigate a response of perceived values rather than espoused values by the organization. Again, all items of the CVF measure and MVT were randomized for each respondent.
Calculation of Subjective PO Fit. The calculations of the MVT’s PO fit scores acquired through match count and difference scores are shown in equation 1 and 2 respectively. First, the match count equation refers to first multiplying the amount of
matching words from both questions of the MVT by 2. To limit response style bias (Babbie, 2007), the newly acquired value is then divided by the total amount of chosen words from the
first and second question combined. Excluding this step would only allow individuals that pick many values to acquire a high PO fit. The outcome of this equation is the percentage of matching values of all the values that respondents believe fit best with their desired and current organization.
Second, for the difference scores equation, each value was assigned one point. Depending on with how many cultural quadrants a value corresponds (one or two), the respective cultural quadrants acquired a score of 1 or .5 for every picked value. When
calculating PO fit, the scores for each of the four cultures are first divided by the total sum of all four culture scores combined on both the preferred (P) and perceived (O) dimension of PO fit. This way, response style bias is minimized in similar fashion as in equation 1 by
calculating percentages rather than total scores. Then, the sum of the absolute difference between the four respective culture scores of the preferred and perceived organizational values is taken to arrive at the level of PO fit (Drezin & Van de Ven, 1985; Gresov, 1989; Leung & Chaturvedi, 2011). In order to avoid the inverse relationship and only test for direct and positive relationships, the score difference was multiplied by – 1. For the PO fit scores that were calculated through the CVF measure, the same equation was used as by Leung and Chaturvedi (2011). This equation is highly similar to equation 2. However, dividing by the total sum of culture scores was not necessary here, because every respondent provides the same amount of data. Using difference scores to calculate PO fit with the CVF measure as well allowed for effective comparisons between the MVT and the CVF measure.
PO Match Count = (match_count * 2) / (total_amount_chosen_values) (1)
PO Difference Scores = − Σ ABS((perceived_culture_scores / sum_perceived_scores) (2)
Job Satisfaction. As recommended and used in past research, general or overall job satisfaction of employees was measured by three items (e.g., Cable & Judge, 1996; Judge, Cable, Boudreau, & Bretz, 1995). First, the Gallup Poll measure was used to indicate whether respondents are generally satisfied with their job by answering with either ‘yes’ or ‘no’. Second, the non-graphic version of the G. M. Faces Scale was used, as adapted by Scarpello and Campell (1983). Here, respondents rate their general job satisfaction on a five-point scale (ranging from 1 = very dissatisfied to 5 = very satisfied). Finally, an adapted version of the Fordyce Percent Time Satisfied Item was used (Diener, 1984), whereby respondents report the percentage of time they feel happy, neutral, and unhappy with their job on average. However, only one percentage (feeling happy) was used for analysis. As responses were to different scale formats, the three items were standardized before combining them. The internal consistency estimate of this standardized three-item scale was .88.
Intent to Leave. Employees’ intentions to leave an organization were measured by a slightly adapted version of the four-item scale as described by O’Reilly et al. (1991) (e.g., “How long do you intend to remain with this organization?”). In order maintain overall item consistency throughout the survey, the phrasing of each item was changed while maintaining the majority of its original contents (e.g., “I intend to remain with this organization for a long time”). Intent to leave was measured on a seven-point scale (ranging from 1 = completely disagree to 7 = completely agree). To limit response bias, the four statements were randomly presented throughout the survey for each respondent (Babbie, 2007). As responses were to different scale formats, the four items were standardized before combining them. Despite earlier validations (e.g., O’Reilly et al., 1991; Cable & Judge, 1996), the internal consistency estimate of this four-item scale was .61, which is relatively low.
Willingness to Recommend Organization. The willingness of employees to
that you would recommend your organization to friends or acquaintances as a good services and/or products provider?”, and (2) “How likely is it that you would recommend your
organizations to friends or acquaintances that are looking for work?”. This operationalization is consistent with the Net Promotor Score (NPS) measure as described by Reichheld (2003) and previous research (Cable & Judge, 1996). Previous research on PO fit has solely
approached the willingness of employees to recommend their organizations as employer (Cable & Judge, 1996). However, I propose a more holistic view to this concept by
incorporating both the employer brand and the overall brand. As the NPS measure, responses were recorded on a ten-point scale (ranging from 1 = not at all likely to 10 = extremely likely). The internal consistency estimate of this two-item scale was .89.
Control Variables. This study controlled for demographics that could influence respondents’ value preferences or perceived organization’s values. This way, it could be determined whether certain respondent characteristics may significantly influence the measurements as acquired with the MVT. The control variables included in this study are gender, age, education, job type and organizational tenure.
4.4. Statistical Analyses
In order to test the reliability and validity of the MVT, different statistical analyses were used. The MVT’s internal consistency reliability was assessed by computing the Cronbach’s α of the scales underlying dimensions, which is in line with other studies that asses the reliability of dichotomous variables (e.g., Santos, 1999; Stöber, Dette, & Musch, 2002). Test-retest reliability was assessed by correlating scores obtained in the first rating session with the second rating session. In terms of the MVT’s criterion-related validity, Pearson correlation coefficients between the MVT’s and CVF measure’s PO fit scores were calculated; Pearson correlation coefficients between the measures’ underlying cultural quadrants were calculated
as well. Finally, concurrent validity was assessed by computing Pearson correlation coefficients between PO fit scores and each of the three external correlates as used in this study. Below, the results of these statistical analyses are presented.
5. Results
In this study, the psychometric properties of the MVT – a new instrument that measures PO fit in an efficient manner – were assessed. The MVT’s overall reliability was assessed by the measure’s internal consistency and test-retest reliability. First, to assess the MVT’s internal consistency reliability, values for Cronbach’s α were calculated for the measure’s underlying dimensions. An overview of these values can be found in table A2 in the appendix. When differentiating between items that belong to either one or two underlying dimensions, mean Cronbach’s α = .34 for preferred organizational values (P-dimension) and mean Cronbach’s α = .39 for perceived organizational values (O-dimension). However, when combining all values in the four main underlying dimensions, mean Cronbach’s α = .53 and .59 respectively; the internal consistency reliability of each underlying dimension increased. This indicates that the values that belong to two underlying dimensions at least to some extent measure the same underlying dimensions as values that belong to only one underlying dimension. Furthermore, the results show that the MVT’s subscales on the O-dimension have a slightly better internal consistency reliability. Nevertheless, the overall reliability on both dimensions is insufficient as almost none of the MVT’s subscales are nearing acceptable values for Cronbach’s α of above .6 for new measures (Field, 2009; Kline, 1999). In contrast, the values for Cronbach’s α of the CVF measure indicate a good internal consistency reliability, with a mean Cronbach’s α = .75 (P-dimension) and Cronbach’s α = .79 (O-dimension). Here, all subscales show
Second, test-retest reliability of the MVT was assessed in terms of PO fit scores and underlying dimensions scores. The PO fit score acquired through match count yielded a test-retest correlation of r = .48, p < .01, which was slightly higher than the test-test-retest correlation of the PO fit score acquired through difference scores, r = 43, p < .01. The CVF measure had a higher test-retest reliability for PO fit scores, namely r = .56, p < .01. However, this
reliability score does not substantially exceed the reliability scores of the MVT.
In table A3 in the appendix, an overview of test-retest reliabilities is presented for each underlying cultural dimension. When comparing the mean test-retest values between the MVT and the CVF measure, the test-retest reliability of the MVT was found to be generally higher on the P-dimension, whereas the test-retest reliability of the CVF measure was found to be generally higher on the O-dimension. Furthermore, test-retest reliabilities for the MVT range from r = .34, p < .01 to r = .74, p < .01, whereas test-retest reliabilities for the CVF measure range from r = .47, p < .01 to r = .69, p < .01. Herewith, the test-retest reliability of the CVF measure appears to be fluctuate less between underlying dimension scores, which may indicate a slightly higher overall test-retest reliability. Despite minimal differences between the MVT and the CVF measure, based on the results of this study the test-retest reliability of both measures is questionable (Trochim, 2001). However, in comparison with the test-retest reliability of the CVF measure, the test-retest reliability of the MVT can be considered as satisfactory and thus H1b was accepted.
The MVT’s criterion-related validity was assessed by the measure’s convergent and concurrent validity. First, in order to assess the convergent validity of the MVT, the PO fit scores of the MVT were correlated with the CVF measure’s PO fit score. Both PO fit scores of the MVT were positively, moderately correlated with the PO fit scores of the CVF measure. However, the correlation with the MVT’s PO fit scores acquired through match count (r = .40, p < .01) was stronger than the correlation with the MVT’s PO fit scores
acquired through difference scores (r = .34, p < .01). Based on these results, H2a was accepted. However, a more detailed assessment of the convergent validity of the MVT was made by calculating the convergent correlations between the underlying dimension scores of the MVT and the CVF measure; this is solely relevant for measuring PO fit through
difference scores.
The convergent correlations of employees’ preferred and perceived organizational values are shown in table A4.1 and A4.2 in the appendix respectively. In each table, convergent correlations are shown in bold along the diagonal and discriminant correlations are shown on the off-diagonal. With regards to preferred organizational values, convergent correlations (mean r = .27, p < .01) well exceeded discriminant correlations and none of the discriminant correlations exceeded r = .17. Moreover, the majority of discriminant
correlations was not significant. For perceived organizational values, convergent correlations (mean r = .47, p < .01) also well exceeded discriminant correlations in general and here none of the discriminant correlations exceeded r = .36. Here, the majority of discriminant
correlations was significant. Thus, while the convergent correlations are generally stronger for perceived organizational values, the respective distance between convergent and discriminant validities is comparable between P- and O-dimensions. Based on these results, however, the overall convergent validity of the MVT with regards to calculation through difference scores appears to be passable, but generally low. Nevertheless, H2b was accepted.
Second, the concurrent validity of the MVT was assessed by calculating the
correlation between the PO fit scores of the measure and three external correlates. Hereby, I controlled for gender, age, education, job type and organizational tenure. The outcomes of these correlations are presented in table 2. Both MVT’s PO fit scores were most strongly correlated with employees’ overall job satisfaction. Similar correlations were found between MVT’s PO fit scores and employees’ willingness to recommend the organization they work
for. These were all positive, and moderately to strong correlations, whereby the strongest correlation was between the PO fit score acquired through match count and employees’ overall job satisfaction, r = .45, p < .01. Weak negative correlations were found between the MVT’s PO fit scores and employees’ intention to leave the organization they work for. In all instances, the correlations for MVT’s PO fit score acquired through match count (absolute mean r = .39, p < .01) were higher than the correlations for MVT’s PO fit score acquired through difference scores (absolute mean r = .31, p < .01). Yet, all correlations were significant and strong enough to state that the overall concurrent validity of the MVT is acceptable. Nevertheless, all concurrent validities found by correlating the CVF measure’s PO fit scores with the three external correlates exceeded the concurrent validities of the MVT (absolute mean r = .54, p < .01). Still, H3a-H3f were accepted. In the following section, I will reflect on the results of this study as previously described.
Table 2 – Concurrent validity of the MVT and the CVF measure (N = 253) Person-organization fit score
MVT, MC MVT, DS CVF
External correlate
Job Satisfaction .45 .37 .52
Willingness to Recom. Org. .42 .34 .67
Turnover Intention -.29 -.21 -.44
Note. ‘MVT, MC’, ‘Matching Values Test, Match Count’; ‘MVT, DS’, ‘Matching Values Test,
Difference Scores’; CVF, Competing Values Framework measure (PO fit is also based on difference scores); Willingness to Recom. Org., Willingness to Recommend Organization. The values in this table were acquired while controlling for gender, age, education, job type and organizational tenure. p < .01.
6.
Discussion
The purpose of this study was to vastly decrease organizations’ unnecessary loss of time and resources on flawed selection methods for PO fit and wrongfully selected applicants with a low PO fit. To achieve this goal, an analysis was conducted whereby the psychometric properties of a new, efficient measure for PO fit – the MVT – were assessed. This
psychometric assessment determines whether the MVT is sufficiently viable as a predictor of applicants’ PO fit in employee selection processes. The results indicate that the MVT indeed has the potential to be applied in employee selection processes, because sufficient values were acquired for test-retest reliability, convergent validity and concurrent validity. However, a generally low internal consistency reliability clearly indicates that the predictive qualities of the MVT can still be improved. Moreover, the psychometric properties of the well-established CVF measure for PO fit were superior on every psychometric aspect, with the exception of test-retest reliability. With regards to MVT’s PO fit calculations, PO fit acquired through match count had consistently a superior criterion-related validity over PO fit acquired through difference scores. Next, I will explain these findings and provide recommendations for
improving the MVT accordingly.
The low internal consistency of the MVT may primarily be explained by its ambiguous underlying dimensions. Whereas well-established PO fit measures, such as the CVF measure, solely include individual items that measure one underlying dimension (De Clercq et al., 2008), the MVT also includes items that measure two underlying dimensions at the same time. This, by definition, directly harms the unidimensionality of the MVT’s
underlying dimensions (Santos, 1999). Moreover, the number of three items that clearly measure one underlying dimension may be too low, especially for dichotomous variables. Using dichotomous variables to measure a construct may minimize response effort, but it also minimizes the amount of measurement information. An increase of items per each of the four
