N.J. Scheffers
Employers want
big
, and applicants feel
smallExploring the differences of the required education level and job skills in online vacancies, among different sized companies
Abstract
With the cooperation of recruitment companies, we acquired over 25 million online job vacancies in different sectors. In this research we used text mining techniques to extract data; education level requirements and job skills. The extracted information was used to study differences in required education level and job skills among differently sized companies. To create an effective recruitment system we first need to understand how companies use recruitment techniques, this research contributes to that knowledge. We found several results: (1) anno 2015 smaller companies are frequent users of online vacancies, (2) smaller companies ask relatively more often for a higher education level for the same job and (3) there are indications that small companies cause real overqualification, because their demand for higher education is not accompanied by a higher amount of skills. These results are conflicting with existing literature and thus ask for further research. We conclude that there still is a lot that needs to be discovered on the topic of recruitment systems, if we want to improve the process.
Keywords: Text mining, education, job skills, company size and online job vacancies
Bachelor Thesis By: N.J. Scheffers (10206124)
2 | P a g e N . J . S c h e f f e r s
Table of Content
I. Introduction ... 3
II. Theoretical Framework ... 5
2.1 E-recruitment ... 5
2.2 Education level ... 6
2.3 Company size ... 7
2.4 Framework and Hypotheses ... 8
III. Research design and methodology ... 10
3.1 Dataset ... 10
3.2 Variables ... 10
3.3 Analyses and predictions ... 12
IV. Results ... 16
4.1 Usability, Validity and Reliability ... 16
4.2 Results ... 19
V. Discussion... 29
5.1 Summary of results ... 29
5.2 Explanation of the results ... 29
5.3 Weaknesses and strengths ... 31
5.4 Implications and future research ... 32
VI. Conclusion... 34
Literature ... 36
This document is written by Nick Jerome Scheffers, who declares to take full responsibility for the contents of this document.I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it. The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
3 | P a g e N . J . S c h e f f e r s I. Introduction
There are some managers who argue that human capital is the best resource to gain a possible competitive advantage. Gaining the right human capital is not a static process, because human capital is ever changing within each company. People retire, find new jobs or need to be laid-off and new people need to be hired to fill these gaps. Searching for new employees is still mostly done by manually evaluating a lot of vacancies. The financial crisis had a huge impact on the share of unemployed workers and the use of e-recruitment enlarged the recruitment reach for most companies. These are some of the reasons why companies now have a lot more applicants to choose from, which could be a possibility for them. But this immense pool of possible applicants could be too large to handle for many companies. For instance; a study that made use of eye tracking found that recruiters only spend an average of six seconds to study one resume (“Keeping an eye on recruiter behavior”, 2012). When studying the resumes, recruiters use 80% of the time to look at 6 factors: name, previous title/company, current title/company, education, previous position start and end dates and lastly current position start and end dates. An explanation for them doing this could be that they are not able to handle the overload of applicants with similar resumes.
Given the validity of the findings of that study, the inevitable conclusion would be that neither the applicant nor the company benefits from the current system. An applicant with the right skills but with a dissuasive name and a lower level of education might not be able to get a dream job. But also the companies who would like to find the best fit applicant, without the proper resources and/or time, are losers in this situation. The only ones who might benefit, are the companies with the right resources and/or knowledge to fully exploit internet recruitment. Assuming that these companies gained a competitive advantage, this would implicate that they probably are successful companies. So let’s take a look at the mission and vision statements of some top earning companies and see if they have anything to say about human capital. First the statement from the CEO of apple Tim Cook about his company’s doctrine: “We believe in deep
collaboration and cross-pollination of our groups, which allow us to innovate in a way that others cannot. And frankly, we don't settle for anything less than excellence in every group in the company” (Lahinsky, 2009). It seems like Apple does not pick new employees at random, but strives to select only the excellent ones. Next is one out of four mission objectives from one of the largest companies in the world, Royal Dutch Shell; “Employing a diverse, innovative and
4 | P a g e N . J . S c h e f f e r s
results-oriented team motivated to deliver excellence” (Shell; Vision, Mission and Objectives, n.d.). So these two large and successful companies seem to recognize that their employees are drivers for their success.
But how do you find the best job, when recruiters would not even look at your resume? And how do you find the best person for a job when you have an overload of applicants? We think the answer creating a knowledge base for a system where the applicant knows what a company is looking for and in a parallel manner; the company knows what an applicant has to offer. For instance a database programmer who is looking for a job, should know what characteristics companies in his sector are looking for. This would give applicants the possibility to know whether he or she should take an extra course in SQL or Python, to increase the possibility they will get a job offer. On the other hand companies should be stating clearly what they are looking for in a candidate. This opens the possibility to attract candidates who fit the profile, based on their specific skills.
Before such a knowledge base can be of any practical use, we first should get to know more about how companies currently search for employees. As stated above, recruiters use their small amount of time to look at name, work history and education level of applicants. We think asked education level is something companies will use differently, according to their selection strategy. Also the examined mission statements of large companies, made us to believe that the selection strategy depends on company size. This research seeks to find if these two factors have any effect. We will do this by analyzing a large dataset of online job vacancies. Our research question is:
“Is there a difference in asked education level for the same job between large and small sized companies and is the difference in required education level reflected by the difference in asked
skills?“
Answers to this question could help us understand more about which education level and job skills certain companies require from their applicants, and if these requirements are justified. This knowledge could be used to close part of the gap between companies and applicants in their process of finding the perfect match.
5 | P a g e N . J . S c h e f f e r s II. Theoretical Framework
A valuable resource we possessed and needed was our database. This database contained the data from about 25 million of online vacancies acquired from large recruitment companies. Most of the vacancies were semi-structured with certain specific fields, which directly delivered useful lists of variables. The available variables are as follows: start date of the job, organization name, sector, location, country, postal code, position title, position name, type of contract, education level and number of employees in the searching company. Besides these variables each vacancy contained a differing piece of text, which described the available job. Using text-mining techniques we extracted another variable from this content; job skills. In short, job skills are characteristics and/or knowledge the applicant should possess in order to qualify for the job. We will take a closer look at these variables and the research design in chapter three. In the following sections we will discuss the findings of existing literature for several topics relevant for our research.
2.1 E-recruitment
E-recruitment, or internet recruitment, consists of all the internet-based practices used by companies in the recruitment process. Online job vacancies are part of e-recruitment. The use of these practices grew exponentially in the last two decades. In the year of 1998 only 15% of unemployed jobseekers in the U.S.A. used the internet to seek for jobs (Kuhn and Skuterud, 2000), but in 2002 already 22% used the internet as primary method to find a job (Stevenson, 2008). The growth has been substantial and it seems that in 2015, the job market cannot function without the use of internet (Medved, 2014). Before the year 2000, researchers were still looking if internet recruitment was the right choice to make, but they now focus on ways to effectively use it. So why did companies increase their use of internet recruitment over traditional means of recruitment? Already fifteen years ago, Kuhn and Skuterud (2000) knew that internet recruitment had the potential to “dramatically change the methods workers use to search for work”. They saw that it could simplify and increase the search capabilities, and that it could decrease costs. Galanaki (2002) states that internet recruitment could shorten the recruiting cycle time and so reduce the time of labor which is needed. He also states that internet recruitment provides a more global coverage compared to traditional recruitment forms. Despite its growth, online recruitment also has some drawbacks. It can only be useful if it’s applied as part of the recruitment process, this means that companies need to be ready to deal with the needed IT tools (Galanaki, 2002).
6 | P a g e N . J . S c h e f f e r s Secondly the implementation of internet recruitment increases the risk of an overload of applicants, if there is no proper pre-selection by computer. Bartram (2000) agrees with this argument by saying that internet based recruitment has helped many companies to attract applicants, but not yet by improving the process of selection. In Cappelli’s (2001) and our opinion, the most important drawback of internet recruitment is that it enhances the risk of unwanted discrimination. Discrimination occurs when someone bears a loss because a distinction is made between individuals or groups, which is not acceptably relevant for that situation. Cappelli means by discrimination; the risk companies suffer when automatized selection procedures accidentally filter out certain groups or individuals. Besides this risk, we think that by knowing or unknowingly using discriminating online selection procedures, both applicants and employers suffer certain forms of damages. We will elaborate on this in the next section.
2.2 Education level
In introduction we already saw that education is one of the variables that recruiters take time to look at when studying vacancies (“Keeping an eye on recruiter behavior”, 2012). The importance of this variable for recruiters is also confirmed by the initial analyses of the database; most of these vacancies contain required minimum education level. You might also have experienced this requirement of a certain education level when searching for a new job and you might expect that there has to be a logical reason for this. Using education level as a selection method could for instance ensure later job performance. Schmidt and Hunter (1998) wrote an important article about the validity and utility of certain selection methods. One variable that they found to be of almost no relevance in predicting job performance is education. So if it is not a clear predictor of performance, maybe it predicts job satisfaction? Allen and Van der Velden (2001) explored the relation between education mismatches and skill mismatches and their effect on wages and job satisfaction. Assignment theory suggested that these two concepts should be closely related; the one causes the other and vice versa. But their results showed that the one exists where the other doesn’t, and that each mismatch has its own effect on wage and job satisfaction. Skill mismatches are a better predictor for job (dis)satisfaction than education mismatches. Education mismatches on the other hand are a better predictor for higher wage than skill mismatches. The most important implication from this paper is that future research should make a careful distinction between education and skills. This distinction between skill and
7 | P a g e N . J . S c h e f f e r s education is also backed up by the meta-analysis on the measurement of over and under education by Groot, Van den Brink (2000). So if education level is not a good predictor for either job performance or satisfaction, both the applicant and the employer suffer either social or economical of damage. This is because the applicant cannot reach the highest amount of possible job satisfaction and the employer cannot reach the applicant with the highest job performance. In this case they bear a loss during the recruitment process because a distinction is made based on education, which seems of no relevance for that situation. More or less, this touches the definition of discrimination. So if it causes damages, why is this overqualification still present? Green and Zhu (2010) did research on the effects of overqualification in Britain. They made a distinction between ‘Formal’ and ‘Real’ overqualification, the second form only occurs when underutilization of education is accompanied by underutilization of skill. Their results are very similar to the ones mentioned before. Real overqualification is predicting job dissatisfaction, while formal isn’t. They expect that the increase of overqualified workers in Britain, are partially the result of changing demographics but mostly explained by a lack of change in the educational system. They propose that the educational system should become more transparent, so students know in advance which elements of education will pay off in the long-term.
2.3 Company size
As stated in the last section Allen and Van der Velden (2001) found that mismatches between job and education level are a good predictor for relatively higher wage. But which companies are attracting overqualified applicants and thus paying higher wages than needed? Brown and Medoff (1989) found that there is a positive relationship between company size and wages. They cannot completely explain the effect they witnessed, but they do have several smaller explanations for some of the positive relationship. First they found that larger companies try to attract higher quality workers. Secondly, larger companies compensate inferior working conditions with higher wage. Lastly, it seems like larger companies are more enabled to pay these higher wages. However when the effect is corrected for these three variables, there is still a large proportion of unexplained variance. The search for higher quality employees could mean that larger employers also ask for higher education levels compared to smaller ones. Evans and Leighton (1989) did some research on the opposite effect; why do smaller companies pay less. First they found that indeed wages increase with firm size. But they also found the effect we were
8 | P a g e N . J . S c h e f f e r s expecting; larger companies have a workforce with a higher level of education. They argue this is happening because smaller companies are less stable and lack the amount of resources larger companies have, which is needed to hire higher level of employees. In a more recent work Galanaki (2002) argues that these differing amount of resources and capabilities, are the reason why larger companies are more successful in implementing internet recruitment. Concluding we could say that company size seems of relevant effect when researching education level during the recruitment process. We hope to find similar effects of company size, when looking at differences in required job skills.
2.4 Framework and Hypotheses
In the last sections we took a closer look at the subjects we ought to study in our research. Some possible relations between the variables became clear and they can be formulated into hypotheses. First we concluded that larger companies could engage more often in internet recruitment, because of their resources.
H1: “Larger sized companies use internet recruitment more often than smaller sized companies”
We also saw that the results of multiple researches showed that larger companies have a higher degree of overqualification. We expect that this overqualification also will be present in our database, regarding the level of education.
H2: “Larger sized companies seek higher levels of education for the same job than smaller sized companies”
But regarding this overqualification or mismatches, we should keep in mind that there is a difference between ‘Real’ and ‘Formal’ mismatches on the job. The first is the one who imposes serious problems for employees. While larger companies seem to be a host of inferior working conditions, we hypothesize that larger companies also will not fulfill the needs of employees regarding their skills.
9 | P a g e N . J . S c h e f f e r s
H3:”Larger sized companies seek a lower number of skills for the same education level as small size companies”
The variables and these hypotheses can be displayed in the following framework displayed in figure 1.
Figure 1: Hypotheses framework Internet
Recruiting
•H1: “Larger sized companies use internet recruitment more often than smaller sized companies”
Level of Education
•H2: “Larger sized companies seek higher levels of education for the same job than smaller sized companies”
Number of Job
Skills
•H3:”Larger sized companies seek a lower number of skills for the same education level as small size companies” Company Size
10 | P a g e N . J . S c h e f f e r s III. Research design and methodology
In this chapter we will elaborate on the methodological parts of this research. Starting with the collection and refining procedure of our dataset. Then we will look at the construct and measurement of the variables used during the analyses. Lastly we will explain the procedure we followed during the analyses and give our results.
3.1 Dataset
Our dataset was created with the help of recruitment companies, together they supplied around 25 million online job vacancies. UWV is governing Dutch unemployment benefits and tries to help people find new jobs. Our research on employer behavior in vacancies may help them to improve that process. All the vacancies were offering jobs in the Dutch labor market and located in four sectors: ICT, caregiver, hospitality and nursing. The vacancies were semi-structured and directly delivered us a number of variables: start date of the job, organization name, sector, location, country, postal code, position title, position name, type of contract, education level and number of employees in the searching company. Besides these variables from the structured part of the vacancies, we also managed to extract the variable job skills from the unstructured job content using text mining.
3.2 Variables
3.2.1 Job skills
This variable was not directly given by the employer and needed to be extracted from the job description in each vacancy. But what did we define as a job skill? In their research Litecky, Aken, Ahmad and Nelson (2010) did a similar analysis on job skills in vacancies in the IT industry. They selected various technical, programming, business, and soft skills, which were needed for the job. This used technique is called the ‘bag of words’ technique, which simply counts the times each word is present in a piece of text. However simple, it seems to work in a lot of situations if it is used correctly (Tirilly, Claveau and Gros (2008). We did a similar thing with our data. The extraction of all job skills was done by using the following steps. First, the vacancies of each sector were collected in a separate file. Each file contained a list of all separate sentences from the job descriptions. Each of these sentences was marked with a unique number
11 | P a g e N . J . S c h e f f e r s from the vacancy it belonged to. Next we manually labeled around 10.000 sentences of each sector by giving them a certain number. These numbers indicated that a sentence contained a job skill, another variable or neither of them. The list of sentences which contained a skill was then refined, so they could be used to extract the skills from the remaining sentences. First all unique words from the selected sentences were formatted in a list. Then we manually selected the unique skills from this list, belonging to the sector. This selection of unique skills was then transformed so it only contained lower case letters and no unnecessary punctuation such as ‘;’. This modification was needed because many skills were written in different ways. An example: sql, SQL, msSql, MSsql etc. We then removed skills which would be detected as part of other non-relevant words, for instance the skill C was detected in each word starting with an C. Removing all of these short skills could affect the results, so we added new variables in which the skill would be only detected if it was found as a stand -alone character. For the example of “C” this means that we created the new job skill “space C space”. With these adjustments we created the final lists, which were used for the tables to run the analyses. We will further discuss these tables in section 3.3.
3.2.2 Education level
The minimum level of education was given by the employer according to the ISCED code. This code ranges from 0 to 8 and had the following values (Schneider, 2013): 0 = early childhood education, 1 = primary education, 2 = lower secondary education, 3 = upper secondary education, 4 = post-secondary non-tertiary education, 5 = short cycle tertiary education, 6 = bachelor degree or equivalent, 7 = master degree or equivalent and 8 = doctoral or equivalent. This codification made this variable usable as a quantitative ordinal scaled variable. This variable did not need any further adaptations for this research.
3.2.3 Company size
The last variable was company size; this variable was used to divide the dataset in subgroups. To create these subgroups, we had to define what a large or small company exactly is. When we looked at existing literature, we found differentiating numbers. For example; Hart and Oulton (1996) used 8 employees as a border between large and small, but Evans and Leighton (1989) saw companies with less than 1000 employees as small. Our definition was calculated by
12 | P a g e N . J . S c h e f f e r s analyzing the available dataset. In the dataset the number of employees was filled out by the employer regarding the choice options; 1, 2, 5, 10, 20, 50, 100, 200, 500, 750 and 1000. Then we created a table which showed the number of vacancies for each job function name in a certain company size interval (see table 2). The border(s) between the two or multiple classes were chosen in such a way that each subgroup contained around 5% of the vacancies.
3.3 Analyses and predictions
When we started this research, there was no other research which could help guiding our specific analyses in advance. This resulted in the fact that a large part of the labeling, transforming and analyzing was based on trial and error. However, we did create some sort of ‘roadmap’ in advance. This was done by walking the road backwards; we started with the ideal results and then deducted what we needed to do to get there.
First we had to find out which specific business sector(s) would be usable for our research. Each sector would need to be of sufficient size for analyses; this was the case for all four available sectors. Each sector was even of such size that we would not be able to analyze all four, this because we had a limited amount of time and processing capacity. When choosing the sectors we were going to use, we estimated which one would give us the greatest chance in finding relevant results. The first sector we chose was ICT. This sector showed a varying demand in the level of education and possessed numerous knowledge skills which would be easier to extract than characteristics. For instance ‘sql’ is easier to find than ‘committed’, while this words needs a lot of stemming and can still be used in a non-skill sentence; ‘we have committed customers’. We wanted to analyze a second sector, mainly because this could greatly enhance the reliability of our research. The second sector we chose was nursing, this choice had a number of reasons. First, similar to the ICT sector nursing showed a promising deviating in education level. Secondly, this sector had a larger range of employer size than the two remaining sectors.
Now that we had selected the sectors, we could start refining the data into a usable dataset. First we took the codex we found during the labeling and processing, also see section 3.2.1, and put them in a table with all available sentences. We then used the computer to find whether or not each sentence contained one or more of the skills we sought. Before we were able to do this, the sentences got the same treatment as the skills (lower case font and removal of unnecessary punctuation). Now that we knew for each sentence which and how many skills it
13 | P a g e N . J . S c h e f f e r s contained, we combined all sentences and their skills belonging to a certain vacancy. This table was combined with the meta-information from each vacancy; size of the employer and the level of education (see table 1).
Company Size Education Level
Total
Skills Skill 1 Skill 2 …
Vacancy
1 800 4 15 1 1 …
Vacancy
2 100 6 9 0 1 …
… … … … …
Table 1: Skills and vacancies
In their research on job skills Litecky, Aken, Ahmad and Nelson (2010) excluded the vacancies with either too much or too few skills. They also excluded skills which either occurred in too many or too few vacancies. We followed their example and excluded some of the vacancies and skills for each sector.
H1: “Larger sized companies use internet recruitment more often than smaller sized companies”
To answer our first hypothesis we plotted the vacancies according to their company size in a histogram. This enabled us to tell if larger companies were more present in our dataset than smaller ones.
H2: “Larger sized companies seek higher levels of education for the same job than smaller sized companies”
To answer our second hypothesis, we first had to create appropriate subgroups based on company size. We did this by creating a table in which the number of vacancies was counted, depending on their education level and company size (see table 2).
14 | P a g e N . J . S c h e f f e r s Vacancy Count Education Level 1 2 …. 7 8 Size Small 0 10 … 40 10 Medium 1 15 … 50 1 Large 0 40 … 80 40
Table 2: Vacancy Grid
The goal of creating this table was to create groups that contained around 5% of the total vacancies, this way each group would stay statistically significant. This was achieved by combining several education level groups and by changing the division between company sizes. To check if larger companies ask a higher level of education for the same job, we created a new table containing job names and sorted them by using the size of companies (see table 3).
Size Small Large
Education 4 5 6 4 5 6
Job Name Name 1
Name 2
…
Table 3: Job name, size, education level grid
Each grid then displayed the counted number of vacancies for each group of company size and education level. The data from this table was viable for a chi square analysis. This analysis is able to check if two populations are relevantly different, and thus could tell if there was a difference between large and small companies.
H3:”Larger sized companies seek a lower number of skills for the same education level as small size companies”
To test our last hypotheses we used the format of table 3 again. Instead of counting the number of vacancies, we now counted the average number of asked skills. Each group could then be compared with the similar group of another size; this was done using the standard deviation.
15 | P a g e N . J . S c h e f f e r s We predicted to find positive results for all our hypotheses in both sectors. However we also expected that the results in the ICT industry would show a greater effect, this because in this sector the collection of skills would be more reliable to collect. Also we expected that company size and level of education would fluctuate more in the ICT sector.
16 | P a g e N . J . S c h e f f e r s IV. Results
In this chapter we will go through the results step by step. First we will look at the usability, validity and reliability of the research. Then we will look at the calculations and actual results of the research. These results will be discussed according to the order of hypotheses.
4.1 Usability, Validity and Reliability
For our research we made use of online job vacancies. The most important benefit of using these online vacancies are their usability. When compared to a regular research instrument (such as; questionnaires or interviews), our dataset is a lot easier to collect and can become as large as wanted. Using direct data as a source helps disposing the problems of instrument usability. Moreover, regular research instruments need to be examined on how much time the data collection is going to take and if there are sufficient resources to do so. Also the use of online vacancies as a source makes it easier to run analyses digitally.
When looking at the validity of online job vacancies as an instrument, we cannot really determine a number which would prove that it is a valid variable to use. This is caused by the fact that we lack the right knowledge about the population. This knowledge is not so easy to collect because it depends on what we would define as our population. Did we make a valid sample of the total job market, the total ICT job market, the total online vacancy market or only the online ICT vacancy market? We can however look at the internal validity of the separate components: company size, education level and job skills. The company size in our dataset was not represented by an exact number, but by an interval of sizes between 0 and 1000. This interval affected the accurateness of company size, because companies with 0 employees do not exist and companies larger than 1000 fell off the charts. The alternative would be to find the exact number of employees for each company, but with such a large database this was close to impossible.
The level of education was represented by the ISCED code (see chapter 3.2.2). Defining what level of education someone has reached cannot be fully objective, because there are a lot of differences between educational forms in each country. The ISCED code is in our opinion the closest ranking you get, when looking at international educational formatting.
The variable job skill was created by using the ‘bag of words’ method (see chapter 3.2.1). Despite the crudeness of this method, research showed that it seems to work for a lot of different tasks. By manually selecting which is and which isn’t a job skill, we ran the risk of affecting the
17 | P a g e N . J . S c h e f f e r s dataset in a bad way. But there was no alternative which gave us the same possibilities, with such little means.
When we would find any relevant results in this research, the question is to what extent it tells us something about the total population. By selecting multiple sectors we tried to exclude the possibility that our findings are only there by chance. But we expect that further research will be needed to really ratify eventual results.
The reliability of our research was tested during several stages for both sectors. To test the reliability we counted three different errors (see figure 2). We defined these errors as follows: a class one error was an error where a job skill was not detected when it should have been, a class two error occurred when a job skill was not detected because the skill couldn’t be incorporated by our model (skill contained only 1 letter etc.) and a class three errors occurred when skills were not detected because they consisted out of a string with coherent meaning. The first class error is the one which was most important for our reliability tests, because these were the skills our model should have recognized.
Error Sentence
#1 JFD23561406 “Requirements: CCU nursing license” #2 JFD25130799 “Competent in ORM”
#3 JFD23542391 “Experience in a clinical setting with a complex target group” Explanation
#1 CCU should be detected, but was not filtered during labeling
#2 ORM could not be incorporated, ORM is also part of the word ‘normal’ #3 Sentence contains a skill, but not a single skill keyword
Figure 2: Error example and explanation
ICT 1:
After we collected and refined the ICT’s sector unique bag of words, we ran our first reliability test (see table 4). We used the bag of words on the 1174 manually labeled sentences, which should contain a job skill according to the labelers. We randomly took 100 sentences which contained 0 skills according to our model, and counted which errors occurred. The three class 1 errors we found in this test were only present in 7 out of the 1174 sentences, which was less than 1% per missed skill.
18 | P a g e N . J . S c h e f f e r s Reliability tes ICT
1
Class 1 Class 2 Class 3
Errors 3 7 42
Table 4: Reliability ICT 1 ICT 2:
The second moment that we checked the reliability of the ICT sector was when we finalized the list with all vacancies and their count of job skills. Again we randomly selected 100 sentences, about 5 complete vacancies, and checked for the three classes of errors (see table 5).
Reliability ICT 2
Class 1 Class 2 Class 3
Errors 3 3 10
Table 5: Reliability ICT 2
Nursing 1:
For the nursing sector we ran the same reliability tests. We used the bag of words on 850 labeled sentences which should contain a skill and then we randomly selected 100 of the
sentences which contained 0 job skills according to our model. The four class 1 errors were only present in 14 of the 850 sentences, less than 1% per missed skill (see table 6).
Reliability Nursing 1
Class 1 Class 2 Class 3
Errors 4 1 33
Table 6: Reliability Nursing 1
Nursing 2:
The second moment that we checked the reliability of the nursing sector was when we finalized the list with all vacancies and their count of job skills (see table 7). Again we randomly selected 100 sentences, about 5 complete vacancies, and checked for the three classes of errors.
19 | P a g e N . J . S c h e f f e r s Reliability Nursing 2
Class 1 Class 2 Class 3
Errors 4 0 8
Table 7: Reliability Nursing 2
All tests indicated that our model was functioning very well for such a simple method. When we looked at the class one error count, we could conclude that our model found at least 95% of the job skills in each sentence.
4.2 Results
About the ICT dataset
The most important task of collecting the data for the ICT sector, was extracting the relevant codex of job skills. With the available resources we extracted about 181.000 sentences from vacancies in the ICT sector. From this set we manually labeled 10.000 of them and marked sentences which contained job skills. We collected 1174 sentences containing job skills. These sentences consisted out of a collection of 1606 unique words. We manually selected 270 of these words and marked them as job skills. We refined this list by transforming uppercase letters to lowercase and excluding certain punctuation. This list was used to form a table in which we plotted the 1174 sentences. We searched for unwanted results, such as hidden duplicates (for example: visual and basic instead of visual basic) and checked the reliability. The remaining list of 252 unique skills was then used to analyze all 180.000 sentences. When all sentences were analyzed, we combined them back into individual vacancies. We managed to collect 4108 unique vacancies, each containing at least 1 job skill. The top 10 asked job skills in these vacancies are listed in table 8.
20 | P a g e N . J . S c h e f f e r s
Rank Skill Count # Percentage %
1 .net 3387 82% 2 hbo 2965 72% 3 c 2341 57% 4 sql 2074 51% 5 nim 2065 50% 6 msc 1987 48% 7 asp 1908 46% 8 erp 1652 40% 9 zelfstandig 1295 32% 10 microsoft 1243 30%
Table 8: Top 10 job skills in ICT
As a last step we deleted all vacancies which contained either no employee count or no educational level, we now had 2552 vacancies left for analyses. Of these vacancies the mean and median number of employees were μ = 146,12 and median = 10. And the mean and median level of education was μ = 4,75 and median = 5. The skills per vacancy had a mean and median of μ = 14,41, median=14. At this point we ran our second reliability test.
# Job skills Employees Educ. Level
Mean 14,41 146,12 4,75
Median 14 10 5
Table 9: Mean and median in ICT
About the nursing dataset
For the nursing dataset we followed the same process as in ICT. We managed to extract 173.000 sentences from vacancies in the nursing sector. From this set we manually labeled around 9000 sentences and we identified 850 sentences containing a job skill. These sentences consisted out of 1108 unique words, from which we manually selected 137 as a job skill. At this point we ran our first reliability test. After refining and searching for unwanted results, 124 skills remained. This remaining codex of job skills was then used to analyze all 173.000 sentences. After analyzing and compiling, we managed to find 3994 vacancies consisting at least 1 job skill. The top 10 asked skills in the nursing sector are listed in table 10.
21 | P a g e N . J . S c h e f f e r s
Rank Skill Count # Percentage %
1 zelfstandig 2508 63% 2 flexibel 2306 58% 3 big 2108 53% 4 verantwoord 1981 50% 5 niveau 4 1841 46% 6 specialist 1626 41% 7 hbo 1486 37% 8 enthousiast 1410 35% 9 mbo 1391 35% 10 betrokken 880 22%
Table 10: Top 10 job skills in nursing
We then deleted all sentences containing either no employee count or educational level, the remaining dataset contained 3501 vacancies. Of these vacancies the mean and median number of employees were μ = 656,35 and median = 1000. And the mean and median level of education was μ = 3,53 and median = 3. The skills per vacancy had a mean and median of μ = 8,37,
median = 8. At this point we ran our second reliability test.
When we compare the two datasets we see that, in the nursing sector more companies filled out the minimum education level and/or the employer size. The average company in nursing is much larger, asks lower level of education and a lower number of skills. Also when you look at the top asked skills, you see that in ICT the ‘hard’ skills are dominant while in nursing the ‘soft’ skills are important.
# Job skills Employees Educ. Level
Mean 8,37 656,35 3,53
Median 8 1000 3
Table 11: Mean and median in nursing
Checking hypothesis 1: Large companies use internet recruitment more often
To test our first hypothesis we plotted the number of vacancies for each company size of both the ICT and nursing sector (see figure 3 and 4).
22 | P a g e N . J . S c h e f f e r s Figure 3: vacancies by company size in ICT
Figure 4: vacancies by company size in nursing
When we looked at the results of both sectors, we could conclude that hypothesis 1 couldn’t be supported based on the results in this dataset. This while in the ICT sector, the larger companies were not delivering more vacancies than the smaller ones. And in the nursing sector almost all vacancies came from the 1000+ sector, this is probably the case because nursing jobs are mainly created within large hospitals. Besides the high amount of vacancies within this largest class, there was no noticeable regression in the other vacancies such that it indicated our hypothesis 1 could be confirmed.
1000 750 500 200 100 50 20 10 5 2 Vacancies 235 61 44 179 157 113 302 500 182 263 0 100 200 300 400 500 600 # Vac an ci e s
Vacancies by Company Size in ICT
1000 750 500 200 100 50 20 10 5 2 Vacancies 2142 87 52 156 210 111 245 90 91 317 0 500 1000 1500 2000 2500 # Vac an ci e s
23 | P a g e N . J . S c h e f f e r s
Checking hypothesis 2: Large companies ask higher levels of education
To answer our second hypothesis, we created appropriate subgroups based on company size. We did this for both sectors by creating a table in which the number of vacancies was counted depending on their education level and company size (see tables 12 and 13).
Subgroup
ICT Education Level
2,3,4 5 6 Total Size 200 to 1000 5,80% 19,45% 0,25% 25,49% 20 to 100 3,00% 24,21% 0,88% 28,09% 2 to 10 4,67% 40,62% 1,13% 46,41% Total 13,46% 84,28% 2,26% 1
Table 12: Subgroup size in ICT
Subgroup
Nursing Education Level
2,3,4 5 6 Total Size 1000+ 46,79% 13,88% 0,51% 61,18% 50 to 750 12,60% 4,74% 0,26% 17,59% 2 to 20 14,45% 6,57% 0,20% 21,22% Total 73,84% 25,19% 0,97% 1
Table 13: Subgroup size in nursing
In the ICT sector, education level 2 and 4 were only present in less than 20 vacancies and thus we directly combined them with level 3. Education level 6 was not sufficiently present in each subgroup and thus combined with 5. The lowest combined group in the ICT sector was that of medium sized company in lower education (3%). We did however maintain the medium sized subgroup because the subgroup was present in an absolute of 61 vacancies. Also this middle subgroup could prove important for comparison in our research, instead of only comparing two groups. In the nursing sample we combined educational level 2, 3 and 4, because of their small individual size. We did the same with educational level 5 and 6. Each subgroup then added up to above 5% each.
24 | P a g e N . J . S c h e f f e r s We then checked for the ICT sector if there were differences in asked education based on function name, sorted by company size. Only one job function (.NET programmer) was present in all subgroups and thus viable for analyses (see table 14).
Small Medium Large
Function 2 to 4 5 to 6 2 to 4 5 to 6 2 to 4 5 to 6 Total
1 95 795 61 430 118 300 1799
2 0 55 0 79 0 101 235
3 0 0 0 2 0 0 2
Total 95 850 61 511 118 401 2036
Table 14: Vacancy count per subgroup ICT
Factor between high and low education ICT
Small Medium Large
Function 1 8,37 7,05 2,54
Total 8,95 8,38 3,40
Table 15: Factor between high and low education ICT
When we look at the factor between high and low education we see the opposite from what we were expecting (see table 15). How larger the company, how lower the relative frequency of higher education for the same function. This observation was also ratified by the chi square test for this specific job function (X^2=72,02 ; df=2), which reveals that there was less than .5% chance that these groups belonged to the same population.
The nursing sector showed different results. From this dataset more function names were present in all subgroups and thus viable for analyses (see table 16).
25 | P a g e N . J . S c h e f f e r s
Small Medium Large
Function 2 to 4 5 to 6 2 to 4 5 to 6 2 to 4 5 to 6 Total 1 419 184 289 118 1200 300 2510 2 41 5 50 10 148 33 287 3 16 20 22 23 99 43 223 4 3 10 28 8 41 31 121 5 10 7 8 3 84 28 140 6 2 3 14 6 9 32 66 7 7 3 15 3 25 14 67 8 7 2 8 2 16 15 50 9 0 0 5 0 0 1 6 10 0 0 0 0 6 0 6 11 0 1 0 0 3 2 6 12 0 0 1 1 2 3 7 13 1 1 1 0 5 2 10 14 0 1 0 0 0 0 1 15 0 0 0 1 0 0 1 Total 506 237 441 175 1638 504 3501
Table 16: Vacancy count per subgroup nursing
Factor between high and low education ICT
Small Medium Large
Function 1 0,44 0,41 0,25 2 0,12 0,20 0,22 3 1,25 1,05 0,43 4 3,33 0,29 0,76 5 0,70 0,38 0,33 6 1,50 0,43 3,56 7 0,43 0,20 0,56 8 0,29 0,25 0,94 Total 0,47 0,40 0,31
26 | P a g e N . J . S c h e f f e r s In the nursing sector we saw that a lower degree of education is asked relatively more often. But also in the nursing sector we saw the same results as in the ICT sector; how larger the company, how lower the relative frequency of higher education for the same function (see table 17). This observation was also ratified by the chi square tests for the three largest functions (see table 18). So in both sectors our hypothesis 2 couldn’t be confirmed and did it seem like the opposite of our expectation was true.
df=2
Function ChiSquare P-value 1 7,86 >0.02 2 8,14 >0.02 3 12,28 >0.0025
Table 18: ChiSquare of 3 largest functions in nursing
Checking hypothesis 3: Large companies ask a lower amount of skills
To check our third hypothesis we used the same table format as in our last hypothesis, but now we counted the average number of asked skills instead of the number of vacancies (see table 19). For the ICT sector this resulted in the following data.
Small Medium Large
Function 2 to 4 5 to 6 2 to 4 5 to 6 2 to 4 5 to 6
1 13,22 12,99 14,95 15,78 14,34 16,53
2 14,51 15,13 13,65
3 21,50
27 | P a g e N . J . S c h e f f e r s Groep StdDev Small 5,57 Medium 5,90 Large 5,39 2 to 4 5,61 5 to 6 5,77 Total 5,75
Table 20: Standard deviation ICT sample
Table 19 indicated that medium sized companies asked relatively more skills than small sized companies and large sized companies slightly asked more skills than medium sized companies. This effect however couldn’t be statistically proven. The standard deviation was relatively high, which made the differences statistically insignificant (see table 20). The largest difference in this sector was 16,53 - 12,99 = 3,54 , this fell beneath the standard deviation. We ran the same analysis for the nursing sector.
small medium large
Function 2 to 4 5 to 6 2 to 4 5 to 6 2 to 4 5 to 6
1 8,48 9,10 8,86 9,11 8,02 7,56
2 7,27 7,60 9,18 11,70 9,95 9,03
3 9,19 8,60 8,41 9,30 10,30 7,40
Total 8,39 8,70 8,57 9,07 8,37 7,77
28 | P a g e N . J . S c h e f f e r s Group StdDev Small 3,34 Medium 3,64 Large 3,21 2 to 4 3,26 5 to 6 3,51 Total 3,33
Table 22: Standard deviation nursing sample
In the nursing sector the results showed another effect than the ICT sector. Here larger companies seemed to ask for a slightly lower amount of skills, but the differences are nihil (see table 21). Also in this sector the differences fell beneath the standard deviation, and thus we couldn’t statistically prove that there was a significant difference (see table 22). We could conclude that with our dataset, hypothesis 3 couldn’t be confirmed.
29 | P a g e N . J . S c h e f f e r s V. Discussion
5.1 Summary of results
In short, we found relevant results to one of our three starting hypotheses. Firstly, we were not able to determine whether larger companies are more frequent users of online job vacancies. It seems that this effect is different for each sector. According to our data it seems that in the ICT sector smaller companies are more often using online job vacancies. In the nursing sector most of the vacancies were belonging to very large companies, which made it impossible to run proper analyses. Secondly, we did find an effect between different kinds of company sizes and their education level demand for the same job. We predicted that larger companies would relatively ask higher education for the same job, but we found the opposite effect. Both in the ICT sector and in the nursing sector it seemed that the group of smaller sized companies asked for a higher level of education relatively more often. Lastly we hypothesized that larger companies would ask a relatively lower amount of skills for the same level of education, because they were offering inferior working conditions. We were not able to find an effect of this sort in either sector. In the nursing sector it seemed that all company sizes asked for about the same amount of skills and in the ICT sector it even seemed that smaller companies were the group who asked for a relatively lower amount of skills. These results would indicate that smaller companies are the group who create real skill mismatches; however we couldn’t statistically prove these indications with our dataset. In the next section we will try to find explanations for the observed relations.
5.2 Explanation of the results
It was unexpected that we found no results indicating the higher use of online job vacancies by larger companies. For the ICT sector it even appeared that smaller companies were the most frequent users. The first possible explanation we have is that it is caused by the sources from which we collected our dataset. These sources were recruitment companies, both governmental and privately owned. It could well be possible that larger companies use other resources to attract applicants, such as headhunters. For smaller companies the choice for a recruitment company could be more likely, because of the relatively large reach they have. A second explanation for the effect is that the resources Galanaki (2002) spoke off, are less
30 | P a g e N . J . S c h e f f e r s important anno 2015. A lot has changed in the past decade and it is certainly possible that the efficiency of posting job vacancies online has become available to smaller companies. The internet is more transparent, online recruitment companies have grown in size and almost every company has its own website or social media platform. Hart and Oulton (1996) already found twenty years ago that larger companies do not always create the most jobs, but that it’s dependent on the state of the economy. Perhaps nowadays we are in an economical state where small companies are searching more for new employees than large companies. The last explanation we have is concerning the construct of our variable company size. This variable was not created by the observation of an objective scientist, but by hundreds of different individuals. It is statistically inevitable that a proportion of the data was filled out in the wrong way. They could for instance have seen company size as department size or they just filled out a random number. Without an existing dataset it would however have been impossible to manually search the company sizes for all 7500+ companies, just to possibly increase the validity of the variable. We still think our results are therefore relevant.
The second unexpected result that we found was the asked education level for the same function name. We expected large companies would demand a higher degree from employees and thus would be asking for relatively higher levels of education. The opposite seemed true in both our analyzed sectors. In the ICT sector we found that there was a relatively higher demand for higher levels of education in all different groups of company size, this is expected regarding the fact that these jobs are knowledge based. But the factor between high and lower education was almost three times as high in the smaller company sizes as in the larger company sizes. In the nursing sector we found that there is a relative higher demand for lower levels of education, which is evident because the jobs are skill and less knowledge based. But when we looked at the factor between the demands of high and low education across different company sizes, we found that small companies often seek higher levels of education for the same function compared to large companies. This contradicts with the existing theory and with our own hypothesis. But what could be the reason for such a contradictory finding? The first explanation follows the thinking route of the first explanation in the last section. It could well be possible that larger companies use other resources than recruitment companies to find their employees. It is especially plausible that they search the most important and highly educated employees using other techniques, such as headhunters or special selection programs. This explanation would also be supported by the
31 | P a g e N . J . S c h e f f e r s resource theory of Galanaki (2002); larger companies will use their resources to use more exclusive techniques for hiring their employees. In 2002 this meant that they were able to fully exploit the internet and its possibilities, perhaps now this has shifted to different techniques. A second explanation for the observed effect could be that in fact it are the smaller companies who are in need of higher educated staff. Globalization and the massive use of internet in all sectors created a situation in which more companies are competing for the same market. In order to keep up with larger competitors, smaller companies need to deliver the same results and quality, but with less employees. Perhaps this is the reason why smaller companies seek employees who have more potential and thus a higher level of education, just to keep up.
Lastly we expected to find that eventual differences in asked education level would be accompanied by differences in the asked number of skills. We hypothesized that larger companies would ask for a lower amount of skills, because they would offer lower working conditions. The combination of searching higher level educated employees, who do a lower amount of skills, would be indicating a possible real skill mismatch. The real skill mismatch we predicted for larger companies was not found in our dataset. In the nursing sector it seemed that all different size companies ask for about the same number of skills, and in the ICT sector it even seemed that larger companies ask a higher amount of skills. Because of the high standard deviation in jobs skills, we could not statistically prove these findings. But these results are indicating that it could be possible that small companies are creating real skill mismatches. These real skill mismatches are the cause of job dissatisfaction and loss of value (Green and Zhu, 2010). Future results will have to prove that smaller companies are indeed asking for a lower amount of skills while recruiting a higher level of education, but the indications are worrying at least.
5.3 Weaknesses and strengths
There are a number of advantages and limitations for this research. The largest advantage is that we could make use of a very large database, which we would not be able to gather as researchers alone. Also the subject of text mining in vacancies is a rather undiscovered field and so our research can be of practical use and possibly of relatively large importance.
However, our research also had its limitations. First, we were dealing with a predefined dataset. This narrowed our research possibilities because of the limited available variables. These
32 | P a g e N . J . S c h e f f e r s variables were however sufficient to accomplish our research goals and with cooperation of the recruitment companies we could extract a lot more variables in the future. The next limitation was that we were dealing with a very short timeframe and a limited amount of resources. This forced us to focus on only two of the available sectors and we had to make a selection of the available vacancies we had. However these constraints, we were able to run our text mining techniques on more than 350.000 sentences while only using simplistic hardware and basic software. If we look back at the total process, we can say that this has been a large accomplishment.
The last weaknesses, which are the most interesting for future research, are embedded in the constructs of our variables. We filtered out around 95% of the job skills using the ‘bag of words’ technique, but we could only detect those skills that consisted out of one word. More sophisticated algorithms are becoming available to run text mining analyses, which can detect the meaning of strings. These algorithms can prove very useful if a higher coverage is desired and possibly to lower the standard deviation of certain variables. This way it becomes possible to run proper analyses on the difference in for instance the amount of asked skills between different sized companies. The other variable which contained some limitations was company size. We saw that in the nursing sector most of the companies were containing 1000 or more employees and thus falling of the scale. There are more possible sectors which have relatively large companies, such as the financial or legal sector, so changing this scale in the future would be wise.
5.4 Implications and future research
In the last sections we covered some of the possible explanations for the observed results and we highlighted the weaknesses we witnessed during the research. Despite the fact that our research had some flaws, we think our results will have some useful implications. On theoretical level we contradicted some of the literature we have mentioned. Evans and Leighton (1989) argued that larger companies would try to attract higher educated people for the same job, but this is not supported by our results. It could well be that in this era of internet and globalization, smaller companies are the ones which search for higher quality to keep up with competition. Also the resource based view of Galanaki (2002) on online job vacancies was not supported by our results. Greater use of online job vacancies seems to become less dependent on resources owned
33 | P a g e N . J . S c h e f f e r s by large companies. It even seems that smaller companies are becoming the dominant players in some sectors using certain forms of internet recruitment.
There are also some practical implications to our results. We think that this research could help in understanding the dynamics of internet recruitment a bit better. Applicants should be aware that smaller companies might unnecessary ask for higher education levels for the same job. If they seek to find high job satisfaction, they should look for a job which aligns best with their skills not their level of education. Also our findings can help applicants in the Dutch ICT or nursing sector which are unable to find a job. We selected the most wanted skills for them, which could prove helpful when they want to select the right skills they still need to acquire.
All the weaknesses we mentioned in the last chapter can prove very helpful for future research. When we started this project, there was no good example research from which we could learn some of the methods. If someone else would now decide to dig deeper into this subject, we got the following recommendations. We argue that people should try to find effects in multiple sectors, because in our research we saw that results between sectors can differentiate. Also we think that ruling out vacancies which are made by recruiters could prove helpful for the results. We observed that recruiters will use the same format and ask the same job skills for different vacancies.
Other topics that would be very interesting to research in combination with job skills are: the paid wage or salary and the difference between family and non-family businesses. When researching the relationship between wage and job skills, it can become clear whether or not frequently asked job skills also have a high pay-off or not. This could be of use for educational institutions who want to determine which skills are not only frequently asked, but also maximize pay-off in the long term. Researching the relationship between job skills and family businesses is something which could help to further understand the behavior of companies during selection. Reid and Adams (2001) found that family businesses use human resource management in a different way than other companies, we think that this is the same for asked job skills and education level.
34 | P a g e N . J . S c h e f f e r s VI. Conclusion
The main intention of this research was to contribute to the knowledge base on companies’ behavior during selection procedures. This knowledge could help to enlarge the effectivity of recruitment in the future. To do this we used text mining techniques to evaluate approximately 8000 online job vacancies from different sectors. We found several results regarding company size, education level and asked job skills. It appears that smaller companies are becoming more active in the online job vacancy market. Also it are not the larger but the smaller companies who are searching for higher levels of education for the same job. The results also indicate that in certain sectors small companies ask for higher education, but not for a larger amount of skills. This is called ‘real’ overqualification and can be harmful for both applicants and employers.
The results we found can be of both theoretical and practical use. Some leading beliefs in literature are contradicted by our results. The roles between small and large companies seemed to have changed. Also our results can be of use for both employers and applicants. Small employers should become more aware that selecting higher levels of education for the same job, does not result in gaining higher job performance. And when applying for a job in a small company, applicants could consider highlighting their level of education to gain more attention. Lastly, the results can be of use for applicants in the ICT or nursing sector and for educational systems, who want to synchronize their skills / educational program with industry demands. We suggest that for future research other sectors are studied and an even larger pool of vacancies is used. Also we think that it could be of great social use to combine our research topics with the variable wage. This way it is possible to see what skills or education have a high pay-off. We strongly believe that there is a lot which has yet to be discovered in the suggested fields.
35 | P a g e N . J . S c h e f f e r s Final Remarks
I want to give special thanks to my tutor V. Kobayashi for his help steering me in the right direction and for his time to think about wise analytical strategies. I also want to thank my (little) brother Tom for helping me running the text mining model on over 350.000 sentences, without you this would not have been possible. Sometimes little brothers are actually the great ones.
36 | P a g e N . J . S c h e f f e r s Literature
Allen, J., & Van der Velden, R. (2001). Educational mismatches versus skill mismatches: effects on wages, job satisfaction, and on‐the‐job search. Oxford economic papers, 53(3), 434-452.
Bartram, D. (2000). Internet recruitment and selection: Kissing frogs to find princes. International journal of selection and assessment, 8(4), 261-274.
Brown, C., & Medoff, J. L. (1989). The employer size-wage effect (No. w2870). National Bureau of Economic Research.
Cappelli, P. (2001). On-line recruiting. Harvard business review, 79(3), 139-146.
Evans, D. S., & Leighton, L. S. (1989). Why do smaller firms pay less?. Journal of Human
Resources, 299-318.
Galanaki, E. (2002). The decision to recruit online: a descriptive study. Career Development
International, 7(4), 243-251.
Green, F., & Zhu, Y. (2010). Overqualification, job dissatisfaction, and increasing dispersion in the returns to graduate education. Oxford Economic Papers, 62(4), 740-763.
Groot, W., & Van den Brink, H. M. (2000). Overeducation in the labor market: a meta-analysis.
Economics of education review, 19(2), 149-158.
Hart, P. E., & Oulton, N. (1996). Growth and size of firms. The Economic Journal, 1242-1252. Keeping an eye on recruiter behavior (2012). Retrieved from
http://cdn.theladders.net/static/images/basicSite/pdfs/TheLadders-EyeTracking-StudyC2.pdf
Kuhn, P., & Skuterud, M. (2000). Job search methods: Internet versus traditional. Monthly Lab.
Rev., 123, 3.
Lahinsky, A. (2009, January 22). The Cook Doctrine at Apple. Retrieved from http://fortune.com/2009/01/22/the-cook-doctrine-at-apple/
Litecky, C., Aken, A., Ahmad, A., & Nelson, H. J. (2010). Mining for computing jobs. Software,
IEEE, 27(1), 78-85.
Medved, J. (2014, February 20). Top 15 Recruiting Statistics for 2014. Retrieved from http://blog.capterra.com/top-15-recruiting-statistics-2014/
Reid, R. S., & Adams, J. S. (2001). Human resource management-a survey of practices within family and non-family firms. Journal of European Industrial Training, 25(6), 310-320.
37 | P a g e N . J . S c h e f f e r s Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel
psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262.
Schneider, S. L. (2013). The international standard classification of education 2011. Comp Soc
Res, 30, 365-379.
Shell; Vision, Mission, Objectives. (n.d.). Retrieved from http://www.shell.com/src/about- src/vision.html
Stevenson, B. (2008). The Internet and job search (No. w13886). National Bureau of Economic Research.
Tirilly, P., Claveau, V., & Gros, P. (2008, July). Language modeling for bag-of-visual words image categorization. In Proceedings of the 2008 international conference on
