_________________________________________________________________
Personalized Training
A narrative review of end-user training during IS implementation _________________________________________________________________
Manon Kuijers S3858332 16 August 2020 University of Groningen Faculty of Economics and Business
MSc Business Administration: Change Management
Supervisor: Dr. H. Bruns Co-assessor: Dr. I. Maris-de Bresser
Word count: 11.761
Abstract
Purpose: Given the importance of training in IT implementation, the growing IT use and on the other hand the need for more research on training, the goal of this paper is to review and synthesize the fragmented research literature into a comprehensive overview that contributes to and shapes our understanding of training in IS implementation and highlights research gaps and opportunities
Methods: This research is a narrative literature review on training in an IT setting and contains peer-reviewed articles from 2010 up to 2020. In total, 70 articles were reviewed.
Findings: The way training should be designed, delivered and executed depends greatly on the organization, the trainee’s characteristics, the environment, the content of the training and the job or task. Out of the three training phases; pre-training, training and learning and post-training, recent articles have focused the most on the training and learning phase. Last 10 years there have been more articles on complex systems compared to more simple systems in the past.
Conclusion: Research reports on various training methods, techniques and factors that have to be taken into account when designing a training. However it is still not clear as to when a certain type of training is best suited and which factors are more important in different settings.. More empirical insights are needed to provide a good contingency theory in this area. Future research should focus on developing and validating new training methods, voluntary training, a common measurement of EUT, cultural differences and base their research on more qualitative measures.
Abstract 2
Introduction 4
2. Methods 6
3. Findings 11
3.1 Statistics 11
3.2 Pre-training 12
Pre-training techniques 14
3.3 Training and learning process 15
Content training 16
Training approaches 17
3.4 Post-training 20
3.5 Environment 21
Feedback 21
Support 22
4. Discussion 23
Future research 26
Limitations 27
Appendix 1 Sample 28
1. Introduction
Information and communication technology (ICT) has become a crucial tool for determining the competitiveness of organizations, economic growth, and their innovative development (Giotopoulos, Kontolaimou, Korra, & Tsakanikas, 2017; Olszak, 2016; Tan, Chong, Lin, & Eze, 2010). IT training or End-user training (EUT) is broadly acknowledged as a key factor in successful technology adoption and use (Aladwani, 2001; Chou, Lin, Lu, Chang, & Chou, 2014; Cooper & Zmud, 1990; Mendoza, Stern, & Carroll, 2008). Training can be defined as the systematic acquisition of knowledge, skills, and attitudes that together leads to improved performance in a specific environment (Salas, Wilson, Priest, & Guthrie, 2006). Training is used to permanently change people’s behaviors and actions. By carefully crafting training you can make sure that change will happen and people will get the right competencies to do their job. Information systems (IS) that used to be simple one-user systems have evolved into larger, complex, integrated systems for multiple users resulting in a need for more learning and coordination (Gattiker & Goodhue, 2005; Santhanam, R., Seligman, & Kang, 2007;
Sharma & Yetton, 2007). This review differs from prior ones (Ford, Baldwin, & Prasad, 2018; Gupta, Bostrom, & Huber, 2010; Salas, Tannenbaum, Kraiger, & Smith-Jentsch, 2012) in that it focuses on training in a technology context. A focus on technology means that most of the articles are about training that teaches the use of a certain technology. Additionally, this review covers a few articles on training other than technology offering insights relevant to the topic at hand.
This review will provide a summary of the literature that has appeared since the last review on training in a technology context in 2010. The prior reviews from 2012 and 2018 have focused on training in general and the transfer of training, respectively. Transfer of training means the extent to which learning during training is applied to the job or affects later job performance (Salas et al., 2012). These reviews were written from a psychological perspective. Prior literature reviews were used to find relevant articles and to compare findings and see what has been realized since. One of the objectives of this literature review is to see where we stand now, regarding literature on training in IS
implementations, since this literature review in 2010. Besides using more recent articles this review will also be different in the sense that this review will include lesser-known journals and conference papers to create a more complete picture of training during IS implementations. The literature review by Gupta, Bostrom, and Huber (2010) used only articles from top journals. This review only included a limited scope of articles. The scope of this literature review is broad and focused on all the literature published after the last literature review in 2010. A broad scope provides a comprehensive summary of all the evidence and offers an opportunity to explore the consistency of findings and therefore generalizability across different industries.
The goal of this review is to provide a clear overview of our current understanding of training during IS implementation. The objective of this review is to answer the following research questions:
1. What is currently known about training in IS implementation?
2. What are the current gaps in the literature and suggestions for future research?
Given the importance of training in IS implementation and the growing use and on the other hand the need for more research on training, the goal of this paper is to review and synthesize the fragmented research literature into a comprehensive overview that contributes to and shapes our understanding of training in IS implementation and highlights research gaps and opportunities. A review of training in IS implementation is urgent and timely because considerable research has been done since the last review (Gupta et al., 2010). This review is a narrative review due to the largely fragmented literature on the topic. By providing a better understanding of what processes are involved in training during implementation, managers and consultants will hopefully be able to better orchestrate training interventions within an IS implementation context to encourage continued use of the newly implemented IT. This research is important to IT consultants and managers who are working on IT implementations and anyone interested in training in a IS implementation context.
2. Methods
The purpose of a literature review is to objectively report the current knowledge on a topic based on previously published research. A literature review provides the reader with a comprehensive overview and helps place that information into perspective. Narrative reviews help present a broad perspective on a topic and often describe the history or development of a problem or its management. following the guidelines on how to write a narrative review by Green, Johnson, and Adams (2006) this section is divided into sources of information, search terms, and delimiting and selection criteria.
Initially, a review protocol was defined. The protocol provided a plan for the method to be followed, including the research questions, search strategy, inclusion and exclusion criteria, quality assessment, the data to be extracted, and data synthesis. I used Web of Science and EBSCO as well as snowballing to identify relevant research [scanning reference lists of articles ]. Google scholar served to find all the sources citing a particular article since this database showed more hits. Figure 3. shows the search process and the number of papers identified at each stage. The first step consisted of searching the titles, abstracts, and keywords of the articles included in the electronic databases and conference proceedings. The search string for the preliminary search was:
(“end-user training” OR “training” OR “user training”) AND (“information system*” OR
“technology”) AND (“implementation”). The complete list of search terms is shown in Figure 1.
1. “Information system*” OR “Information technology” AND “implement*”
2. “learning” AND “implementation” AND “IS”
3. “success factor* implementation*” AND “training”
4. “support” AND “implementation success”
5. “Interventions” AND “ Technology implementation”
6. “training” AND “adoption” AND “information technology” OR “information system*”
7. “End-user training”
8. “End-user training” AND “implementation”
9. “End-user training” AND “implementation” OR “adoption”
10. “Transfer of training”
11. “pre-training”
12. “post-training”
13. “transfer of training” AND “implementation”
Figure 1 search terms
This database search yielded 3,403 hits on the Web of Science searching in the topic. The database search in EBSCO yielded 2,216 hits. I narrowed down the articles to 134 articles by selecting scholarly peer-reviewed articles, published between 2010 (since the last review) and 2020 in academic journals, written in English with subject information technology, management, or training. The database searches yielded 288 hits. These articles were screened to see if they were potentially relevant. After that, I removed duplicates and articles that were already used in the last literature review on end-user training in 2010. Figure 2 presents an overview of the paper selection process.
The relevance of the articles was assessed against the predetermined inclusion and by screening all titles and abstracts. All publications, journals, and study types were included. Studies were eligible for inclusion if they presented data on training and passed the minimum quality threshold (Fig. 3).
Quality criteria
1. Is the paper based on research?
2. Is there a clear statement of the aims of the research?
3. Is there an adequate description of the context in which the research was carried out?
4. Is there a clear statement of findings?
4. Is the study of value for research or practice?
Figure 2. Quality criteria
The review includes both qualitative and quantitative research studies, as well as empirical and theoretical studies of any particular study design in EUT, applied in any organization, published up to 2020. Only studies that are written in English and available online were included.
Based on the abstract and how central the topic was to the article, I narrowed down the number of studies to 37. After the initial selection of papers, I performed a forward as well as backward search
and came to a total of 70 articles included in this review. The complete list of the articles included in this review can be found in appendix 1. Forward and backward citation tracking complemented the database searches. Forward citation tracking was done more often since the aim of this study was to find the most recent developments. Forward citation tracking refers to identifying new papers based on those papers citing the focal paper. I examined all the citations to the focal paper using Google Scholar. The first screening was based on the information provided in Google Scholar. I examined the full text of the citing paper when the abstract on Google Scholar was insufficient (Wohlin, 2014).
Full-text manuscripts were obtained for all studies entering the review. Keywords and search terms were derived from the research questions, a conceptual model for reviewing end-user training, and keywords from the studies found. Other strategies used to generate search terms in this review included alternative spellings and synonyms of key terms, boolean OR was used with relevant terms;
and boolean AND was used to combine search terms to limit the search.
Figure 3 Search process
The synthesis methodology or theoretical framework which underpins the synthesis is a thematic synthesis. The reason why this paper uses a thematic synthesis is that the literature is segmented and scattered. Bringing the literature back to just a few broader themes will help create a comprehensive overview of the existing literature. The approach to searching was pre-planned which means comprehensive search strategies were used to seek all available studies. I extracted the data from the articles independently to an Excel spreadsheet. The data included information about the title, authors, the year it was published, relevant themes discussed within the paper, theoretical or empirical, document type, publication setting, knowledge type, journal, context, methodology, key quotes, and how many times the article was cited. Initially, the information in the articles was grouped into the predetermined themes introduced in the conceptual framework for the EUT literature review (Fig. 3).
The grounded theory approach was used for identifying new themes. The grounded theory approach is a method to identify themes in literature. The grounded theory approach started with ‘in-vivo’ coding to identify new themes. Initially, 35 in-vivo codes were created from the definitions captured from the papers. Examples of in-vivo codes were; Self-efficacy, Perception, Learning styles, Attitudes towards the system, Emotions, Motivation, Prior experience, Motivation to Transfer and Self-regulated learning. These were all in-vivo codes for an overarching group called individual differences.These codes were then analyzed and grouped into overarching groups-of-themes. Eight groups of themes were identified. The eight groups were individual differences, pre-training techniques, training content, training approaches, importance of post-training, continuous learning environment, support and feedback. Lastly, the groups-of themes were integrated into the conceptual framework (Fig. 4).
Figure 4 Conceptual framework
The framework for this literature review is shown in figure 4. The framework is based upon Gupta, Bostrom, and Huber (2010) and Krompho and Porrawatpreyakorn (2013). The existing frameworks served to group the literature into the 3 themes; pre-training training and learning and post-training. I divided these themes into subgroups based on the themes I identified in the literature between 2010 and 2020 and added a new theme; environment. Pre-training influences training and learning and training and learning influence post-training. It is important to note that this is not a process model because the description in the boxes is merely added to show how the literature on training can be categorized. The pre-training process consists of individual differences and pre-training techniques.
Individual differences refer to the difference in the capability of each individual that affects EUT (Krompho & Porrawatpreyakorn, 2013). The training and learning process can be divided into training content and training approaches. Training content covers what needs to be included in the training in order to learn as much as possible. Training approaches are the form in which the training will be given. The post-training phase can be divided into literature on the importance of the
post-training phase and how to establish a continuous learning environment. Finally, the environment
covers the effects of work environment factors such as feedback and support on training transfer (Kazbour, McGee, Mooney, Masica, & Brinkerhoff, 2013).
A variety of different disciplines is used in the sample to provide a broader perspective on training approaches. The training literature for this study was drawn from a variety of journals. The sample data consisted of a total of 46 different journals. The journals are grouped into broader fields of interest because of the number of different journals. The journals on computing, education and human resource were the most common type of journals in the sample data. Followed by information technology and management journals. The articles used were relatively equally divided over the years but most articles came from 2010 and the least amount of articles came from 2019.
3. Findings
Firstly, I discuss the general statistics of the sample I used for this literature review. The reviewed literature on training can be roughly divided into five themes namely pre-training, training and learning, post-training, and environment This literature review is structured in the same order.
Pre-training, training and learning and post-training represent a process model in which the
pre-training is the input, training and learning the process and post-training the output. Environment describes the effects of work environment factors such as support, feedback, and goal setting, on training.
3.1 Statistics
The sample data consists of both quantitative and qualitative research methods as well as mixed methods. However, the quantitative design was the most common method among all the articles used in this review see table 3.1. A quantitative design was mostly used by researchers that conducted research that involved the training and learning process. Only a few studies combined both
quantitative and qualitative methods. Most of the articles described the training and learning process, followed by pre-training.
Research type
Categories
Total %
Pre-training Training &
learning
Post-training Environment
Quantitative 7 58,3% 21 63,6% 3 50% 5 50% 36 52,2
Qualitative 4 25% 14 27,3% 2 33,3% 6 33,3% 26 37,7
Mixed
methods 2 16,7% 3 9,1% 1 16,7% 1 16,7% 7 10,1
Total 13 100% 38 100% 6 100% 12 100% 69 100
% 18,8 55 8,7 8,7 100
Table 3.1 Category versus research method
Among all the papers included in this review, the survey method was the most common method followed by an experiment as a research method. Not every article specified the type of technology.
However, based on the available information we can see that there are quite a few studies on complex IT. Complex IT are programs like ERP, SAP, and Electronic Health records. Simple IT are programs like Excel, Word, and E-learning programs. The type of participants involved in the research articles were mostly employees or students and there have only been a few articles on IT experts in IT training.
3.2 Pre-training
Considering the significance of training to organizational productivity, effectiveness, and the costs connected to the design and implementation, researchers and practitioners must have a clear
understanding of the pre-training factors influencing training to foster successful training (Boothby, Dufour, & Tang, 2010). Pre-training preparation is done to maximize the learning outcomes of training initiatives. Pre-training techniques promote cognitive skill, and affective learning regardless
of the training method employed or content of the training (Mesmer-Magnus & Viswesvaran, 2010).
This chapter is divided into two subparts namely; individual differences and pre-training techniques.
Individual differences
This section discusses research on individual differences between trainees, the interaction between methods and individual differences, which influence training. Prior literature shows a positive relationship between user training and implementation success, but this relationship between user training and implementation success can be influenced by individual differences like the end-users’
attitudes towards the system and user involvement (De Waal & Batenburg, 2012). This is why user-specific traits are an important part of the training literature. Research shows a variety of individual differences which influence training like; attitude, motivation (De Waal & Batenburg, 2012), conscientiousness (Sitzmann, 2012), learning style (Gupta & Anson, 2014) , emotions (Bala &
Venkatesh, 2016; Kodwani & Prashar, 2019) and perceived relevance of the system (Bala &
Venkatesh, 2016). De Waal and Batenburg (2012) even mentioned attitude to be critical to the potential contribution of training to implementation success and therefore the training must be designed for different trainees. De Waal and Batenburg (2012) conducted a mixed-method study on the effect of involvement and attitude on end-user training (EUT). Their study is based on 143 surveys and 49 semi-structured interviews among end-users within a large Dutch social insurance
organization. The results show that a positive attitude towards the system helps, when parts of the training, like training time, environment, and other arrangements are not well organized. However, users with a negative attitude reduced the chance of implementation success since their user satisfaction and intention to use reduced during the training (De Waal & Batenburg, 2012).
The literature on learning processes forms the base of how individual differences affect learning outcomes and why paying attention to these individual differences is so important. However, before 2014 there were only a few studies on the actual learning process based on empirical data. Gupta and Anson (2014) noticed this gap and looked at training from a learning style perspective and researched
the link between learning style and training method. Their results suggest that learning styles
influence how trainees interact with the training method and paying attention to individual differences and personalizing the training enhances computer learning (Gupta & Anson, 2014).
The fact that different individual characteristics play a role in training success has become clear over the years but the degree in which they are important seems to differ per type of training. For example;
pre-training self-efficacy and motivation are more important when training open skills. Open skills like management skills give the trainees more choice on how to apply the trained skills on the job.
Closed skills like programming have a prescribed character (Blume, Ford, Baldwin, & Huang, 2010).
Next to the difference in open and closed skills, there is also a difference in individual characteristics that play a role, between voluntary and obligatory training. An important factor for voluntary training is conscientiousness since it is up to the trainee to attend the training or not. Trainees drop out of the training more often when their conscientiousness is low (Sitzmann, 2012). A high level of
conscientiousness, on the other hand, provides a buffer against dropping out when trainees' commitment and self-efficacy declined during training (Sitzmann, 2012).
Pre-training techniques
Pre-training techniques are not the same techniques as the techniques used in post- or during training because the impact of the technique is related to time (Lee & Xia, 2011) . There is still a lot unknown about what technique to use under which circumstances. One thing we now know about research on pre-training techniques is that motivation to transfer plays an important role. Trainees enter the training with different levels of motivation to transfer. Motivation to transfer in this sense means the desire to apply the knowledge and skills from the training to the job (Massenberg, Schulte, &
Kauffeld, 2017). Which is in most cases the goal of the training. The learning environment and techniques before the training influence this motivation to transfer (Massenberg et al., 2017).
Research shows multiple techniques that can be used before the start of the training; making sure that the trainees and management are aware of the importance of the training and are prepared for training
to start (Hughes, Zajac, Spencer, & Salas, 2018) creating open lines of communication (Grossman &
Salas, 2011), encouraging trainees to set a personal goal for the training (Noe, Clarke, & Klein, 2014;
Puhakainen & Siponen, 2010; Salas et al., 2012), creating training-related rewards and identifying resources necessary (Grossman & Salas, 2011).
There are numerous different pre-training techniques and this number is still rising as technology develops and the systems become more complex. Business process modeling and integrating end-user training with systems development are examples of new techniques (Davis & Brattin, 2019;
Oinas-Kukkonen, Hohtari, & Pekkola, 2010). The pre-training techniques described in recent literature are all influencing the training positively but this makes it challenging to choose which pre-training techniques to apply in practice since resources are often limited. Articles in which researchers compare the effectiveness of different techniques are scarce. Mesmer-Magnus and Viswesvaran (2010) compared the effectiveness of the most common pre-training techniques.
Attentional advice and goal orientation were the most effective techniques for learning gains.
Although these techniques showed the best results, other techniques may be more suitable depending on the desired learning outcome. The results indicate that the best-suited pre-training technique depends on the desired learning outcome of the training. However, the outcome of the training can be selected, and enhanced by using the right pre-training techniques (Mesmer-Magnus & Viswesvaran, 2010). Massenberg, Schulte, and Kauffeld (2017) on the other hand state that effectiveness depends on the organization and the type of training. This highlights the importance of a training needs analysis (TNA) during the pre-training phase to align the needs and goals of the business and those of the trainees to realize the potential advantages of the pre-training techniques (Hughes et al., 2018;
Mesmer-Magnus & Viswesvaran, 2010; Salas et al., 2012)
3.3 Training and learning process
The actual training and learning process is the learning process during the training. The literature on this topic can be divided into the content of the training and the different training approaches. In
general, Noe, Clarke, and Klein (2014) noted a trend towards more learner-centered training due to the growing dependency on technology-mediated training and informal learning, giving trainees a more active role in their learning process. Amadi-Echendu and de Wit (2015) provide empirical data that confirms the existing ideas about training having a very strong influence on how a user not only perceives and accepts a technology system but also how he or she utilizes the technology afterward.
They found that training should not only focus on the functionality of IS but also the content (Amadi-Echendu & de Wit, 2015; Sitzmann & Ely, 2010)
Content training
The content of the training is about what elements need to be taken into account when designing the training. The literature on this topic can be roughly divided into three themes. Firstly, research on how to make sure the trainees are attracting as much knowledge as possible from the training.
Secondly, how to retain as much knowledge as possible during and after the training and thirdly, on how to foster the fit between training material and the actual work situation. These themes will be discussed further in this paragraph. Most articles were about how to attract as much knowledge as possible and improve competence.
Research on attracting as much knowledge as possible shows predictors of motivation to learn;
affective organizational commitment, job involvement, utility perceptions, and computer attitude (Downey & Smith, 2011; von Treuer, McHardy, & Earl, 2013). These predictors are aspects that have to be taken into account when designing training to make the trainees attract as much knowledge as possible. Sitzmann and Elly’s research (2010) has implications for both attaining and retaining as much information as possible. They used a longitudinal design to research how the self-regulation mechanism works in voluntary online training because there was at that time a lack of empirical data and the results were often contradicting. The results show that trainees need to be encouraged to self-regulate throughout the entire course to realize the benefits of the training as it reduces attrition and increases learning. The self-regulation intervention involves asking the trainees questions about
whether they are concentrating on learning, monitoring the effectiveness of their learning strategies, and evaluating their knowledge levels. Trainees who received the questions before the training or only in the middle of the training did not learn significantly more (Sitzmann & Ely, 2010). Contradicting other researchers who found that a pre-training intervention can produce substantial increases in learning outcomes (Mesmer-Magnus & Viswesvaran, 2010).
Olfman, Ryan, Shayo, & Coulson (2010) write about retaining the knowledge after the training and looked at knowledge levels and user understanding based on two strategies namely; procedural and tool-conceptual strategy. Procedural training is about how to carry out specific tasks in a prescribed step-wise manner. Conceptual training is about using metaphors and looking at similarities with familiar software. Their experiment showed that training involving a tool-conceptual strategy leads to higher mental models, compared to training with a procedural strategy. High mental models mean that the trainees were able to describe the systems concepts more accurately after the training than trainees with lower-level mental models (Olfman, Ryan, Shayo, & Coulson, 2010). The conceptual training makes it also easier to master new applications and improves the quality of use (Li & Chang, 2011).
Other scholars highlight the importance of linking the training to the trainee’s actual work (Friedman
& Ronen, 2015; Kazbour et al., 2013; Polites & Karahanna, 2013). Discussing how to adapt the training material to the real-work situation (Kazbour et al., 2013) and establishing if-then plans help trainees in knowing what to do when a specific work situation arises (Friedman & Ronen, 2015).
Training approaches
When it comes to training approaches and techniques there are a lot of aspects that have to be taken into account for example; is the role of the trainee or trainer passive or active, using new approaches, the correct mix of formal and informal learning, task characteristics and timing.
Luse, Mennecke, and Townsend (2013) looked at the passive or active role of trainees and trainers.
Their findings show that trainees who viewed someone else performing the task, differ in technology
acceptance dimensions when compared to trainees who actively performed the task during training.
The role that you give a trainee or trainer whether it is active or passive matters when you look at technology acceptance and learning competence (London & Hall, 2011; Luse et al., 2013). This difference in activeness is shown in different types of training approaches like traditional instructor-led, informal, collaborative, technology-mediated, self-directed, and blended methods (Al-Busaidi, 2012; Gupta & Bostrom, 2013; Hickman & Akdere, 2019; Noe et al., 2014; Sykes, 2015). This distinction between passive and active can also be seen in London and Hall’s (2011) research. They mention the instructor-driven face-to-face, or often called adaptive training, being a passive training from the trainee’s point of view. The opposite of this passive form is the generative type of learning which is “.. learner-driven, collaborative and problem-focused” (London & Hall, 2011, p.757). Adaptive learning is more passive and about trainees learning a specific behavior or skill. Generative learning, on the other hand, is much more about exploring and trainees creating their own best way of working (London & Hall, 2011). However, the question remains which technique is the best. Therefore, Gupta and Bostrom (2013) researched three different training approaches namely;
technology-mediated, collaborative, and blended methods. The results showed that
technology-mediated and collaborative methods worked for some training outcomes but the blended method worked for all training outcomes.
New technology enables people to create new training approaches. One type of technology-mediated training that is relatively new is gamification. Gamification is” applying game design to non-gaming contexts, is proposed as a way to add engagement in technology-mediated training programs.”
(Santhanam, Radhika, Liu, & Shen, 2016, p.453) Gamification is a promising new technique since research already shows better trainee performance using a gamified approach as opposed to a non-gamified approach (Alcivar & Abad, 2016). Santhanam, Lui, and Shen (2016) researched gamification and more specifically competition in gamification since competition is an important element in driving enhancing learning and engagement. The findings show that competition needs to be personalized based on the trainee’s training goals and characteristics (Santhanam, Liu, & Shen,
2016). Other approaches fueled by this digital age are mobile learning and the use of social media as a collaborative media tool in training (Lac, 2018; Thomas & Akdere, 2013). Another new approach is the use of simulation. The goal of using simulation is to “make learners independent as quickly as possible and establish a learning environment where people feel free to experiment, and to fail.”
(Feldstein, 2011, p.46). Especially the part about feeling free to experiment and fail is important in simulation because the transfer of training is more successful when the trainee can detect and correct his/her mistake and see the mistake as an opportunity to learn (Feldstein, 2011; Lauzier & Mercier, 2018). This is very different from the traditional approach to training in which people try to avoid mistakes.
Another element that needs to be considered when designing training is the people within the training and if and how they should interact with each other. An example of a collaborative approach is team-based learning in which trainees are instructed to learn from not only the trainer but also their peers. A realistic training environment, positive and negative examples of behavior, and peer support are advantages of team-based learning and are important factors in successful training (Grossman &
Salas, 2011; Hart, Steinheider, & Hoffmeister, 2019; Salas et al., 2012). This approach in which trainees learn from their peers is also called informal learning. Informal learning can occur through formal training approaches but also by being a team member (Choi & Jacobs, 2011; Kukenberger, Mathieu, & Ruddy, 2015; Milia & Birdi, 2010). Organizations should not only consider whether or not to facilitate formal or informal learning but also how to determine the optimal mix (Choi &
Jacobs, 2011). Liang, Peng, Xue, Guo, and Wang (2015) look at training approaches from another interesting point of view namely; the task characteristics point of view. They discovered that:
“employees with high autonomy tend to have a strong sense of responsibility and ownership of the job, and they will proactively develop new skills through various means so that they are capable of exploring the system. In contrast, employees with high task variety have a heavy cognitive load and are unlikely to proactively seek new ways to explore the system.” (Liang et al., 2015, p.345)
Therefore, adaptive training approaches seem to suit employees with high task variety, and a
generative approach by employees with high autonomy (Liang et al., 2015; London & Hall, 2011).The timing of the training is also a component that has been researched in the past. Researchers found that the training should be spaced and close to the go-live date but at the same time, the trainees should have enough room for practice (Liu, Feng, Hu, & Huang, 2011; Norton, Coulson-Thomas,
Coulson-Thomas, & Ashurst, 2012).
3.4 Post-training
The literature on the post-training phase can be divided into two themes namely; the importance of the post-training phase and how to create an environment in which trainees can continuously learn. The importance of the post-training phase is highlighted by many researchers because to maximize the transfer of training, different factors have to interact, not only during training but also before and after the training (Chang & Chou, 2011; Najmul Islam, 2014). The training before the go-live usually focuses on functional use of the system however the information systems have become more and more complex and this focus on functional use of the system does not seem to be enough (Chang &
Chou, 2011) Besides, trainees often experience situations on the job that were not covered in the training (Venkatesh, Zhang, & Sykes, 2011). Therefore, researchers highlight the importance of gaining tacit knowledge in the post-training phase so trainees can better understand the system and enhance how they use the software (Chou et al., 2014; Hickman & Akdere, 2019; Hsieh & Hsu, 2015;
Zhang et al., 2011). Providing mentors is one tool that can help with gaining tacit knowledge (Hsieh
& Hsu, 2015). Another reason why training should be continuously provided is to increase
self-efficacy and prevent workarounds (Chou et al., 2014; Osnes, Olsen, Vassilakopoulou, & Hustad, 2018).
This environment in which trainees can continuously learn can be created by establishing the right culture, fostering social capital, and designing suitable incentives (Chang & Chou, 2011). Supervisors also play an important role in this post-training phase. “Only when supervisors are aware of their role
and know what they can do in the different phases will they deal with facilitating subordinates’
transfer throughout the training cycle.” (Sitzmann & Weinhardt, 2018, p.548) And once again some researchers call for a personalized approach based on different roles trainees have within the company to fully support each user type (Liu et al., 2011; Sykes, 2015).
3.5 Environment
The effects of work environment factors such as feedback, support and goal setting, on training transfer, were an under-researched area for a long time (Kazbour et al., 2013). Since the last reviews on training, there have been a couple of scholars who researched this particular area of interest (Cheng, Wang, Moormann, Olaniran, & Chen, 2012; Kazbour et al., 2013). Kazbour et al. (2013) performed quasi-experimental research based on surveys, interviews, and behavioral measures to research the influence of work environment factors on training transfer. As trainees receive more post-training support, they become more inclined to use what they have learned during the training on the job (Kazbour et al., 2013). The next paragraphs will discuss the literature on feedback and support in more detail.
Feedback
Several researchers wrote about feedback or often called debrief, and the role of managers in this process since debriefing can help trainees retain the things they learned during the training and guide managers in giving instructions based on problems that trainees encountered (Feldstein, 2011).
Hughes et al. (2018) mentioned: “managers should give both positive and negative feedback and continue to debrief with trainees often, including both positive and negative examples of behavior as well as a period for reflection and planning for future issues concerning the trained skills.” (p.337).
However, Van den Bossche, Segers, and Jansen (2010) mentioned that it is not just a matter of giving a lot of feedback. Their research shows that it is more important for the transfer of training to increase the number of people providing feedback and more feedback of the same people works even
counterproductive (Van den Bossche et al., 2010).
Support
One important aspect of the environment is that trainees need to feel supported to facilitate successful training (Grossman & Salas, 2011). Managerial support, job support, and organizational support are three types of support that have proven to be significant motivators for employees’ technology acceptance (Cheng et al., 2012). Management should support trainees in such a way that trainees feel comfortable discussing concerns, issues, and ideas this make trainees retain the things they learned during the training and fosters the transfer of training (Al-Busaidi, 2012; Blume et al., 2010; Govaerts et al., 2018; Hughes et al., 2018; Kazbour et al., 2013; Wan Hooi, 2010). Local experts can be seen as a fourth form of support. Local experts are “employees who voluntarily help others to learn”
(Woldesenbet & Klay, 2016, p. 693). Wan Hooi (2010) mentioned the use of superusers who provide on-the-job training which can also be local experts and according to his findings; this is the most effective method for delivering technical training (Wan Hooi, 2010).
Researchers have overlooked these informal learning processes in the past. This seems to have changed in the past 10 years as we see more research on the importance of these informal learning processes (Choi & Jacobs, 2011; Hickman & Akdere, 2019; Kukenberger et al., 2015; Milia & Birdi, 2010; Sasidharan, Santhanam, Brass, & Sambamurthy, 2012; Sykes, 2015; Woldesenbet & Klay, 2016). Woldesenbet and Klay (2016) found that local experts help trainees in the same way as researchers have recommended, namely by “engaging in needs assessment, instructional design, development-oriented implementation, and evaluation.” (p.708). Sykes (2015) compared the
employee outcomes of four traditional support structures and peer advice ties. The traditional support structures were training, online support, help desk support, and change management support. The outcome showed that peer advice ties was the strongest predictor of system satisfaction, job stress, job satisfaction, and job performance (Sykes, 2015). However, some researchers state that more support is not always better since not every trainee needs the same amount of support (Al-Busaidi, 2012).
4. Discussion
The goal of this literature review is to map the current landscape of the literature about training in an IT implementation context. As mentioned in the introduction, prior literature reviews on training are the starting point of this literature review (Ford et al., 2018; Gupta et al., 2010; Salas et al., 2012). If one thing has become clear over the years, it is that there is no one-best-way to training. Training is complex, there are a lot of factors influencing different subparts of training. Training depends not only on the type of technology but also on the organizational environment, the job or task, and the different characteristics of the trainees. Prior literature reviews on the pre-training phase have mainly reported on the importance of a TNA and setting personal goals before the start of training. The literature in the past 10 years has confirmed its relevance once again. However, researchers seem to have shipped the focus to more personal differences influencing training like emotions, learning style, and attitude.
Although, there seems to be a lack of comparative research on individual differences. Therefore we do not know which individual differences are more important. Research does show that the importance depends on the learning style, the type of training, and the content of the training.
When it comes to training approaches numerous articles are reporting on various training methods like mobile learning, simulation, collaborative learning showing promising results. The number of new training approaches will keep on rising as technology develops and the systems become more complex However, it is still not clear as to when a certain type of training is best suited. We need more empirical insights to provide a good contingency theory in this area. Gupta et al., (2010) have suggested more research on how these pre-training interventions influence the development of the training method exactly. This research has not been done yet in the past 10 years so there is still a gap.
Gupta et al,(2010) also suggested more research on self-regulated learning because there was at that time a lack of empirical data and the results were often contradicting. Sitzmann and Ely (2010) responded with research on self-regulation. The results of their study show that trainees need to be
encouraged to self-regulate throughout the entire course to realize the benefits of the training as it reduces attrition and increases learning. It was also the only article that explicitly stated that the research was conducted in a voluntary training setting. Salas et al. (2012) further suggest that practice opportunities should require trainees to engage in the same cognitive processes they will need to engage in when they return to work. Kazbour et al. (2013), and Friedman and Ronen (2015) mention new ways of how to accomplish this. Discussing how to adapt the training material to the real-work situation (Kazbour et al., 2013) and establishing if-then plans help trainees in knowing what to do when a specific work situation arises (Friedman & Ronen, 2015). Olfman et al., (2010) looked at knowledge levels and user understanding based on two strategies namely; procedural and
tool-conceptual strategy. Procedural training is about how to carry out specific tasks in a prescribed step-wise manner. Conceptual training is about using metaphors and looking at similarities with familiar software.
Gupta et al., (2010) urged scholars to look into the learning process because this was a theme that was under researched and not understood. Since then a considerable amount of researchers have
researched this learning process as can be seen in table 3.1. The research has mostly been focused on different factors influencing the learning process (Darban & Polite, 2016; Gupta & Anson, 2014;
Mesmer-Magnus & Viswesvaran, 2010; Sitzmann & Ely, 2010) Noe, et al., (2014) noted that they saw a trend towards a more learner-centered training due to the increased reliance on
technology-based training and informal learning, giving trainees more control over their learning (Noe et al., 2014).
This difference in activeness is shown in different types of training approaches like traditional instructor-led, informal, collaborative, technology-mediated, self-directed, and blended methods (Al-Busaidi, 2012; Gupta & Bostrom, 2013; Hickman & Akdere, 2019; Noe et al., 2014; Sykes, 2015). However, the question remains which technique is the best. Therefore, Gupta and Bostrom (2013) researched three different training approaches namely; technology-mediated, collaborative,
and blended methods. The results showed that technology-mediated and collaborative methods worked for some training outcomes but the blended method worked for all training outcomes. It is still unclear how to design blended training. Another interesting aspect is that there are quite some
researchers who were interested in the social/ informal learning process (Choi & Jacobs, 2011;
Kukenberger et al., 2015; Milia & Birdi, 2010). Despite the interest in this specific area, there is still a lot unknown about how to combine formal and informal learning (Choi & Jacobs, 2011). When it comes to feedback, we already knew that people learn through practice and feedback (Salas et al., 2012) What we did not know was that it is more important for the transfer of training to increase the number of people providing feedback and more feedback of the same people works even
counterproductive (Van den Bossche et al., 2010).
Gupta, et al., (2010) mentioned that many of the studies in EUT have focused on simple target systems as well as novice users. And since technology becomes more prevalent, training in complex systems is becoming more important. While research from simple systems might be generalized to complex systems, empirical support for such a generalization could not be found. There is still no research on whether or not the results from research on simple systems are generalizable to complex systems. There is however a lot of research published in recent years on complex systems like SAP, ERP, and HDIS . Besides, the learner profiles are changing from novice to intermediate computer users. A lot of research on training has been on students while only one study focused on IT experts.
Ford etal., (2018) mentioned that we know little about what types of support there are and when to use the different types of support. We now know that there are at least four types of support that have proven to be significant motivators for employees’ technology acceptance which are managerial support, job support, organizational support and local experts (Cheng et al., 2012; Woldesenbet &
Klay, 2016,)
Future research
In the previous paragraph, I already discussed some gaps within the literature but in this paragraph, I elaborate more on a few other areas of interest for future research. At this point, there is little need for more research on the effectiveness of training and predictor-outcome relationships. Future research should focus on developing and validating new training methods. As technology evolves over time, possibilities arise for new training methods like incorporating virtual reality, augmented reality, and how to train people in even more complicated technology in which employees have to build and maintain the system themselves. It is also important that there are more comparative studies. Since the resources within organizations are often limited. Organizations will have to choose between training approaches, pre-training interventions and individual differences they want to focus on. There is a lot of research on which factors are influencing the transfer of training. Consistent with other researchers (Blume et al., 2010; Grossman & Salas, 2011 ) future research should focus on the conditions under which these factors are most important and when they are important within the training process.
Another interesting area for future research brought to light by De Waal & Batenburg (2012) which has not been reached yet is a common measurement for EUT. Currently, there is no common measurement for EUT. Research on EUT is mostly measured by quantity or intensity, and not using qualitative measures. The sample for this literature review alone shows a lot more quantitative studies than qualitative studies. To develop more knowledge about the different processes within the training process and how the trainees experience the training, qualitative research would be a better fit.
Besides, there are other, more reliable methods than surveys. Future research should use a stronger measure that would include some form of longitudinal research based on observation of behavior (Kazbour et al., 2013).
Another area of interest is the possibility that certain factors influencing training differ between cultures (Gelfand et al. 2007; Noe, Clarke, & Klein, 2014). In this sample were articles included from different countries but there is no comparative study available. Future research should investigate
learning from a comparative perspective to gain more insight into how culture influences the effect of individual differences on training but also the transfer of training and training design. Mamonov (2018) concludes that there is still little research on the factors influencing voluntary training. The sample used for this review included one article which explicitly stated that the research was
conducted in a voluntary training context. This is not enough, future research should, therefore, focus on differences between the importance of factors influencing training effectiveness of voluntary and obligatory training. New training challenges will certainly arise in the upcoming 10 years when the systems get more complex, the trainees more experienced with IT, training more virtual and the workforce more diverse. So research on end-user training will stay relevant and much needed.
Limitations
The focus on articles in an IT setting might have excluded papers on training that were not about IT systems but had generalizable insights for IT training. Another limitation is the fact that only articles were used that were publicly available online. There could have been valuable articles online to which I did not have access and were therefore not included.
Appendix 1 Sample
1 Coulson, Tony & Olfman, Lorne & Ryan, Terry & Shayo, Conrad
2010 Enterprise Systems Training Strategies: Knowledge Levels and User Understanding.
2
A Puhakainen P Siponen M 2010 Improving Employees' Compliance Through Information Systems Security Training- An Action Research Study
3
Sitzmann T, Ely K. 2010 Sometimes you need a reminder: the effects of prompting self-regulation on regulatory processes, learning, and attrition
4 di Milia L Birdi K 2010 The relationship between multiple levels of learning practices and objective and subjective organizational financial performance
5 Hooi L 2010 Technical training in the MNCs in Malaysia: A case study analysis of the petrochemical industry
6 Oinas-Kukkonen 2010 Organizing End-User Training: A Case Study of an E-Bank and its Elderly Customers 7 Mesmer-Magnus J
Viswesvaran C
2010 The role of pre-training interventions in learning: A meta-analysis and integrative review
8 Blume B Ford J Baldwin T Huang J
2010 Transfer of Training- A Meta-Analytic Review
9 Van Den Bossche P Segers M Jansen N
2010 Transfer of training: the role of feedback in supportive social networks
10 Boothby D Dufour A Tang J 2010 Technology adoption, training and productivity performance
11 Léger P Charland P Feldstein H Robert J Babin G Lyle D
2011 Business Simulation Training in Information Technology Education: Guidelines for New Approaches in IT Training
12 Yuan Li & Kuo-Chung chang 2011 Exploring the dimensions and effects of computer software similarities in computer skills transfer
13 Choi W Jacobs R 2011 Influences of formal learning, personal learning orientation, and supportive learning environment on informal learning
14 Liu L Feng Y Hu Q Huang X 2011 From transactional user to VIP: How organizational and cognitive factors affect ERP assimilation at individual level
15 London M Hall M 2011 Unlocking the value of Web 2.0 technologies for training and development: The shift from instructor-controlled, adaptive learning to learner-driven, generative learning
16 Gwanhoo Lee a,*, Weidong Xia
2011 A longitudinal experimental study on the interaction effects of persuasion quality, user training, and first-hand use on user perceptions of new information technology
17
james P. Downey, Lloyd A.
Smith
2011 The role of computer attitudes in enhancing computer competence in training
18
Rebecca Grossman and Eduardo Salas
2011 The transfer of training: what really matters
19
Sitzmann T 2012 A theoretical model and analysis of the effect of self-regulation on attrition from voluntary online training
20
Norton A Coulson-Thomas Y Coulson-Thomas C Ashurst C
2012 Delivering training for highly demanding information systems
21
Al-Busaidi K 2012 Learners’ perspective on critical factors to LMS success in blended learning: An empiri- cal investigation.
22
Waal, B.M.E. de, Batenburg, R 2012 What makes end-user training successful? A mixed method study of a business process management system implementation.
