Defining key factors that determine students’ satisfaction from e-learning use

(1)

Defining key factors that determine students’ satisfaction from e-learning use

Master’s thesis Business Administration: Digital Business

Name: Pawel Janczarski Student number: 11829583 Supervisor: Barış Altaylıgil

(2)

Statement of Originality

This document is written by Student Pawel Janczarski who declares to

take full responsibility for the contents of this document.

I declare that the text and the work presented in this document are

original and that no sources other than those mentioned in the text and

its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the

supervision of completion of the work, not for the contents.

(3)

Abstract

Development of technologies led to rising accessibility of computer devices and internet connection worldwide. Digital services and goods became more available and people started to notice possibilities coming from them. This led to the development of online learning, where people without leaving their house, can get new knowledge in a new field. This study examines key factors that determine user satisfaction from e-learning use. It is done, on the basis of the Technology Acceptance Model, which determines behavior’s intention to use, through perceived ease of use (PEOU) and perceived usefulness (PU). Based on the literature review, the external variables were divided into two models. The first model examined relations between Interaction with a teacher, Interaction with peers Mobile Learning and Perceived Ease of Use. The second model examined the effect of predictors: Content Quality, Mobile Learning, Spaced repetition, Microlearning, Gamification on Perceived Usefulness. According to the findings, only one predictor- Mobile Learning in the model 2, did not explain variance of PU in statistically significant way. In the model 1, the greatest effect on PEOU had Mobile Learning at the least effect Interaction with Peers. In case of model 2, Gamification had the greatest effect on PU, and the least effect had Content Quality. Moreover, only a few statistically significant differences in means were found among categorical variables. But none of those difference was big enough, to draw outbreaking conclusions. It shows that the target group of e-learning solutions is coherent and their preferences, based on this analysis, don’t differ much.

Key words: E-learning, Perceived ease of use, Perceived usefulness, Content Quality, Gamification, Microlearning, Spaced Repetition, Mobile Learning

(4)

List of figures and tables

Figure 1. Differences between Micro- and Macro-Learning (Bersin, 2017) ... 11

Figure 2. The Technology Acceptance Model (Davis, et al., 1989) ... 14

Figure 3. Ebbinghaus forgetting curve ... 19

Figure 4. Overcoming the forgetting curve (Conway, Date not defined) ... 20

Figure 5. Analytical model with hypotheses structure ... 21

Figure 6. Mean and Skewness Plot ... 26

Figure 7. Means plot of Content Quality per Gender ... 29

Figure 8. Means plot of Content Quality by Age ... 30

Figure 9. Perceived Ease of Use regressed on Interaction with an Instructor, Interaction with Peers and Mobile Learning (N=210). ... 32

Figure 10. Perceived Usefulness regressed on Mobile Learning (not significant), Content Quality, Spaced Repetition, Microlearning, Gamification (N=210) ... 34

Table 1. Variables scale description ... 25

Table 2. Correlation of variables in the model 1 ... 27

Table 3. Correlation of variables included in the model 2 ... 28

Table 4. Descriptive statistics ..……….26

Table 5.ANOVA results………..……….29

Table 6. Descriptive statistics………...…27

Table 7. ANOVA results………..……….30

Table 8. Descriptive statistics of Model 1 ... 32

Table 9. Regression Model of Model 1 ... 32

(6)

Table 11. Regression Model of Model 2 ... 34 Table 12. Hypotheses summary ... 34

(7)

Introduction

Since 1975, with the introduction of IBM 5100- first portable computer (Hope, 2018), the potential of electronic devices changed irreversibly. Computers started to be used in text processing, graphic design and very complex calculations (Alfredo, 2017). Digital

technologies started to be a competitive advantage in business models. After the advent of personal computers, the second factor that boosted heyday of digital business is the invention and popularization of the internet. It enabled instant information flow that changed the reality of the way how business is conducted worldwide. The development of computers and internet bandwidth improved the efficiency of companies.

At the beginning of 21st century, the era of Web 2.0 begun. It was a revolution because it fostered the flourish of social media and other interactive crowd-based communication tools (Dentzel, 2014). Along with Web 2.0, e-learning industry started to grow in power. Between 2002-2005, the estimated number of users enrolled in online courses grew by 65% according to National Center for Educational Statistics. Growing potential of technological solutions caused that people started to see untapped potential in the digital business industry.

Entrepreneurs identified a niche in e-business. Between 2011-2012, 4 biggest e-learning platforms were launched: Coursera, edX, Udacity, and FutureLearn. Founders of those companies matched needs of people, that want to enhance their knowledge, with new technological solutions. They believed that using e-business means, people might be willing to give up on the traditional way of getting knowledge in favor of online courses.

E-learning changed the perspective on learning industry forever. Nowadays, distance learning is not an obstacle. Millions of people enrolled in online courses to gain new knowledge, improve their situation on labor market or to be promoted. The Internet gives flexibility. There is no risk of trying new things because if a person finds the course irrelevant, he can just never come back to it. The size of e-learning industry is growing almost exponentially.

(8)

Coursera and Udacity were placed 10 and 12 respectively in the ranking of top 50 disruptors of 2016 (CNBC, 2016). The size of online learning market was estimated to be over USD 165 billion dollars in 2016 (Docebo, 2016). The demand for online learning services is still

growing as well as people’s expectations. E-learning courses have to be top quality, perfectly designed and provided by top experts from the field.

E-business industry develops with the emergence of technologies. People live more and more in the online world. Therefore, the potential of online education is still not covered. Because of that, the main motivation of this thesis was to define crucial factors that determine

customer satisfaction from using e-learning solutions. The end results of this paper aimed to help in building a strategy for an online course. The author built framework where he tried to cover all the relevant fields of e-learning process and include information about new

technologies and learning methods.

1. Literature Review

This chapter provides an overview of the e-learning topic. Firstly, there is a need to formulate an adequate definition of e-learning which covers all the relevant factors which will make the definition extensive and complete. Subsequently, various types of the online learning will be described. Thirdly, define who is the target audience and how the attitude towards them changed from the perspective of e-learning courses providers. Finally, all the features, that are important for customer satisfaction, will be listed.

1.1. Definition of e-learning

Since the beginning of 21st century, e-learning platforms are developing alongside the

development of internet technologies. Because of that, many various definitions of e-learning arose throughout years. Distance learning, which is often called ubiquitous learning, describes the ability to learn anywhere, at any time but it is only one part of the e-learning.

(9)

Analogically, personal learning shows only a small piece of online learning- the

self-organized type of education (Dominici & Palumbo, 2013). This is why there is a strong need to create a comprehensive definition of e-learning that would include all dimensions and functionalities. According to the inclusive definition of e-learning by (Sangrà, et al., 2012), there are 4 perspectives of online learning: 1) technology-driven, 2) delivery-system oriented, 3) communication-oriented, and 4) educational-paradigm-oriented. The authors of the

research found out that two first categories, which emphasize technology and accessibility of resources, are least effective definitions of e-learning itself. Taking all aspects into

consideration, authors created definition as follows:

“E-learning is an approach to teaching and learning, representing all or part of the educational model applied, that is based on the use of electronic media and devices as tools for improving access to training, communication and interaction and that facilitates the adoption of new ways of understanding and developing learning.” Furthermore, the experts involved in the research emphasized that two other aspects should also be mentioned when describing the essence of e-learning: 1) rapid development of technologies used for education, and 2) socioeconomic factors that maybe don’t have to be included in the definition itself but

nevertheless should be taken into consideration. This extensive definition covers main aspects that online education touches, but the reader should be aware that is very hard to build one universal answer that would be accepted by professionals with different backgrounds and academic profiles. There are several reasons behind it. Firstly, e-learning grows up with the development of digital technologies and innovations. Moreover, online can be understood from different perspectives and used for different means. Finally, society is in the state of a constant flux and its expectations are constantly growing and educational industry has to respond to those changes as fast as it is possible (Stein, et al., 2011).

(10)

1.1.1. Synchronous vs Asynchronous e-learning

There are two fundamental categories among the way how the educational content is delivered to the end user: synchronous and asynchronous e-learning.

In synchronous e-learning, meeting sessions are pre-scheduled. Lessons are conducted in real time. They are, in most cases, obligatory and essential to pass the course. During those learning sessions, a student has a chance to interact with a lecturer and other participants. Thanks to that, a learner has a possibility to get an immediate feedback on his work progress and can collaborate with other participants (Redmond, et al., 2007). This solution is especially useful in fields where content is changing dynamically (Prountzos, 2015). It gives a teacher possibility to conduct a lecture with the most relevant topics which reflect current trends and will be most helpful for students in real life. On the other hand, live online learning is not that flexible because a course participant must fit into a specific schedule.

In case of asynchronous learning, a student decides when to learn, this is why this method is often called self-paced learning. Learner completes tasks at different times when it is most comfortable for him. This solution is preferred for people who are better organized and manage to complete all the assignments with appropriate effort and involvement on time.

1.1.2. Micro vs Macro e-learning

Online learning industry is strongly associated with long courses which last for a couple of weeks and finish with a certificate but with the nowadays fast pace of life, there is a growing need for small, well-structured portions of knowledge which can be absorbed within 10 minutes.

(11)

Figure 1. Differences between Micro- and Macro-Learning (Bersin, 2017)

Having said that, e-learning knowledge can be divided into two categories micro and macro learning. Micro-learning considers knowledge that a person needs now (Bersin, 2017).

Most usual forms of micro-learning are short videos or short texts that can be watched or read in a couple of minutes. Thanks to them, a person can understand new functionality of a tool or gain necessary insights which make a broader topic comprehensible. Examples of micro-learning vendors are Youtube, Grovo, Axonify but also social platforms like Twitter. On the other side, there is macro-learning. This is the technique that most of the people associate e-learning with. Most often, the purpose of macro-learning is to learn an entirely new subject. It is a well-thought-out decision and a person agrees to spend time and money to get knowledge from a new field. The largest worldwide providers of MOOC (Massive open online courses) are Coursera, edX, and FutureLearn (Docebo, 2016).

1.1.3. Text driven vs interactive vs simulation

There are also different ways how e-learning courses can provide knowledge to their

audience. According to (Ferriman, 2013), 3 types can be specified: text driven, interactive and simulation.

Text-driven is the most classic method of how knowledge can be presented. The content is rather simple, materials include text, images, and eventually audio recordings. The course is

(12)

not interactive and resembles the traditional offline method of giving classes where a teacher presents only powerpoint slides.

Interactive method to some extent is similar to the text-driven method, but a greater emphasis is put on visualizations. Courses contain many more graphics and charts. Greater interactivity is also achieved thanks to videos implementation.

Lastly, simulation is the most innovative way of providing knowledge. Content is presented in diverse ways to gain the attention of a student. There is a mix of illustrations, graphics, charts, videos, and gamification. Usually, after a theoretical part, there is a “try-it” section where a student can use his new skills in practice.

1.2. Wide audience of e-learning solutions

One of the biggest MOOC trends is that more and more online courses are targeted at high-schoolers (Docebo, 2016). A young teenager is an open-minded person who thinks intensively about his future. Moreover, a lot of young people are still trying to define their own career path that they would feel most comfortable in. They explore all the possible options and are willing to take a step back if needed. According to the survey conduct by Harris Poll, 69% of people between 18-34 years old agreed with the statement that they learn more from

technology than from people (PRWeb, 2017). Development of societies and rising acceptance of new technologies made millennials a very important target group for online course

providers. On the other hand, more professionals, at every stage of their corporate career, are interested in enrolling in an online course as well. According to Paw Research Center’s survey, 63% of people who are working are professional learners. It means that in last 12 month they took a course or got additional training to enhance their job skills (Horrigan, 2016). The reasons behind this are multiple. Sometimes, people see a gap in the market and want to specialize in a new field to start up a new business. In some cases, they want to gain new knowledge to be promoted. Lastly, some professionals are simply tired of their current

(13)

situation and they need to rebrand themselves. Due to the wide audience of e-learning

solutions, expectations towards them are high as they have never been before. On the market, there is plenty of educational organizations that are offering MOOC. Coursera is the leader in the e-learning market with 30 million registered users by the end of 2017 (Shah, 2018). All those factors mentioned above makes education industry very market-oriented. A student is no more perceived as a person who has to fit into a teaching system. In today’s world, a man who subscribes to an online course is treated as a demanding participant, who wants to get a high-quality product, tailored to his needs.

1.2.1. Student as a demanding course participant

According to (Docebo, 2016), the size of the whole learning market (including corporate e-learning) was estimated to reach the level of USD 165 Billion in 2015 and is likely to grow 5% CAGR (Compound Annual Growth Rate) in years 2016-2023. It is clearly visible that online learning market potential is still untapped. Online courses providers understand that perfectly and executives, around the world, are trying to figure out how to attract a learner. That being said, a student of an e-learning course is no more treated as a regular one who has to fit in the well-known, established system. It meets with resistance from the side of

experienced educators who claim that the fact that something works in business, does not guarantee a success in education (Mark, 2013). It is impossible to copy solutions one to one because those are two completely different worlds which have their own laws. Nevertheless, Total Quality Management principles, which claim that the customer determines the level of quality, are implied. Nowadays, a student is more and more perceived as a customer who also demands and wants to find a solution personalized to his needs. According to (Mark, 2013), a learner is satisfied when the final product that he receives is high-quality and contain

(14)

e-learning among professionals, course relevancy on the job market is becoming one of the key issues. In case of the education industry, it is very challenging to measure customer

satisfaction. On the one side, there is a customer whose expectations have to be met to make him happy. On the other side, there is a course provider who has to not only make the course nice and easy for a participant but also has to provide a high-quality level of knowledge which may be challenging for some of the students.

1.3. Measuring customer satisfaction

1.3.1. The Technology Acceptance Model (TAM)

TAM was introduced by (Davis, 1986) and (Davis, et al., 1989). The model is based on the Theory of Reasoned Action which is widely used in social psychology. TRA is a general model and it doesn’t predefine people’s beliefs for a particular behavior that is being

investigated. In case of TAM, there are two main determinants that influence acceptance of an information system: perceived usefulness and perceived ease of use.

Perceived usefulness (PU) is described as “user subjective probability that using a specific application system will increase his or her job performance within organizational context” (Davis, et al., 1989)

Perceived ease of use (PEOU) is defined as "the degree to which a person believes that using a particular system would be free from effort" (Davis, et al., 1989)

(15)

Behavioral intention to use (BI) is influenced by customer’s attitude towards using a given technology (ATT). ATT has two determinants perceived ease of use (PEOU) and perceived usefulness (PU). Furthermore, PU has an independent effect on BI and PEOU influences PU. Taking all these elements into consideration, the TAM’s equation is as follows:

BI= ATT + PU

There are two main goals of the Technology Acceptance Model. Firstly, to predict and evaluate user satisfaction before the implementation phase of the project. Secondly, it should enhance understanding of user acceptance process and provide theoretical insights regarding information system design and implementation (Davis, 1986).

2.3.2. Determinants of customer satisfaction from e-learning use

Based on literature analysis and author’s personal experience three main building blocks of user satisfaction from the use of e-learning solutions were distinguished:

1. Social factors

2. Content quality factors 3. Technological factors- a. Mobile learning i. Spaced learning ii. Microlearning iii. Gamification 1. Social factors

Definitely, the biggest difference between online and offline education is how students communicate with a lecturer and other colleagues. Due to the fact, that student and professor never meet in person, it is very hard to establish a proper relationship between them. Lack of

(16)

such a relationship may cause a feeling of being anonymous among course members, which can negatively influence student’s involvement in assigned tasks. An instructor should interact with students as much as he can (Bolliger, 2004). Some lecturers even call every student before the beginning of the course. According to interviewed person: “A personal touch made a huge difference”. Another method is to communicate with students in a less formal way. Students feel more comfortable and are more willing to engage in the course and to interact with other colleagues (Yoshimura, 2008).

H1: Well-established contact with a teacher positively effects perceived ease of an e-course use

H2: Well-established contact with peers positively effects perceived ease of an e-course use

2. Content quality factors

According to (Statista, 2018), the number of internet users worldwide grew by 349% between 2005-2017, which gives 11% of average annual growth. Still, internet usage is the highest in developed countries but the increase in usage is the highest in emerging markets (Wong & Sixl-Daniell, 2017). Online courses should address current needs of the labor market. It is very important for today’s economy because it gives societies chances to develop. For developing countries, like the Philippines and South Africa, it is a chance to change young generations and teach them how to write and read, for a relatively low cost (Wong & Sixl-Daniell, 2017). In case of developed markets, young students can fill their gaps in education or change working field (Cooper, 2016), what results in the lower unemployment rate and more satisfied society.

Furthermore, the quality of the course determines learner satisfaction from eLearning (Sun, et al., 2008). Materials have to be from reliable sources and written in appropriate language style.

(17)

3. Technological factors

It is not easy to learn a completely new topic, especially in a new field. Big online courses provide students with lots of material that can be overwhelming. It is rather a rare situation when a person enrolls in an e-learning course and it becomes his main occupation. In most of the cases, e-course is an additional activity which someone tries to combine with professional and private life.

According to studies, students who learn intensively just before final exams lose much of their knowledge just a couple of weeks after. On the other hand, students who learn regularly are able to retain knowledge for much longer time (Bersin, 2017).

a. Mobile learning

Low cost of communication and constantly enhancing the quality of smartphones led to the worldwide proliferation of mobile devices (Nam, et al., 2012). The emergence of technology created new opportunities for learning. E-learning takes place in the radically different environment than people are used to, and it changes people perception of how education might look like (El-Hussein & Cronje, 2010). There are multiple definitions of mobile

learning. Some of them emphasize the technological part and focus on a device itself as a tool for knowledge transfer. On the other side, there is a definition that looks more at the learning experience part and how mobile learning stands out from other learning methods, even e-learning (Traxler, 2007). In this paper, the author wants to stress both sides as equally important components to accomplish customer satisfaction from mobile learning use. The emergence of mobile applications in e-learning is very dynamic. Online education platforms provide a mobile version of their courses to make the learning process more convenient. Applications are intuitive and user-friendly. M-learning is a natural extension of e-learning (Brown, 2005) and should not be treated as a substitute. It enables to reach a wider audience

(18)

with personalized content. That being said, looking more from the technological perspective, mobile learning makes education easier and more useful (more personalized). Therefore, the hypothesis is as follows:

H4a: Mobile learning positively affects perceived ease of use and perceived usefulness of an e-course

H4b: Mobile learning positively affects perceived ease of an e-course use H4c: Mobile learning positively affects perceived usefulness of an e-course

Looking at the educational experience part of mobile learning, there are many new concepts that might enhance the efficiency of the learning process. That being said, next chapter will describe three promising solutions that might change user experience and help to retain knowledge longer.

i. Modern learning techniques

Ability to adjust to constant changes and to learn fast is elementary factors for individual quality of life (Gassler, et al., 2004). Technological development constantly redefines limits of online education. Spaced learning, microlearning, and gamification are new concepts which seem to be most promising and effective in a mobile use.

The Ebbinghaus forgetting curve and spaced learning

The Ebbinghaus forgetting curve shows how people tend to forget knowledge if they do not revise it. Human’s brain is designed in that way, that repeated retrieval of information is essential to remember things after a time (Karpicke & Roediger, 2007)

(19)

Figure 3. Ebbinghaus forgetting curve

According to Figure 3, after one day, human remembers less than 40% remember of

information that he acquired on a day 0. The graph shows how crucial it is, to review learned material over time. That being said, the goal of an online course should be not only to teach quickly and cover the greatest number of topics but to provide solutions which will minimize forgetting.

Spaced repetition

Spaced repetition is a learning technique that incorporates reviewing previously learned material. Each revise is done after a longer interval of time. The first one is done immediately after class, the second one after 24 hours, third after a week and fourth after a month. Thanks to this method, a student is not only able to remember new information longer but also, he will forget less after a time.

0.00% 20.00% 40.00% 60.00% 80.00% 100.00%

Immediately 20 Minutes 1 Hour 9 Hours 1 Day 2 Days 6 Days 31 Days

Ret

ent

ion

(%

)

(20)

Figure 4. Overcoming the forgetting curve (Conway, Date not defined)

H5: Spaced repetition positively affects perceived usefulness of an e-course ii. Microlearning

Microlearning, as it has already been discussed, is a learning method that divides big chunks of a material into smaller pieces. This method seems to be appropriate to implement in mobile applications because it makes mobile learning very efficient. The solution also helps in

collaborative learning. A student can ask a colleague about a particular small piece of material that he found difficult to understand. It easier to point out a problematic issue.

H6: Microlearning positively effects perceived usefulness of an e-course iii. Gamification

Def: “Gamification is the application of game-design elements and game principles in non-game contexts”.

In addition to traditional learning materials only, course providers can create a gamification app, which is very interactive and can be very helpful in making the learning process easier. H7: Gamification positively affects perceived usefulness of an e-course

(21)

Figure 5. Analytical model with hypotheses structure

3. Data and Method

3.1. Design

To answer the research question and test defined hypothesis, a quantitative research was created. The study took the form of an online survey. It was administrated via Qualtrics software. In order to complete the survey at 100%, a respondent had to answer all the questions. This type of setting, allowed to avoid missing values. In data analysis, only fully completed surveys were taken into consideration. The online version of a survey was selected because it gave a possibility to access the broadest audience of e-learning users. Moreover, the survey was held in English for the same reason.

3.2. Sample

As it has been already mentioned, the data were collected online. At first, I approached coordinators of courses on the Coursera platform that were created by the University of

(22)

Amsterdam. My idea was to collect responses among students of these courses, but at the end, it did not end up with success. Because of that, I decided to collect sample among Facebook users that were found on Facebook groups which gather people from a particular e-course. Thanks to it, I was able to gather responses from people that were already experienced with online education.

The survey was collected between 7th and 23rd of May and a total of 268 respondents took part in it. 210 persons finished it completely. The only missing values were in those 58 cases where a person closed the questionnaire before completion, so those values were assumed to be missing completely at random (MCAR) and deleted listwise. Moreover, frequency tests did not prove any errors in collected answers. Participants were categorized by four variables: gender, age, education and employment status.

Sample consisted of 125 females (59,5%) and 85 males (40,5%). Most participants were in age 18-24 (n= 67, 31,9%), 25-34 (n=70, 33,3%) and 45-54 (n=32, 15,2%). Furthermore, 70 respondents (33,3%) had their bachelor’s degree and 84 respondents (40%) had master’s degree. From these figures, it could be concluded that the biggest sample were people that either had just finished education and were relatively short in the labor market or were doing their master’s program. The statement was also confirmed by the analysis of the employment status. 129 participants confirmed that they were currently either full-time workers (n=78, 37,1%), part-time workers (n=28, 13,3%) or they were self-employed (n=23, 11%). The second biggest group were people that still had a student status (n=70, 33,3%). 206 persons (98,1%), claimed that they had already taken an e-course during their life. This was crucial for this study because I wanted to measure preferences of people who had already tried e-learning methods and were knowledgeable to answer, what, in their opinion, are the most crucial features of an online course.

(23)

3.3. Measures

The model used in this study consisted of seven external variables. All constructs were analysed using Likert scale.

Content quality

Content quality was a variable that measures the importance of quality of a content presented during an e-course. It included topicality of presented subjects, language style and diversity of means that were used to present the content.

To measure the perceived importance of the content quality, five-point Likert scale (1=Not at all important, 5=Extremely important) from (Paechter, et al., 2010) was used. The construct consisted of 4 Likert scale items. Cronbach’s Alpha of the scale equaled 0.655 which was not a satisfying score. Therefore, I conducted an additional reliability test that shows potential Cronbach’s Alpha if one item from the construct is removed. According to the results,

removal of any item would not enhance the Cronbach’s Alpha result. The - score was in the 0.65-0.7 range, that being said it could be accepted, but also it has to be mentioned that the consistency and reliability of answers are weaker.

Interaction with a teacher

To measure the importance of the interaction with a teacher during duration of an online course five-point Likert scale (1=Not at all important, 5=Extremely important) from (Paechter, et al., 2010) was used. The construct consisted of 4 Likert scale items. The scale was reliable with the Cronbach’s Alpha= 0.841.

Interaction with peers

To validate the importance of the interaction with other course participants five-point scale (1=Not at all important, 5=Extremely important) was used. Moreover, the construct consisted

(24)

of four items and it was taken from (Paechter, et al., 2010). According to test, the scale was reliable with the Cronbach’s Alpha=0.849.

Mobile learning

In order to measure attitude towards the use of a mobile device during learning process, a seven-point Likert scale (1=Strongly disagree, 7=Strongly agree) was used. The construct consisted of 5 items, which were taken from (Motiwalla, 2007). In the reliability test, Cronbach’s Alpha=0.761.

Spaced repetition

Seven-point Likert scale (1=Strongly disagree, 7=Strongly agree) was used to measure the attitude towards spaced repetition. It was a four-item construct, which I created by myself. Cronbach’s Alpha, during the reliability test, equaled 0.685. The value did not reach 0.7 threshold, but it was still in 0.65-0.7 range, therefore it could be accepted.

Microlearning

To measure the attitude towards microlearning, seven-point Likert scale (1=Strongly disagree, 7=Strongly agree) was used. The construct consisted of 4 items, which I created by myself. The scale had high reliability with Cronbach’s Alpha=0.817.

Gamification

Attitude towards gamification was measure by a seven-point Likert scale (1=Strongly disagree, 7=Strongly agree). The construct consisted of 4 items, which I created by myself. Cronbach’s alpha=0.643, therefore I conducted an additional reliability test with a scale that showed potential Cronbach’s Alpha if an item was removed from the scale. According to the results, removal of item 4- Gamif_4_r improved Cronbach’s alpha to 0.845. Because of that, Gamif_4_r was not included in the further analysis.

(25)

Perceived ease of use

Perceived ease of use was one of two variables taken from Technology Acceptance Model. To measure its value, 7-point Likert Scale (1=Strongly disagree, 7=Strongly agree) was used. The construct consisted of 5 items and originated from three different papers (Paechter, et al., 2010), (Leng, et al., 2017) (Ramírez-Anormaliza, et al., 2015). The scale had sufficient reliability with Cronbach’s Alpha= 0.723.

Perceived usefulness

Perceived usefulness was second of two variables taken from Technology Acceptance Model. Its value was measured through 7-point Likert Scale (1=Strongly disagree, 7=Strongly agree). The constructed consisted of 5 items which originated from (Leng, et al., 2017). The

Cronbach’s Alpha= 0.712 which was a sufficient score.

Table 1. Variables scale description

4.

Results

4.1. Data preparation

After conducting all the necessary frequency tests to check missing values and errors, I looked for counter indicative questions in my survey. I identified only one such a variable: Gamif_4 and recoded to Gamif_4_r by reversing the scale to reflect the real intention of given answers.

(26)

Furthermore, as it had been already mentioned, all of the variables were built upon Likert scale. The Likert scale had the ordinal scale of data. That being said, to provide the normality test, I had to compute a mean value for each of 9 variables in my dataset. Normality test was based on the analysis of Q-Q plots, histograms, skewness and kurtosis. Based on this analysis, only three variables had normal distribution: Interaction with an instructor, interaction with peers and perceived ease of use. Rest of them characterized with negative skewness that so it means that their distributions had left tail longer. Content quality and mobile learning were approximately symmetric (skewness less than 0.5) and have high kurtosis (~1) which indicates that distribution is pointy. Gamification was also approximately symmetric (= -0.465) and had low Kurtosis score (=0.191), what was a sign of a flatter distribution. Distribution of spaced repetition and microlearning were moderately negative and pointy. In case of perceived usefulness, the distribution was substantial negative (Skewness=-1.336) and very pointy (Kurtosis=5,119).

(27)

In order to reduce the impact of bias, I decided to transform variables to normalize their distribution. Almost all variables were transformed by the equation: X*= √𝐾 − 𝑋, where K is the highest value of variable X, plus 1. Only in case of perceived usefulness, due to

substantial skewness, I used following equation X*=Log10(K-X), where is the highest value of variable X, plus 1 (Howell, 1992), (Tabachnick & Fidell, 2012). After the transformation, all the variables had normal a distribution.

The transformed variables were named as follows: C_QLTY_SQRT, MOB_LEARN_SQRT, SPC_REP_SQRT, MIC_LEARN_SQRT, GAMIF_SQRT, PU_LG10.

Next, I checked correlation between variables included in the analysis. I divided them into two sets models. The first model consisted of Perceived Ease of Use as a dependent variable, and Interaction with an Instructor, Interaction with Peers and Mobile Learning as independent variables. In the second model, on the other hand, Perceived Usefulness was a dependent variable, and Content Quality, Spaced repetition, Microlearning and Gamification are independent variables.

(28)

Table 3. Correlation of variables included in the model 2

According to the Table 2, there was statistically significant, positive, correlation between Interaction with Peers and Interaction with an Instructor, as well as Interaction with Peers and Mobile Learning. But the strength of linear correlation between those variables was weak (below 0.5), which was a statistically satisfying result (Emory & Cooper, 1991). The smiliar situation was in Table 3, where correlations between independent variables of model 2 were presented. The only two variables that were not correlated in a statistically significant way were Gamification and Content quality. Analogically to Model 1. all variables had weak correlations. Having said that, both Model 1 and 2 fulfilled the condition of non-collinearity of independent variables. This was one of the assumption needed for the regression analysis, which was conducted in the next step of the paper.

ANOVA analysis

As the next step in my analysis, I ran ANOVA test to check if there were differences of means between two or more populations.

As factors, I used four categorical variables: Gender, Age, Education, Employment Status. As dependent variables, I chose all the remaining variables: Content quality, Interaction with an Instructor, Interaction with Peers, Mobile Learning, Spaced learning, Microlearning,

(29)

check if any of 4 categorical variables included in this research had statistically significant effect on any of 9 continuous variables.

In the first step, I set Gender as an independent variable. The ANOVA test was significant only for the relation between Gender and Content Quality, F(1, 207)= 10.476, p<0.05. The assumption of homogeneity of variances was checked and confirmed by using Levene’s test, F (1.207)= 0.137, p= 712>0.05. That being said, there was significant effect of Gender on the perceived importance of Content Quality in e-learning. However, the actual difference of means between two genders was small η²= 0.048 (Cohen, 1988). Therefore, men perceived the quality of e-learning content as a bit more important in comparison to women.

Table 4. Descriptive statistics Table 5. ANOVA results

Figure 7. Means plot of Content Quality per Gender

In the second step I set Age as an independent variable. The ANOVA test was significant for the Content Quality F(6, 202)= 4.627, p<0.05 and for the Interaction with Peers

(30)

fulfilled. Levene’s test for both variables ended up being not statistically significant,

F(6,202)=1.913, p>0.05 and P(6,202)=0.816, p>0.05, respectively. Post hoc comparison was conducted and it showed that there were significant pairwise differences only between mean scores of Age and Content Quality.

Tukey post-hoc test revealed that content quality was significantly more important for people under the age of 18 than for people in age of 25-34, p<0.05 and in age of 45-54, p<0.05.There was also a significant difference between people in age of 18-24 and 25-34, p<0.05.

No significant differences between means scores of Age and Interaction with Peers were found.

Table 6. Descriptive statistics Table 7. ANOVA results

Figure 8. Means plot of Content Quality by Age

In case of two remaining categorical variables: Education and Employment Status, either the ANOVA test was not significant for any continuous variable (Education as a dependent

(31)

variable), or the post-hoc comparison did not reveal any statistically significant pairwise differences between the mean scores (Employment Status as a dependent variable). Therefore, there were no significant results that would contribute to the outcomes of the research. Having said that, no further ANOVA analysis were proceeded.

4.2. Regression analysis

As a next step in my data analysis, I conducted two multiple regression analyses to investigate relations between independent variables (predictors) and dependent variables (criterions). The outcomes of the analyses aimed to show which external variables could significantly predict the value of two criterions- Perceived Ease of Use (Model 1) and Perceived Usefulness (Model 2). Concerning the analytical model presented in the previous chapter, regression analysis was the appropriate analytical methodology to predict two key

determinants of student’s intention to use e-learning (Liawa, et al., 2007).

In the first stage, I conducted a multiple regression on the Model 1. It consisted of Interaction with an Instructor, Interaction with Peers and Mobile Learning as independent variables and Perceived Ease of Use as a dependent variable. The created model 1 was statistically

significant with F (3,206) = 92.818, p<0.001. 3 predictors explained the variance of Perceived Ease of Use in 57.5%.

The Mobile Learning recorded the highest standardized Beta value; =0.433; p<0.001 (In the table the value was -0.433, but at the beginning of the statistical analysis I had reflected the variable to normalize it, so now, I needed to reverse the direction of the interpretation as well). Second highest standardized Beta value had Interaction with an Instructor, =0.386; p<0.001, and at the last place Interaction with Peers =0.243; p<0.001. That being said, if a person’s mobile learning importance perception increases by 1 than Perceived Ease of an e-course Use increases by 0.433. Analogically, if person’s importance of Interaction with an

(32)

Instructor or Interaction with peers increases by 1 than Perceived Ease of Use rises by 0.386 or 0.243 respectively. That being said, with the regression analysis of Model 1, hypotheses: 2,3 and 4b were supported.

Table 8. Descriptive statistics of Model 1

Table 9. Regression Model of Model 1

Figure 9. Perceived Ease of Use regressed on Interaction with an Instructor, Interaction with Peers and Mobile Learning (N=210). All reported standardized regression weights were significantly different from zero (p<0.05)

(33)

Secondly, I conducted a multiple regression analysis on the Model 2. The purpose was to investigate and measure the ability of 4 independent variables: Content Quality, Spaced repetition, Microlearning and Gamification, to predict the value of Perceived Usefulness of an e-course. In the first step of the hierarchical regression, two predictors were entered: Age and Education Status.

Created Model 2, was statistically significant with F(5, 204)= 32.523, p<0.001. 4 out of 5 predictors were statistically significant (p<0.05). The only predictor that was not significant was Mobile Learning, = -0.057; p=0.352. Therefore, it was not taken into consideration in terms of Perceived Usefulness value prediction. 4 remaining predictors explained the variance of Perceived Usefulness in 44.4%.

The Gamification recorded the highest standardized Beta value; =0.443; p<0.001. The second highest standardized Beta value had Microlearning, =0.220; p<0.001, then Spaced repetition =0.169; p<0.05 and at the last place Content Quality =0.162; p<0.05. That being said, if a person’s gamification usefulness perception increases by one than the perceived usefulness of an e-course increase by 0.443. Analogically if person’s attitude to Microlearning or Spaced repetition or Content Quality enhances by 1, then Perceived Usefulness rises by 0.22, 0.169 and 0.162 respectively.

That being said, with the regression analysis of Model 2, hypotheses: 1, 5, 6 7 were supported. Due to statistical insignificance of Mobile Learning in Model 2, hypotheses: 4a and 4c were not supported.

(34)

Table 11. Regression Model of Model 2

Figure 10. Perceived Usefulness regressed on Mobile Learning (not significant), Content Quality, Spaced Repetition, Microlearning, Gamification (N=210). All reported standardized regression weights (besides Mobile Learning) were significantly different from zero (p<0.05)

(35)

5.

Discussion and conclusions

This final chapter of the thesis elaborates about results that were presented in the previous chapter. Firstly, the general discussion will be presented, where all relevant results are interpreted and discussed. The first section is then followed by practical implementation, where discussed outcomes are interpreted in broader perspective of the real market and how those findings can be linked to nowadays’ questions and problems. Finally, limitation and suggestions for future studies were presented.

5.1. General Discussion

The paper was designed to be valuable and useful from theoretical as well practical

perspective. This study introduced a new framework that consisted of newest technologies, like mobile learning, and conventional learning methods, like gamification. It aimed to boost a discussion regarding creative but at the same efficient and convenient ways of transferring knowledge through internet. Practically wise, the research could be treated as a guideline during the user experience design stage of an e-course creation. The conclusion of this paper should outline, on the general level, what customer perceives valuable and important. The above-mentioned goals were accomplished through 3 main findings.

Model 1, explained 57.5% of the variance in perceived ease of e-learning use, what is a very satisfying score. It means that 3 factors included in this model reflect user preferences as well. Among listed factors, mobile learning had the strongest effect on perceived ease of use. The result proves that nowadays people heavily rely on a technology during their daily activities (Collins & Halverson, 2018). In this study, 65,4% of respondents were 18-34 years old, so it can be concluded that the biggest target group of e-learning are young people-millennials. Moreover, according to the (Becker, et al., 2013) findings, the older students are, the greater reluctance to e-learning adoption. Such a young audience has very little reluctance towards

(36)

using technology. The real question is, how good the offered technology has to be to make them willing to use it. That being said, authors of e-learning should take a great care of the quality of technology offered in a course Furthermore, the second strongest effect on

perceived ease of use had interaction with a teacher. It proves that people are still used to the traditional learning methods, where contact with a teacher is easily accessible. The effect was only a bit weaker than mobile learning’s effect. It is an important reminder for course creators that while great emphasis should be put on new technologies, they cannot forget about

traditional elements of education to which people are used since they started their school education. The weakest effect had contact with peer students. According to the findings, it still significantly enhances people’s ease of online learning use, but it is not a top priority. The model 2 explained 44,4% of the variance in perceived usefulness of online learning. The score is lower, in comparison to the Model 1, but it is still significant. The score is even more satisfying when taking into consideration diversity of factors included in the Model 2. From one side there is gamification, which is a very modern learning technique and from the other side, there is spaced repetition which has been known for decades. According to the results, the strongest influence on the perceived usefulness of an e-course has gamification. Second strongest influence has microlearning, followed by content quality and spaced repetition. Last two predictors had almost the same effect size on the dependent variable. That being said, young people (65,4% of respondents were in the age of 18-34) who are either still studying or have just started their professional career are really open to new learning concepts like

gamification. Moreover, statistically significant were also microlearning and spaced

repetition. These two learning concepts are very important and are not broadly included in the literature, especially not in the context of perceived usefulness of an e-course. Microlearning enables division content to smaller pieces and spaced repetition is a method to revise learned material over time. These concepts enable a person to retain knowledge for a longer time what

(37)

is desired by every person that starts a learning process. Lastly, content quality is also perceived to be very useful. People find an e-course useful when it is up-to-date with the latest information from a field, when it reflects current needs of the labor market and when it is presented in a diversified way.

Thirdly, according to the findings of the research, there are not many significant discrepancies between groups derived from categorical variables like gender, age, education or employment status. The only significant differences of means were found in the perception of content quality by gender (Male perceived content quality a bit more important) and by age (People in the age of 18-34 perceived content quality more important than people in the age of 25-34 and 45-54). However, based on the findings, actual differences between means were small. That being said, it can be assumed that these seven factors included in this analysis are universal for the people who are interested in online learning. From one side, some can say that the results are then too obvious, but from the other side, lack of discrepancies might be a good sign, because it may show that this paper included factors which are really important for every e-learner despite gender, age, education or employment status.

5.2. Theoretical and Practical Implications

This study brings theoretical, as well as practical implementations to the topic of online learning. Part of previous studies focused more on the features of e-learning that originated from the traditional education system- interaction with an instructor and peers, learning achievements etc. (Paechter, et al., 2010). It is understandable for studies that were made in 2010 or earlier. Mobile technology just started to flourish than, and the biggest e-learning platforms were yet to be launched. At that time people were not fully aware of possibilities and how technology can link the world of education. Furthermore, there are many studies that tried to implement Technology Acceptance Model (TAM) to online learning. Some of them

(38)

focus on the model itself, by examining how perceived usefulness and ease of use influence behavior intention to use (Masrom, 2008) or (Al- Adwan, et al., 2013). Some of them

examine only one external factor, like social influence, that impacts the entire model (Farahat, 2012). That being said, from the theoretical perspective, this research, with seven external variables, is quite unique.

Firstly, it combines traditional learning practices with new concepts like gamification and mobile learning. The results proved that meanwhile, traditional elements are very important for the users, new technologies are the difference makers. The outcomes revealed that e-learning is a multidimensional field where new technologies are its integral part. Therefore, the approach of the future studies should be analogical. Researchers should focus not only on the traditional elements of a learning process but also survey and analyze newest concepts in the e-learning field.

Secondly, all the factors measured in the research were set into TAM. In the paper, I defined precise features of e-learning as external factors and matched them to either perceived ease of use or perceived usefulness. Based on the results, it can be concluded that these links

explained the variances really well. Even though, features were defined very precisely, I was able to retrieve desired results. That being said, I think that there is an untapped potential of TAM in the studies concerning e-learning because it reflects the needs of a student really well. Furthermore, the model can be enhanced with new elements to make the study case specific.

The research is also very practically oriented. At the early stage of the study, based also on my personal experience, I tried to answer the question: What are the key features of an e-course? By doing so, I wanted to map out determinants that influence learner’s satisfaction the most. In the theoretical framework, I included a wide range of features that significantly

(39)

diversify from each other. From one end, there is a traditional analysis of the learning materials, and from the other end, there are new learning methods like mobile learning. Results showed that young audience of e-courses has a high appreciation for technology and novel solutions in the learning process. Mobile learning turned out to be the strongest predictor for ease of use value. Authors of e-courses should put great emphasis on creating a convenient mobile application that would be an integral part of a course. It does not have to replace a web-browser version of a course, but it should be a complementary mean of

studying. Moreover, the results of the paper revealed that gamification and microlearning are techniques that students perceive as very useful. Those elements can be easily implemented into the mobile application as well. Small portions of knowledge can be taught during gamification, so those two techniques can be even combined. Moreover, the last feature that respondents found useful is spaced repetition. Revising material over time helps you to keep knowledge for longer. Spaced repetition and microlearning have the strongest correlation among all dependent variables. It shouldn’t be surprising, because revising materials is the easiest and most convenient in small portions. Therefore, there is untapped potential in spaced repetition which can be implanted in both mobile application, as well as a web-browser version of an e-course. That being said, there is still plenty of space for development in the online learning industry. This study revealed, that to address effectively young target group of online learning, the combination of traditional and novel means of teaching is required.

Mixing digital and analogical solutions will result in user’s high intention to use and high user satisfaction.

(40)

5.3. Limitation and future research

First of all, I was unable to get access to a database of participants of any particular e-course. I approached coordinators of several courses on Coursera but unfortunately with no success. Conducting a research on the sample of random people that have taken at least one e-course in their lifetime forced me to generalize a bit the framework of the study.

I wanted conclusions to be applicable to the broadest scope of online courses possible and be a good starting point for further research. That being said, there is potential in conducting a research among participants of e-courses from different learning fields. I do believe that people’s perception of ease of use and usefulness may significantly vary between participants of humanistic, technological or art courses. Firstly, students’ personality is much different. People who take coding course are probably much more skilled in using new technological tools than people who study the history of design. Secondly, methods of knowledge transfer are different. In case of a language course, for instance, vocabulary revision can be easily implemented into a mobile app, but in case of a coding course more complex formulas have to be made on a computer so the mobile application might be less important.

Furthermore, in this study, I precisely defined several features that can be implemented in an e-course. Because, I focused my analysis on the one side of the TAM- external variables, I wanted to keep the second side of the model standard. This is why the only two elements that influence behavior intention to use are perceived ease of use and perceived usefulness, but in future studies, this framework can be expanded by additional factors. In (Tseng & Hsia, 2008) for instance, authors added perceived flexibility of a course as an additional determinant. The proposal for a new variable was supported by the analysis of working habits in Taiwan. Taiwanese employees work very late and this is why authors concluded that perceived flexibility may significantly impact user’s intention to use e-learning. That being said,

(41)

the extension of TAM can be based on the analysis of a target group that we would like to examine.

(42)

Bibliography

Al- Adwan, A., Al- Adwan, A. i Smedley, J., 2013. Exploring students acceptance of

e-learning using Technology Acceptance Model in Jordanian universities. International Journal of Education and Development using Information and Communication Technology , 9(2), pp. 4-18.

Alfredo, D., 2017. Biz Fluent. [Online]

Available at: https://bizfluent.com/about-5299598-history-computers-business.html [Data uzyskania dostępu: 2018].

Anon., brak daty Wikipedia. [Online]

Available at: https://en.wikipedia.org/wiki/Coursera

Balcikanli, C. i inni, 2013. Identifying factors that contribute to the satisfaction of students in e-learning. Turkish Online Journal of Distance Education, 14(1), pp. 85-101.

Becker, K., Newton, C. i Sawang, S., 2013. A learner perspective on barriers to e-learning. Australian Journal of Adult Learning, 53(2).

Bersin, J., 2017. John Bersin. [Online]

Available at: https://joshbersin.com/2017/03/the-disruption-of-digital-learning-ten-things-we-have-learned/

Bolliger, D., 2004. Key Factors for Determining Student Satisfaction in Online Courses. International Journal on E-Learning, 3(1), pp. 61-67.

Brown, T. H., 2005. Towards a model for m-learning in Africa. International Journal on E-learning, 4(3), pp. 299-315.

CNBC, 2016. CNBC. [Online]

Available at: https://www.cnbc.com/2016/06/07/2016-cnbcs-disruptor-50.html Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences. brak miejsca:Lawrence Erlbaum Associates.

Collins, A. i Halverson, R., 2018. Rethinking education in the age of technology: The digital revolution and schooling in America. brak miejsca:Teachers College Press.

Conway, Date not defined. [Online]

Available at: https://sites.google.com/a/swinn.k12.ia.us/mr-conway-s-history/mr-conway-s-classroom/importance-of-and-tips-for-note-taking

(43)

Cooper, S., 2016. eLearning Industry. [Online]

Available at: https://elearningindustry.com/job-prospects-with-elearning-improve-ways [Data uzyskania dostępu: 2016].

Davis, F., 1986. A technology acceptance model for empirically testing new end-user information systems : theory and results. Doctoral dissertation, Massachusetts Institute of Technology.

Davis, F., Bagozzi, R. i Warshaw, P., 1989. User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science, 35(8), pp. 982-1003. Dentzel, Z., 2014. Open Mind. [Online]

Available at: https://www.bbvaopenmind.com/en/article/internet-changed-everyday-life/?fullscreen=true

[Data uzyskania dostępu: 2018].

Dick, A. i Basu, K., 1994. Customer Loyalty: Toward an Integrated Conceptual Framework. Journal of the Academy of Marketing Science, 22(2), pp. 99-113.

Docebo, 2016. ELEARNING MARKET TRENDS AND FORECAST 2017-2021, brak miejsca: brak nazwiska

Dominici, G. i Palumbo, F., 2013. How to build an e-learning product: Factors for student/customer satisfaction. Business Horizons, 56(1), pp. 87-96.

El-Hussein , M. O. M. i Cronje, J. C., 2010. Defining Mobile Learning in the Higher Education Landscape. Educational Technology & Society, 13(3), pp. 12-21.

Emory , C. W. i Cooper, D. R., 1991. Business Research Methods. 4th red. brak miejsca:Richard D Irwin.

Farahat, T., 2012. Applying the Technology Acceptance Model to Online Learning in the Egyptian Universities. Procedia - Social and Behavioral Sciences, Tom 64, pp. 95-104. Ferriman, J., 2013. LearnDash. [Online]

Available at: https://www.learndash.com/3-types-of-elearning/

Gassler, G., Hug, T. i Glahn, C., 2004. Integrated Micro Learning–An outline of the basic method and first results, brak miejsca: Research Studio eLearning Environments; ARC Seibersdorf research GmbH.

Hope, C., 2018. Computer Hope. [Online]

Available at: https://www.computerhope.com/issues/ch000984.htm [Data uzyskania dostępu: 2018].

(44)

Howell, D. C., 1992. Statistical Methods for Psychology. 3rd Edition red. brak miejsca:International Thomson Publishing.

Karpicke, J. i Roediger, H., 2007. Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, Tom 57, pp. 151-162.

Leng, N. S. i inni, 2017. E-Learning acceptance based on Technology Acceptance Model (TAM). Journal of Fundamental and Applied Science, 9(4S), pp. 871-889.

Liawa, S.-S., Huang, H.-M. i Chenc, G.-D., 2007. Surveying instructor and learner attitudes toward e-learning. Computers & Education, 49(4), pp. 1066-1080 .

Mark, E., 2013. Student satisfaction and the customer focus in higher education. Journal of Higher Education Policy and Management, 35(1), pp. 2-10.

Masrom, M., 2008. Technology acceptance model and E-learning. Kuala Lumpur, International Conference on Education.

Motiwalla, L., 2007. Mobile learning: A framework and evaluation. Computers & Education, 49(3), pp. 581-596.

Nam, M.-W., Park, S. Y. i Cha, S.-B., 2012. University students’ behavioral intention to use mobile learning: Evaluating the technology acceptance model. British Journal of Educational Technology, 43(4), pp. 592-605.

Paechter, M., Maier, B. i Macher, D., 2010. Students’ expectations of, and experiences in e-learning: Their relation to learning achievements and course satisfaction. Computers & Education, 54(1), pp. 222-229.

Prountzos, V., 2015. Talent LMS. [Online]

Available at: https://www.talentlms.com/blog/organization-choose-self-paced-learning-vs-live-online-learning/

PRWeb, 2017. PRWeb. [Online]

Available at: http://www.prweb.com/releases/2017/07/prweb14552192.htm [Data uzyskania dostępu: April 2017].

Ramírez-Anormaliza, R., Sabaté, F. i Guevara Viejo, F., 2015. Evaluating student acceptance level of e-learning systems. Sevilla, International Association of Technology, Education and Development (IATED).

Redmond, J. A., Parkinson, A., Mullally, A. i Dolan, D., 2007. Synchronous e-Learning: Three Perspectives. Innovations in E-learning, Instruction Technology, Assessment, and Engineering Education, pp. 175-180.

(45)

Sangrà, A., Vlachopoulos, D. i Cabrera, N., 2012. Building an Inclusive Definition of E-Learning: An Approach to the Conceptual Framework. The International Review of Research in Open and Distributed Learning, 13(2), pp. 145-159.

Shah, D., 2018. Class Central. [Online]

Available at: https://www.class-central.com/report/mooc-stats-2017/ [Data uzyskania dostępu: 2018].

Statista, 2018. Statista. [Online]

Available at: https://www.statista.com/statistics/273018/number-of-internet-users-worldwide/ Stein, Shephard i Harris, 2011. Conceptions of learning and professional development for e-learning held by tertiary educators in New Zealand. British Journal of Educational

Technology, 42(1), pp. 145-165.

Sun, P.-C., Tsai, R., Finger, G. i Yeh, D., 2008. What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), pp. 1183-1202.

Tabachnick, B. G. i Fidell, L. S., 2012. Using Multivariate Statistics. brak miejsca:Pearson. Traxler, J., 2007. Defining, Discussing, and Evaluating Mobile Learning: The moving finger writes and having writ… .. International Review of Research in Open and Distance Learning, 8(2), pp. 1-13.

Tseng, A.-H. i Hsia, J.-W., 2008. An Enhanced Technology Acceptance Model for E-Learning Systems in High-Tech Companies in Taiwan: Analyzed by Structural Equation Modeling. Hangzhou, Institute of Electrical and Electronics Engineers.

Wong, A. i Sixl-Daniell, K., 2017. The Importance of e-Learning as a Teaching and Learning Approach in Emerging Markets. International Journal of Advanced Corporate Learning. Wranx Blog, 2016. Wranx. [Online]

Available at: https://blog.wranx.com/ebbinghaus-forgetting-curve

Yoshimura, M., 2008. Educators in American Online Universities: Understanding the

Corporate Influence on Higher Education. Journal Journal of Education for Teaching, 34(4), pp. 295-305.

(46)

Appendix 1. The survey

The questionnaire is designed to measure the most important features that determine user satisfaction from an e-learning course use. The results of the survey will be used in the Master's Thesis of Pawel Janczarski (The University of Amsterdam, Faculty of Economics and Business).

Thank you very much for questionnaire in the survey. All responses are anonymous. Collected data will not be given or sold to any third parties.

The survey takes approximately 3 minutes.

Q1 What is your gender?

o

Male

o

Female Q2 What is your age?

o

Under 18 (1)

o

18 - 24 (2)

o

25 - 34 (3)

o

35 - 44 (4)

o

45 - 54 (5)

o

55 - 64 (6)

o

65+ (7)

(47)

Q3. What is the highest degree or level of school you have completed? (If you are currently enrolled in school, please provide the highest degree you have right now)

o

Less than high school

o

High school graduate

o

Professional degree

o

Bachelor's degree

o

Master's degree

o

Doctorate

Q4. What is your employment status?

o

Employed full time

o

Employed part-time

o

Self-employed

o

Student

o

Unemployed looking for work

o

Unemployed not looking for work

o

Retired

Q5 Have you ever taken an e-course?

o

Yes

(48)

Q6. To what extent, below features are important concerning content quality? Not at all important Slightly important Moderately important Very important Extremly important Up-to-date material with recent, relevant insights (1)

o

Diversified content that shows subject from different perspectives using multiple sources (2)

o

High quality, academic style of language (3)

o

Course material reflects current need on the labour market (4)

o

Q7 To what extent, below features are important concerning interaction with the instructor?

Not at all important Slightly important Moderately important Very important Extremely important Easy and fast

accessibility of the instructor

o

Fast feedback from the instructor

o

Support of learning by the instructor

o

Possibility to establish personal contact with the instructor

o

(49)

Q8. To what extent, below features are important concerning interaction with other course participants (peers)? Not at all important Slightly important Moderately important Very important Extremly important Easy and fast

exchange of information and knowledge with peer students

o

Variety of communication tools for exchanging information with peer students (e.g., e-mail, chat, newsgroups)