DATA-DRIVEN RECOMMENDATION MODEL FOR E-LEARNING

DATA-DRIVEN RECOMMENDATION MODEL FOR

E-LEARNING

SUBMITTED IN PARTIAL FULLFILLMENT FOR THE DEGREE OF MASTER

OF SCIENCE

Michelle Lin

12198153

MASTER INFORMATION STUDIES

Information Systems

FACULTY OF SCIENCE

UNIVERSITY OF AMSTERDAM

Date of Defense: 4 July 2019

1st Examiner Dr. Davide Ceolin

Faculty of Science, Vrije Universiteit Amsterdam

2nd Examiner Dr. Frieke Box

ABSTRACT

Technology has become a powerful tool for educators to introduce various sources and initiate online educational activities to promote communication and interaction with students and learners. Many young people are increasingly using social network sites to interact with professors and peers. E-learning has been demonstrating its po-tential for boosting knowledge acquisition and improving academic performance, yet it also faces the challenges of high redundancy of learning material, information overload and learning isolation. Personalizing learning experience using students’ learning style has been applied to many studies to achieve effective online learning. As people are acting more and more on social network sites, we want to investigate whether people’s online behavior could also define their learning styles and recommend learning objects to achieve ef-fective results. This paper presents a recommendation model using data from users’ interaction with an online learning platform Moo-dle and social network site Sina Weibo, to predict learning styles and recommend suitable learning objects to learners. This model was compared with the traditional questionnaire model, and both were tested on a sample of students. The results were validated through t-test on the two types of behaviors. Although there was one significant difference found on one of the learning styles, the overall results did not show strong evidence that this model could produce more accurate results than the traditional model.

KEYWORDS

E-Learning, Learning Styles, Online Learning, Social Network Sites, Social Network Behavior

1

INTRODUCTION

In a traditional classroom situation, teachers usually teach classes to a group of students at the same time in the same place. Bourkoukou et. al(2017) called this "one size fits all approach" which provides only one educational experience[3]. Students are provided with the same learning materials, and receive the same style of teaching without considering individuals’ need and the habit of learning. Furthermore, it is almost impossible for teachers to seek the best teaching strategy to meet the demand of every learner, instead, the learner often has to adapt themselves to the traditional education system. Therefore, poor adaptability and ineffectiveness are the common problem with traditional learning[32].

E-learning is a form of training or teaching that takes place over the internet or intranet[27]. E-learning has become an increasingly popular trend of education development as people can learn anywhere at any time. However, the rapid expansion of e-learning has brought challenges in appropriately distributing learning objects (LOs), such as videos, graphics, presentations, and e-books, to achieve positive educational experiences that fit the needs, goals, and interests of their learners[3]. It is also facing the challenges of high redundancy of learning material, information overload and learning isolation[29].

One way to address these issues is using recommender sys-tem (RS) techniques to personalize learning experience based on students’ learning styles. Learning style, which refers to a set of factors, behaviors, and attitudes used by the individual for facilitat-ing individual learnfacilitat-ing[4], has been playfacilitat-ing a significant role in personalized e-learning systems. Many have studied how learning styles can be used to personalize e-learning systems. Different theories and models are used to measure learner personalities and learning styles, in order to support e-learning. As people are spending a great amount of time on social network sites nowadays, social network sites are also considered as tools for enhancing e-learning. Thus it is interesting to discover the correlation between students’ online behavior on social network sites and their learning styles. This paper introduces a recommendation model using data from users’ engagement on an online learning platform Moodle and social network Sina Weibo, to recommend suitable learning resources to learners. In this study, we aim to access and validate social network analysis as a method to measure learning styles and recommend learning objects on e-learning environment. By obtaining and analyzing the users’ online social interaction data, we predict the learning styles of the users, then recommend different forms of learning resources to them for online learning. By comparing the learning behavior during online learning, we evaluate the effectiveness of the users’ learning experience on the recommended objects. The main research question in this thesis is can the data-driven model generates more accurate prediction than traditional questionnaire-based model on recommending learning objects. To answer this question, we need to answer the subquestions by investigating in both models that,

(1) What is the information collected?

(2) What is the model used for defining learning styles? (3) How is the information matched with different learning

styles?

(4) How much can the performance of e-learning be improved by using the data-driven model?

The thesis is organized as follows, Section 2 presents an overview of the academic uses of emerging e-learning systems and learning styles. Section 3 illustrates the hypothesis and the implementation of the proposed model. Section 4 demonstrates and discusses the study results. Section 5 explained the limitation of this research. Sec-tion 6 concludes this research and demonstrates the future works.

2

LITERATURE STUDY

2.1

Learning Style Models

According to Yilmaz-Soylu and Akkoyunlu (2009), the preferences on the way of giving meaning and acquiring information of every person can vary even though people all have common bio-psychological and social characteristics in learning process. The process of giving meaning can have individual-specific differences, which is the learning style discussed in this study [33]. One of the early studies emphasized the correlations between students’ 2

learning styles and tool preferences by using Felder-Silverman Learning Style Model[8] and the associated Index of Learning Styles (ILS) questionnaire to identify student’s learning styles, and using another set questionnaire to find out learners’ preferences on tools(including wiki, podcast, blogs etc.)[26]. 89 students in a Web programming course participated into this research and responded to both questionnaires. The authors applied the Pearson correlation to discover significant findings between learner style and tool preferences. For example, intuitive learners (who are always prepared to try new things and discover possibilities, according to Felder-Silverman Learning Style Model) showed preference on blogs. Sequential learners (who usually follow logical stepwise paths to gain understanding) would prefer podcasts.

Lau and Lee (2010) used VAK learning model to investigate how different learning styles can affect students’ perceptions on the effectiveness of different e-learning services, tools and content[16]. 31 students participated in this research were categorized as visual learners (who perceive information the best with knowledge presented in photos, diagrams, tables etc[28]), auditory learners (who present high effectiveness when listening to lectures, discussing in groups and talking things out[28]) or kinesthetic learners (who learn best through feeling and doing[28]) by VAK learning model. The learning style was defined by a dedicated inventory and students’ opinions were collected through Likert Scale survey[16]. The findings suggests that both visual and auditory students regarded wikis as highly helpful services; auditory students also considered blogs useful; kinesthetic students showed preference on media sharing services.

2.2

Learning Styles in E-Learning Environment

Many researches on e-learning systems focus on the learner profile based on learning styles for recommendation. Learning style is one of the learners’ characteristics, as well as the individual manner in which they approach a learning task[23]. McLoughlin (1999) advocated that when designing self-instructional learning materials, the learner differences should be taking into consideration[19]. She suggests instructional designer focusing on learning styles to inform the design of adaptive learning material[19]. A programming tutoring system, Protus, built by Klasnja-Milicevica et. al(2011) recommended learning resources to learners based on their habits and learning styles[13]. Protus tested learners’ learning styles and analyzed learners’ server logs in order to identify the patterns of every learning style. Both works indicate that using learning style could bring effective and efficient learning experience to learners.

2.3

Learning style towards Web 2.0

Web 2.0 [20] constitutes a cooperative learning environment with its new form of participatory internet[7]. Web 2.0 has enabled using social network sites as an inevitable composition of most people’s activities on a daily basis. Especially among younger generations, which were found out to be the major users of social network sites by Madge et. al(2009)[17], they are turning to online social networking such as Facebook, Twitter and Whatsapp, to develop social relationships. In the context of education, social networking supports self-initiated learning by

allowing students to develop personal links amongst themselves[5]. It has enabled educational activities such as resource sharing, discussion participating, collaborating and so on. Integrating social media into the educational domain can bring tremendous benefits including bringing students with comfort and fun, encouraging involvement and engagement, decreasing dropout ratio, supporting collaborative learning, and incrementing the learning execution [15].

Utilizing the benefits of social networks sites on e-learning environment can bring significant effect on the domain of education. However, most researchers have focused on student’s social network activities in relation to their privacy concerns and other forms of developments[21], limited studies are found on the domain of social-network-based e-learning. Additionally, few significant results have been found on the works that focused on social-network-based e-learning. Therefore, in this study, we aim to examine the social-network-based e-learning integration and explore the difficulties and any issues may occur during the process.

3

METHDOLOGY

3.1

Hypothesis

The data-driven recommendation model can generate more accu-rate prediction based on students’ online behavior on social net-work sites. This approach can generate, evaluate, predict students’ learning styles and learner preferences more accurately than the questionnaire-based recommendation model.

3.2

Kolb’s Learning Styles

Learning style is the way in which each learner begins to concen-trate on, process, absorb, and retain new and difficult information [22]. Every person may interact with the information in different ways. Therefore, it is necessary to discover how to trigger, maintain, and react to each student’s concentration, in order to achieve long term memory and good results. A comprehensive learning style model produced by David Kolb [14] is one of the most popular schemes used to reveal natural tendencies and styles of learners. It conceives of individuals’ learning processes as differing along two dimensions: preferred mode of perception (concrete to abstract) and preferred mode of processing (active experimentation to reflective observations)[14]. The Kolb’s Learning Styles classifies individuals into four distinct learning styles on the basis of a four-stage learning cycle including Concrete Experience, Reflective Observation, Abstract Conceptualization, and Active Experimentation, as well as a four-type definition of learning styles, each representing the combination of two preferred styles, which Kolb categorized as Diverging, Assimilating, Converging and Accommodating as shown in Figure 1.

Two management development specialists, Peter Honey and Alan Mumford, has adapted Kolb’s Learning Styles into a questionnaire by using a four-way classification that closely resembles that of Kolb but is simplified for use in a practical training situation[12]. The questionnaire asks people to agree or disagree on a 4-point scale to evaluate themselves on whether they learn best through 3

working by themselves and assessing opinions, or they learn best through discussing and sharing with people.

Figure 1: A Diagram of Kolb’s Learning Styles

Kolb (1984) describes the characteristics of each style based on both research and clinical observation[14]; based on the description, Saowaluk et al (2014) has summarized the characteristics and suggested learning objects for each learning style as shown in figure 2 [30].

3.3

Approach

3.3.1 Proposed Model. We will approach a group of students who are currently studying advertising courses in Fuzhou Univer-sity in China. Students will be divided into two different groups, which are the questionnaire group and the data-driven group. Fig-ure 3 presents the proposed model that predicts students’ learning styles using different methods and suggests lecturer to recommend learning objects. In the questionnaire group, students will be asked to take the assessment, their learning styles will be revealed by scores from the questionnaire results. In the data-driven group, students’ online activities will be collected through Weibo group discussion and Moodle online learning platform. Sina Weibo is the most popular microblogging service and one of the most used social network sites in China [9]. A Weibo study group will be formed to allow the instructor and students to regularly share post containing course information and resources, as well as react to the posts. Moo-dle is one of the most commonly used course management systems which enables educators to distribute and share information to stu-dents. it is also one of the most commonly used learning system as it is free and has enabled the creation of powerful, flexible and engag-ing online courses and experiences [24]. On Moodle, educators and students can produce content material, prepare assignments and tests, engage in discussions, manage distance classes and enable collaborative learning with forums, chats, file storage areas, news services [25]. A 5-session experiment will be conducted. Three of

the sessions allow students to actively engage into the discussion and interaction on both platforms. The data produced from students interacting with the social network site and the e-learning platform can be used to predict students’ learning styles in this group. Once the learning styles in both groups are identified, the lecturer is going to distribute the different learning objects according to the learning style of each student in two learning sessions, and observe the behavior during these sessions.

3.3.2 Evaluation. To measure whether data-driven model works better than questionnaire-based model, we use t-test on the total time spent and page visited to validate and compare the student’s online learning behavior between the data-driven group and the questionnaire group. P-value will be used in the t-test to indicate whether the result is true. To discover more significant differences of student behaviors between using the two models, we will also test whether there is a significant difference in each pattern of behavior between the data-driven group and the questionnaire group. After learning sessions completed, students in the data-driven group will be asked to fill the Kolb’s Learning Style Questionnaire to compare with the results obtained through online activities. One more survey will be conducted to discover how students rate both methods.

3.3.3 Data Collection. To gather data for identifying students’ learning styles, two surveys were created. Both surveys had asked students’ to provide demographic information such as name, gender, age, as well as their GPA scales. For students in the questionnaire group, the survey is followed by another 80 questions from Kolb’s Learning Styles Questionnaire. Once the questions are all filled, scores will be added up to determine the strength of preference, then matched to the learning styles. For students in the data-driven group, they were asked to provide their Weibo account information for forming the study group on the social network site. Their behavior data will be collected from both Moodle and Weibo group. The online behavior data to be collected was on the basis of the study Halawa carried out in 2016 [11]. He selected and analyzed the attributes in Figure 4 The result of the analysis produced more accurate evaluation and prediction of student personalities, and has proved to have positive change in the student commitment and engagement to the courses in the e-learning system [11]. Therefore in this study, the same attributes and pseudocode will be borrowed to collect data and predict student’s learning styles.

A Weibo group has been created by the instructor. Students are encouraged to actively engage in the discussion in this group. The data of each post will be scraped for analyzing the students’ online behavior. The data scraped from the Weibo group includes user names, posts, owner of the post, numbers and students of sharing the posts, numbers and students of liking the post, time of the posts, comments on the posts. On Moodle, the administrator/lecture can view user reports which include all activities and logs that students have performed on the site. Figure 5 shows one example of the activity logs of one student from the data-driven group on the first day of Moodle interaction. Moodle log reports are easy to obtain and can provide all the attributes needed for learning style identification.

Figure 2: Characteristics and Suggested Learning Objects of Kolb’s Learning Styles

Figure 3: Proposed Model

4

RESULTS AND ANALYSIS

4.1

Experiment Results

In this study, 15 students filled the survey to join data-driven group, and 29 students filled the questionnaire to join questionnaire group. There were 37 students out of these 44 students having registered on Moodle platform. By filtering out students who didn’t have sufficient data for comparing and analyzing, we obtained data of 7

Figure 4: Attributes Used for Analyzing Learning Styles in the Data-Driven Group

Table 1: Distribution of Learning Styles Between Two Groups

Learning Style Data Driven Group Questionnaire Group Accommodating 2 3

Assimilating 2 5

Converging 1 4

Diverging 2 4

Total 7 16

students in data-driven group and 16 students in the questionnaire group as shown in Table 1.

Figure 5: An Example of Moodle Log Report

4.2

Online Behavior

We compared the average values of the two groups based on the time spent on the e-learning platform and the number of page vis-ited. As shown in Figure 6 and Figure 7, students in the data-driven group appears to spend more time during learning sessions and the total five sessions, than students in the questionnaire group. When comparing the pages viewed between two groups, students in the data-driven group also performed more activities than students in the questionnaire group. To take a detailed view into the differ-ences on each learning style between the two groups, we had found out that students in the data-driven group averagely spent more time than students in the questionnaire group (shown in Figure 8). When looking into the pages visited, the results in Figure 9 show differently, that with accommodating learning style, students in the questionnaire group visited more pages in average than students in the data-driven group.

Figure 6: Comparison of Average Time Spent between Two Groups

Figure 7: Comparison of Average Pages Visited between Two Groups

Figure 8: Comparison of Average Time Spent on Each Learn-ing Style between Two Groups

Figure 9: Comparison of Average Pages Visited on Each Learning Style between Two Groups

4.3

t-Test

The behavioral data collected from students was processed with t-test to validate and compare the results between the two groups. The independent t-test, also called the two sample t-test, independent-samples t-test or student’s t-test, is an inferential statistical test that determines whether there is a statistically significant difference be-tween the means in two unrelated groups [1]. In t-test, a p-value can help determine the significance of the results, which is commonly set at 0.05 as a standard for 95% probability. In the common ground, a p-value of 0.05 (5%) is accepted to mean the data is valid [6]. If the p-value is less than 0.05, it indicates strong evidence that the observed result did not occur by chance. According to [2], t-test re-quires roughly balanced design, such as same number of subjects in each group. It suggests that extremely unbalanced designs increase the possibility that violating any of the requirements/assumptions will threaten the validity of the Independent Samples t Test [2]. In this study, the number of students varies largely between the two groups, thus we randomly chose samples for each learning style from the questionnaire group to match the samples size with the data-driven group, so that the t-test design can be balanced to produce a more accurate outcome. In each group, there will be data of 2 accommodating learning styles, 2 assimilating learning styles, 1 converging learning style, and 2 Diverging learning styles. Table 3 presents the basic statistics between the two groups. To avoid statistical errors, it is necessary to check the assumption of normality before any inferential or statistical hypothesis test to examine the validity of data [10]. In this study we will be using Shapiro-Wilk Test to start with the data validation. Additionally, as [1] suggests, the Independent Samples t-Test needs the assumption of homogeneity of variance tested such as both samples having the same variance. A test for the homogeneity of variance, called Lev-ene’s Test, is often run before processing an independent samples t test.

4.4

Validation

4.4.1 State the Hypotheses. The null hypothesis for this t-test is that the two means of the data-driven group and questionnaire group are equal:

H0 : µDataDrivenGroup = µQuestionnaireGroup In this study, we want to find out if it is possible to reject the null hypothesis and accept the alternative hypothesis, to show that the two means between the two groups are not equal:

H1 : µDataDrivenGroup , µQuestionnaireGroup 4.4.2 Shapiro-Wilk Test. The Shapiro-Wilk test investigates the correlation between the data and the corresponding normal scores, and has been recommended by many researchers as the best choice for testing the normality of data [10]. Table 2 shows the Shapiro-Wilk test results that all Sig. values are larger than 0.05, which

means the data is normally distributed. This indicates that both data from Total Time Spent and Total Page Visited we would compare in the following discussion are normal, thus we can proceed with Levene’s Test and t-test.

4.4.3 Levene’s Test. The Levene’s test sets a precondition for the t-test to assess variance homogeneity. Table 4 presents the test

Table 2: Results from Shapiro-Wilk Test Group n W Skewness Sig. Result Total Time Spent DDG 7 0.911 0.172 0.457 Normal QG 7 0.869 -0.583 0.200 Normal Total Page Visited DDG 7 0.931 0.419 0.641 Normal QG 7 0.907 1.209 0.423 Normal *DDG means Data-Driven Group

*QG means Questionnaire Group

Table 3: Group Statistics

Group N Mean Std. Deviation Std. Error Mean Total Time Spent Data-Driven 7 225.43 59.05 22.32 Questionnaire 7 188.00 42.49 16.06 Total Pages Visited Data-Driven 7 93.00 21.74 8.22 Questionnaire 7 76.71 7.36 2.78

Table 4: Levene’s Test for Equality of Variance

Levene’s Test for Equality of Variance F Sig. Total Time Spent Equal Variance

Assumed 1.392 0.261 Equal Variance

Not Assumed

Total Page Visited

Equal Variance

Assumed 7.482 0.018 Equal Variance

Not Assumed

results, where F represents the test statistic of Levene’s test and Sig indicates the p-value corresponding to this test statistic. The p-value of this test shows 0.261 when testing Total Time Spent, so we can conclude that the variance between the two groups is equal. It tells that we should proceed t-test that assumes equal variances. When looking at the Total Page Visited, it has a significance of 0.018 then the assumption has been violated. In this case, we will continue the t-test that assumes unequal variances.

4.4.4 t-Test. The outcome of Levene’s test has indicated how to read the t-test results in Table 5. To find out whether there is a significant difference in Total Time Spent between the two groups, we need to look at the "Equal Variances Assumed" row. The P value presents as 0.198 is larger than 0.05. It demonstrates that it is not unlikely enough for rejecting the null hypothesis. We usually conclude that the two group means are equal, and there is no significant difference on Total Time Spent between the two groups. The Sig. value on Total Page Visited shows 0.018 in the Lev-ene’s test. This means the variances are not equal and we should read the "Equal Variances Not Assumed" output in the t-test result table. We found 0.103 on p value when comparing Total Page Visited. Based on this number, we do not reject H0 in favor of H1 because 7

Table 5: t-Test Results for Student Behaviors t-Test of Equality of Means

t df p (2-tailed) Mean Difference Std. Error Difference Total Time Spent Equal Variance Assumed 1.361 12 0.198 37.43 6.26 Equal Variance Not Assumed 1.361 11 0.201 37.43 6.26 Total Pages Visited Equal Variance Assumed 1.877 12 0.085 16.29 5.43 Equal Variance Not Assumed 1.877 7 0.103 16.29 5.43

there is not enough evidence to prove the H0 is wrong. Therefore, we conclude that there were no significant differences in Total Page Visited means between data-driven group and questionnaire group.

4.5

Discussion

Although we observed that students in the data-driven group averagely spent more time and visited more pages on Moodle than students in the questionnaire group during online learning, the t-test indicates that the behaviors we observed are likely to be a result of chance. It could not validate this conjecture as it fails to reject the null hypothesis. As a result, neither the comparisons of means in Total Time Spent nor in Total Page Visited is statistically significantly different. There is insufficient evidence to suggest that the data-driven model produces more significant different results than questionnaire model. The probability of data-driven model generating more accurate prediction than the traditional questionnaire-based model on recommending learning objects is considered low.

Furthermore, we also conducted Levene’s tests and t-tests on each learning style between the two groups to find out whether significant difference occurs in any particular learning style (converging learning style has been left out as there is only one sample). Table 7 presents the results of Levene’s test on each learning style, and Table 8 demonstrates the results of t-test corresponding to the outcomes of Levene’s test. There appears a p value less than 0.05, which shows 0.012 on Total Page Visited in Assimilating group and indicates there is a definite difference on Total Page Visited on students with assimilating learning style between the data-driven group and the questionnaire group. This does not inform directly which group has a larger value, since it shows a definite difference, by browsing the group statistics on Table 6, it tells that the mean of Total Page Visited for the data-driven group is 97.00 and the mean of Total Page Visited for the questionnaire group is 77.00. This explains that the data-driven model works better than questionnaire model for the Total Page Visited on students with assimilating learning style, however, it is not sufficient to validate that data-driven model works better than questionnaire model on all learning styles.

Table 6: Group Statistics on Each Learning Style Learning

Style

Beha--vior Group N Mean Std. Deviation Std. Error Mean TTS DD 2 191.50 6.36 4.50 TTS Q 2 195.00 35.36 7.00 TPV DD 2 70.00 1.41 1.00 Accomm--odating TPV Q 2 69.50 2.12 1.50 TTS DD 2 220.50 108.19 4.93 TTS Q 2 226.00 8.49 6.00 TPV DD 2 97.00 1.41 1.00 Assim--ilating TPV Q 2 77.00 2.83 0.29 TTS DD 2 276.50 31.82 22.50 TTS Q 2 173.50 48.79 14.50 TPV DD 2 95.00 24.04 2.00 Diver--ging TPV Q 2 76.50 2.12 0.21 *DD means Data-Driven *Q means Questionnaire

*TTS is Total Time Spent *TPV is Total Page Visited

At the end of the learning session, two surveys were dis-tributed to the students. One survey was run to all students in these two groups to found out how they rated the methods they took to predict learning styles. 8 students in data-driven group and 17 students in the questionnaire group completed this survey. 75% of students in the data-driven group and 71% of students in the questionnaire group think they learnt better with the learning objects recommended to them. Another survey was aiming at students in the data-driven group. We sent the Kolb’s Learning Style Questionnaire to this group of students and wanted to find out whether the outcomes are identical to the results using data-driven methods. There were 8 out of 15 students in the data-driven group who completed this survey. In this survey, three students have learning styles tested differently when taking the questionnaire, as highlighted in red in Table 9. The two methods have produced 62.5% of identical outcomes. 37.5% of the results are different, however we could not conclude which method is more accurate as the t-test failed to reject the null hypothesis or to prove there is significant difference.

Halawa’s work in 2016 shows strong evidence that his data-driven model is more accurate than questionnaire on predicting students’ learning style [11]. This study, however, did not generate the same result or prove there is significant difference between the two models. Several reasons have been considered to affect the outcome and validity of this study,

• Form of Participation: Halawa (2016) conducted the exper-iment and worked with students who were studying the whole course through Moodle [11]. Therefore, the students were motivated to fully participate into the whole learning process; so that rich data was produced. This study asked students to voluntarily participate into the experiment. They were encouraged to do this experiment for gaining additional knowledge which might potentially help them to get higher grades at the end of the course. As it did not mandatorily require students to work through the whole experiment, stu-dents may not have motivation to actively take part into the learning process.

• Sample Size: in the beginning, 44 out of 200 students in this course filled the questionnaires for starting the experiment, 8

Table 7: Levene’s Test for Equality of Variances on Each Learning Style

Levene’s Test for Equality of Variances Learning Style Behavior

F Sig. Accommodating Total Time Spent Equal Variances_Assumed 1.151 0.396

Equal Variances Not Assumed

Total Page Visited Equal Variances_Assumed 76.000 0.013 Equal Variances

Not Assumed

Assimilating Total Time Spent Equal Variances_Assumed 24.585 0.038 Equal Variances

Not Assumed

Total Page Visited Equal Variances_Assumed 3.698 0.194 Equal Variances

Not Assumed

Diverging Total Time Spent Equal Variances_Assumed 0.383 0.599 Equal Variances

Not Assumed

Total Page Visited Equal Variances_Assumed 59.381 0.016 Equal Variances

NotAssumed

Table 8: t-Test for Equality of Means on Each Learning Style t-Test for Equality of Means

Learning Style Beh--avior t df p (2 tai--led) Mean Differ ence Std. Error Differ ence Accomm--odating Total Time Spent EVA -0.138 2.000 0.903 3.500 2.500 Total Page Visited EVNA 0.277 2.000 0.808 0.500 0.500 Assim--ilating Total Time Spent EVNA -0.072 1.000 0.954 5.500 1.071 Total Page Visited EVA 8.944 2.000 0.012 20.000 0.714 Diver--ging Total Time Spent EVA 2.501 2.000 0.130 103.000 8.000 Total Page Visited EVNA 1.084 1.000 0.474 18.500 1.786

*EVA means Equal Variances Assumed *EVNA means Equal Variances Not Assumed

however as the experiment went longer, students’ interest in participating in the online sessions started to drop, and fewer and fewer students were participating in the sessions as the experiment continuing (shown in Figure 10). Halawa (2016) obtained a "40 vs 40" data sample at the end of his experiment [11], while this study was only able to collect a "7 vs 16" data sample for analysis. The small sample size and the unbalanced distribution of sample could have significant influence on the outcome of the experiment.

Table 9: Outcomes of Data-Driven Group Using Question-naire to Test Learning Styles

Student Name Data-Driven Method Outcome

Questionnaire Method Outcome

Yifan Tang Accommodating Diverging

Aiping Dai Accommodating Accommodating Ziyao Lin Diverging Diverging

Junjie Chen Diverging Diverging

Shuhao Zheng Diverging Assimilating

Dong Yuan Converging Converging

Rui Shen Converging Diverging Wenling Cai Converging Accommodating

Figure 10: How The Number of Students Decreases as The Experiment Went Longer

• GPA: GPA is another factor that may influence the outcome of this experiment. The sample size we obtained is relatively small, thus we did not take students’ GPA into account when analyzing the data. However, students’ knowledge level varies and it could affect their interest and knowledge perception in learning.

5

LIMITATION

Although researchers had stated that the integration of social network sites and e-learning could bring tremendous benefits, difficulties and challenges had been discovered from the study. Firstly, when students were studying different objects, the form of learning objects could take different time to consume. For example, Tsur (2014) suggested that 90 percent of information transmitted to the brain is visual, and visuals are processed 60,000 times faster in the brain than text [31]. Different learning objects could also affect the time spent and the rate that people retain information. However, when comparing the total time spent on the e-learning platform, taking these differences into account and balancing the differences have been considered an extremely difficult job.

Secondly, many social network sites have been imposing tighter privacy and security protection on users’ online data. It is now becoming more and more difficult to scrape online behavior data from users. In this study, we were unable to use Weibo API to scrape student’s online activities through social network sites. Alternatively, we manually counted the number of likes, number of shares and number of comments on the study group discussion page. In the future, or when it comes to larger scale experiments, accessing user online behavior data will be extremely difficult due to strict laws on privacy and security of user data.

Moreover, many teachers and students have been building a thick line between education and social life[18]. Some people may consider the integration of social settings and online education could be seen as violating or interfering people’s personal life. Connecting social network sites with e-learning systems could motivate the participation of some people, however others may reject it by considering it as a violation of privacy.

6

CONCLUSION AND FUTURE WORKS

This study proposed to use student’s online behavior data to predict students’ learning style and recommend corresponding learning objects to them. Based on Kolb’s learning style theory, two groups of students responded to the questionnaires and took part into the observing sessions and learning sessions. However, using the proposed model did not show a significant change or improvement in student commitment during e-learning. There was no sufficient evidence to prove that the data-driven recommendation model can generate more accurate prediction based on students’ online behavior on social network sites. This can be caused by several reasons such as lack of motivations from students, small and unbalanced sample size gained, and overlook of GPA fact. In the future, we will retry this model on a wider scale and a larger dataset under a more motivating study circumstance, in order to collect sufficient data to test its validity again. We also plan to test this model under different learning style models such as Felder-Silverman Learning Style Model, to find out whether learning style models also make differences to the outcomes. Combining the model with recommender system approaches (such as collaborative filtering) will be also a potential direction of future research.

REFERENCES

[1] [n. d.]. Independent T-Test for Two Samples. ([n. d.]). https://statistics.laerd.com/ statistical-guides/independent-t-test-statistical-guide.php

[2] 2019. SPSS Tutorials: Independent Samples t Test. (2019). https://libguides.library. kent.edu/SPSS/IndependentTTest

[3] Outmane Bourkoukou, Essaid ElBachari, and Mohamed ElAdnani. 2017. A Rec-ommender Model in E-learning Environment. Arabian Journal for Science and Engineering 42, 2 (2017), 607–617. https://doi.org/10.1007/s13369-016-2292-2 [4] Judith C. Reiff. 1992. Learning Styles. What Research Says to the Teacher Series.

(1992).

[5] Yu Ching Chen. 2015. Linking learning styles and learning on mobile Facebook. International Review of Research in Open and Distance Learning 16, 2 (2015), 94–114.

[6] Tukur Dahiru. 2008. P - value, a true test of statistical signifi-cance? A cautionary note. Annals of Ibadan postgraduate medicine 6, 1 (2008), 21–6. http://www.ncbi.nlm.nih.gov/pubmed/25161440%0Ahttp://www. pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4111019

[7] Jan P Ehlers and Kai Sostmann. 2013. Web 2.0 and social networks. (2013), Doc15 pages. https://doi.org/10.3205/zma000858

[8] Richard M Felder and Linda K Silverman. 1988. Learning and Teaching Styles In Engineering Education. Technical Report. Institute for the Study of Advanced Development. 674–681 pages.

[9] Qi Gao, Fabian Abel, Geert-Jan Houben, and Yong Yu. 2012. A Comparative Study of Users’ Microblogging Behavior on Sina Weibo and Twitter BT - User Modeling, Adaptation, and Personalization, Judith Masthoff, Bamshad Mobasher, Michel C Desmarais, and Roger Nkambou (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 88–101.

[10] Asghar Ghasemi and Saleh Zahediasl. 2012. Normality tests for statistical analysis: a guide for non-statisticians. International journal of endocrinology and metabolism 10, 2 (2012), 486–489. https://doi.org/10.5812/ijem.3505

[11] Mohamed Soliman Halawa, Essam M.Ramzy Hamed, and Mohamed Elemam Shehab. 2016. Personalized E-learning recommendation model based on psycho-logical type and learning style models. 2015 IEEE 7th International Conference on Intelligent Computing and Information Systems, ICICIS 2015 (2016), 578–584. https://doi.org/10.1109/IntelCIS.2015.7397281

[12] Peter Honey and A Mumford. 1992. The Manual of Learning Styles. Peter Honey. https://books.google.nl/books?id=CkiTQgAACAAJ

[13] Aleksandra Klašnja-Milićević, Boban Vesin, Mirjana Ivanović, and Zoran Budimac. 2011. E-Learning personalization based on hybrid recommendation strategy and learning style identification. Computers and Education 56, 3 (2011), 885–899. https://doi.org/10.1016/j.compedu.2010.11.001

[14] D.A. Kolb. 1984. Experiential learning: experience as the source of learning and development. Prentice Hall December (1984), 19–38. https://doi.org/10.1016/ B978-0-7506-7223-8.50017-4

[15] Kashif Laeeq, Zulfiqar Ali Memon, and Jamsheed Memon. 2018. The SNS-based e-learning model to provide smart solution for e-learning. (2018), 141–152. [16] Simon B Y Lau. 2010. Contextualising e-learning services and content for

com-puting course in higher education based on learning style and competency level Chien-Sing Lee. 5, 3 (2010), 211–242.

[17] Clare Madge, Julia Meek, Jane Wellens, and Tristram Hooley. 2009. Facebook , social integration and informal learning at University : ‘ It is more for socialising and talking to friends about work than for actually doing work ’. January 2016 (2009). https://doi.org/10.1080/17439880902923606

[18] Joseph P. Mazer, Richard E. Murphy, and Cheri J. Simonds. 2009. The effects of teacher self-disclosure via Facebook on teacher credibility. Learning, Media and Technology 34, 2 (2009), 175–183. https://doi.org/10.1080/17439880902923655 [19] Catherine McLoughlin. 1999. The implications of the research literature on

learning styles for the design of instructional material. Australasian Journal of Educational Technology 15, 3 (1999), 222–241. https://doi.org/10.14742/ajet.1859 [20] Tim O’reilly. 2005. What is web 2.0. (2005). https://www.oreilly.com/pub/a/

web2/archive/what-is-web-20.html

[21] Modupe M. Osokoya and Kazeem R. 2016. Social Media and Learning Styles as Correlates of Senior Secondary Students’ Chemistry Achievement in Abeokuta, Ogun State, Nigeria. Journal of Sociological Research 7, 2 (2016), 1–17. https: //doi.org/10.5296/jsr.v7i2.9902

[22] Harold Pashler, Mark Mcdaniel, Doug Rohrer, and Robert Bjork. 2009. Learning Styles Concepts and Evidence. Technical Report. www.learningstyles.net [23] Elvira Popescu, Florin Leon, Gheorghe Asachi, and Orulqq / Hrq. 2013.

Ana-lyzing the Influence of Learning Styles on Students’ Behavior in a Social Learning Environment Social Learning Environments. Technical Report. 657–662 pages. https://www.researchgate.net/publication/280490210

[24] William Rice. 2006. Moodle E-Learning Course Development. Packt Publishing. [25] Cristóbal Romero, Sebastián Ventura, and Enrique García. 2008. Data mining in

course management systems: Moodle case study and tutorial. Computers and Education 51, 1 (2008), 368–384. https://doi.org/10.1016/j.compedu.2007.05.016 [26] Nauman Saeed and Yun Yang. 2008. Using learning styles and preferences

to incorporate emerging e-learning tools in teaching. In Proceedings - The 8th IEEE International Conference on Advanced Learning Technologies, ICALT 2008. https://doi.org/10.1109/ICALT.2008.133

[27] JI Spurlock-Johnson, W Zhang, and L. Allen-Haynes. 2004. Can E-learning Replace the Traditional Classroom? A Case Study at a Private High School. Proc ISECON 21, 1 (2004), 1–7. http://citeseerx.ist.psu. edu/viewdoc/download?doi=10.1.1.574.2399&rep=rep1&type=pdf%0Ahttp: //www.proc.isecon.org/2007/2543/ISECON.2007.Zhang.pdf

[28] Herman Dwi Surjono. 2011. The Design of Adaptive E-Learning System based on Student ’ s Learning Styles. 2, 5 (2011), 2350–2353.

[29] Thoufeeq Ahmed Syed, Vasile Palade, Rahat Iqbal, and Smitha Sunil Kumaran Nair. 2017. A Personalized Learning Recommendation System Architecture for Learning Management System. Kdir (2017), 275–282. https://doi.org/10.5220/ 0006513202750282

[30] Saowaluk Thaiklang and Somjit Arch-Int. 2014. Learning resources recommen-dation framework using rule-based reasoning approach. Journal of Theoretical and Applied Information Technology 69, 1 (2014), 68–76.

[31] Michal Tsur. 2014. Research Confirms Video Improves Learning Results. (2014). https://www.huffpost.com/entry/ 10

research-confirms-video-i_b_5064181?guccounter=1&guce_ referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLm5sLw&guce_referrer_sig=

AQAAAEK4xudSHMXRJKisJHmgEIOwjEmMjHjEbzNj895Z3ZZKXiCLvkP99ybFR1tjWiJQvodj7_ 7KFI5XDr2cc-17UuFgqhUtfgtvt0fpKtlTFvMY

[32] Qingsong Tu and Jian Liu. 2011. Research on autonomous online education system based on intelligent recommendation. ITME 2011 - Proceedings: 2011 IEEE International Symposium on IT in Medicine and Education 2 (2011), 410–413. https://doi.org/10.1109/ITiME.2011.6132136

[33] Meryem Yilmaz-Soylu and Buket Akkoyunlu. 2009. The Effect of Learning Styles on Academic Achievement in Different Forms of Teaching. International Journal of Instruction 8, 4 (2009), 43 – 50. https://doi.org/10.12973/iji.2018.11316a