University of Amsterdam, Faculty of science
Designing a digital system to prevent social loafing in university
group projects
Thesis Master Information Studies - Human Centered Multimedia
Jop Brandenburg, 11670614
jop.brandenburg@student.uva.nl
Final version: July 4, 2018
Supervisor dr. Frank Nack University of Amsterdam Second examinor dr. Bert Bredeweg University of Amsterdam
Designing a digital system to prevent social loafing in university
group projects
Jop Brandenburg
UvA ID 11670614 jop.brandenburg@student.uva.nlABSTRACT
Collaborative learning has gained popularity in universities past years, causing an increase in project groups and social loafing in these groups. Research has suggested several different solutions with regard to social loafing prevention, but these involved non-digital solutions only. In order to explore the potential of non-digital ways to help prevent social loafing in project groups, a prototype of a system is designed and evaluated by means of the Technology Acceptance Model. This paper presents the designed system and the findings of eight interviews done to evaluate the system. Findings suggest that the proposed system’s features, including peer-reviews, task management and a warning-system, were re-ceived positively by the master students who were interviewed. However, suggestions on how the system could be improved are also given.
KEYWORDS
Social loafing, Free-riding, Project management, collaboration
1
INTRODUCTION
Group projects have become a very common phenomenon in uni-versities and high schools, due to their time-saving nature for teach-ers and professors, but also to the fact that group projects teach students how to efficiently work together [10]. Additionally, col-laborative learning brings benefits over individual learning when solving complex tasks [14, 15].
As the amount of group projects is increasing [10], the number of reports of certain group members relying on the rest of their team to do the majority of the work for them increases as well [10]. Hall et al. [10] found in 2013 that this phenomenon was the greatest concern across members of all faculties of Swinburne University of Technology in Australia.
The concept of people relying on their fellow group members has been studied deeply in the past forty years and was called social loafing by Latane et al. in 1979 [16]. Although it has been found not to be a universal issue [8], social loafing is experienced in many group projects [10] in higher education.
Several causes and potential solutions to social loafing have been suggested by different studies, such as peer-evaluation [3–5, 20], group size [3, 17], and documentation [11]. However, no research has been done on potential digital solutions to the social loafing problem. Digital solutions could take away numerous tasks from the supervisor and make managing group projects easier for stu-dents. Digital systems could also provide documentation and easier peer-evaluations for students.
This study looks into digital technology as a potential solution for social loafing and asks the following question:
Can digital systems help prevent social loafing in university group projects?
This paper provides some background into studies concerning so-cial loafing. It then provides a description of a system designed to answer the research question. Subsequently, the evaluation of the system is clarified. Lastly, the results, as well as shortcomings of present study are discussed and suggestions are made for future studies.
2
RELATED WORK
2.1
Causes
Social loafing, or ’free-riding’, has been the subject of past studies in which social loafing is attributed to several causes.
One of the causes is found to be the feeling of lowered impact by group members [16]. Group members often have the feeling that their contributions do not have a big influence on the final product of the group project, thus causing a lowering of motivation in these individuals, which then causes them to put less effort in their tasks [16].
Another potential cause of social loafing, one that might be con-nected to the feeling of lowered impact, is task visibility. Task visibility denotes the level at which the individual efforts are be-lieved to be visible to a supervisor or other group members [9, 17]. A low task visibility can cause individuals to believe that neither their efforts will be rewarded nor their slacking will be punished [17]. The believe that the relation between performance and conse-quences is fading, could lower motivation in individuals, causing them to put less effort in their tasks [9, 16]. This phenomenon is also called ’Hiding in the crowd’ [16].
Lastly, several studies mention group size [3, 17] as a probable cause to social loafing. Since bigger groups allow for more individual anonymity, a feeling of lower individual impact on the final prod-uct is often present. Furthermore, larger groups could also lead to lower task visibility, which in turn could lead to an increase in social loafing. [3, 9, 17].
2.2
Solutions
Throughout the years, many solutions were proposed to combat social loafing in group projects. Generally the strategy is to elimi-nate the causes mentioned above. This section will outline several solutions that have been found by researches over the past several years.
Firstly, a solution that has been found by multiple studies is intra-group peer reviewing [3–5, 20]. By letting intra-group members review each others’ work, social loafing is suppressed. Brooks [4] found that letting students peer review each others’ contributions dur-ing the project helped in providdur-ing feedback on all of the group’s
work. Brooks concluded that this evaluation system could reduce the free-rider problem and make the group project a more pleasant experience. Aggarwal [3] found that peer evaluation shows group members that there will be consequences for free-riding, thus in-creasing task visibility. Therefore, peer reviewing allows free-riders to adjust their behavior and contribute to the group project after all.
A second potential solution that was found to combat social loafing is smaller group size. Aggarwal [3] found that reducing group size was an accurate predictor for encountering less social loafing. By having small project groups, free riders have less of a shield of anonymity to hide behind. Also, when working in small groups, each group member feels like their contributions count more to-wards the end goal [3].
A third potential solution was introduced by Hansen et al. [11]. They proposed that individual contributions should be documented, in order to differentiate between individuals who perform in a group and individuals who do not perform in a group. This solution appears to be twofold. Firstly, it has the potential to increase task visibility, since the individuals’ contributions are out in the open, for all group members, or even a supervisor to see. This also allows under-performing group members to be confronted with the fact that they have been slacking, for example during peer evaluations or group meetings.
Secondly, when issues do arise in the form of social loafing, there will be detailed documentation on who contributed to the project and who has been free-riding, which can act as evidence and can then be used to properly solve the problem.
Finally, the formation of project groups is believed to be another potential solution. The formation of project groups can be done by students, lecturers, or randomly. According to Hansen [11] the formation of groups should not be done randomly, but either by students or by lecturers, where formations selected by the lecturer provide several educational benefits, such as more stability in group members and a more positive learning experience. Separately, Ka-rau et al. [13] found that group cohesiveness is also very important in the prevention of social loafing. They found that groups of friends are less likely to include social loafing when working on a project than groups of strangers, and attribute this to group cohesiveness.
2.3
Digital solutions
Research on potential digital solutions for the free-riding prob-lem are scarce. The studies mentioned above mostly aim for the supervisor to introduce a potential solution, such as controlling group sizes and making group members do peer-reviews, whereas a digital system could potentially prevent social loafing without interference from supervisors.
Since little research has been done on this subject, not much is yet known about such hypothetical systems. However, there are plenty of applications that aim to make the procedure within group projects easier, mostly by means of task management. This section will outline several of these applications.
2.3.1 Examples of existing applications.
A number of applications have been developed in recent years in order to facilitate task management in group projects. Most of these
applications offer an overview of the distributed tasks and some, such as Pintask1and Trello2offer deadlines and reminders to be added to a task, in order to get the task done on time. They also allow attachments to be uploaded to the tasks, which can be added to tasks for a variety of reasons. Some applications additionally offer priority-rankings3, nested tasks4, and automatically recurring tasks5.
Although these type of applications are quite popular to manage tasks in companies, they don’t offer much functionality that specif-ically addresses any of the social loafing-related issues mentioned above, and thus, do not address the issue of social loafing specifi-cally.
2.3.2 Project management.
A number of the above applications, such as Trello2and Pintask1 are based on principles from Kanban [1]. Kanban is a way of manag-ing projects. It aims to visualize the workflow through a board that shows the tasks assigned to each member of a group project [1, 12]. Kanban was first used in in the manufacturing industry in Japan, and was used primarily to schedule the manufacturing. However, nowadays it is used successfully in software development. Kanban is implemented using mostly digital boards in order to visualize the workflow, but also by using code-reviews [12]. Kanban has also recently been adopted as inspiration for several task management tools, such as Trello2and Pintask1. Especially the idea of visualiz-ing the workflow and assignvisualiz-ing tasks to group members has been used in these task managers.
Another way of managing projects is using Scrum [21]. Scrum de-notes a set of ideas introduced in 1997 by Ken Schwaber, originally designed to enhance the process of making object-oriented software [21]. Called after a rugby-technique where a team moves forward in unity [21], Scrum aims to push progress forwards in two-to-four weeks intervals, commonly called sprints [21]. In these sprints, a common goal is set and the team tries to reach the goal within the set time. The idea is to iterate quickly, because the development process is intricate and problems will not be understood completely beforehand [21]. The sprints allow the team to find new problems, address these, and continue with the project [21].
Scrum has been found useful by several big companies such as Honda, Toyota, and Canon [23], and even teams of developers dis-tributed over the world managed to successfully complete a big software project using Scrum [23].
Several case studies described differences between Kanban and Scrum. Whereas one case study described a decrease in bugs as well as a decrease in development time when replacing Scrum with Kanban [22], another case study experienced similar improvements from implementing Scrum and Kanban individually [2]. Subse-quently, combinations of Scrum and Kanban have been proposed, through which benefits of both systems, such as Kanban’s task visualization and Scrum’s sprints, are combined [18].
Concluding, research has been conducted on the causes and so-lutions of social loafing. Also, several systems regarding project
1https://pintask.me/ 2https://trello.com/ 3https://centrallo.com/ 4https://quire.io/ 5https://www.getflow.com/ 2
management as well as task management have been developed. However, research regarding the effect digital technologies have on social loafing does not exist, nor does a system with the purpose of preventing social loafing. Therefore, this paper proposes a system that aims to help prevent social loafing in university project groups by addressing several causes as mentioned in section 2.1, such as low task visibility and the feeling of low individual impact.
3
SYSTEM DESCRIPTION
3.1
Requirements
In order to help prevent social loafing, the proposed system should meet several requirements. These requirements are drawn from the solutions as discussed in section 2.2.
Firstly, the system should provide documentation. Both the dis-tributed tasks and their completion should be documented to pro-vide clarity about the task distribution as well as increase task visibility and create potential evidence as discussed in section 2.2. Secondly, the system should enable peer reviewing of all distributed tasks. As discussed in section 2.2, peer reviewing helps increase task visibility. Peer reviewing also lowers the chances of the tasks being neglected or done sub-par.
Thirdly, in order to further increase task visibility as discussed in section 2.2, consequences should be tied to leaving tasks uncom-pleted, with regard to informing the supervisor and other group members.
Fourthly, a clear overview of the project should be provided to the users, including an overview of the contributions made by the group members, as well as an overview of tasks, deadlines, and overall progress of the project. This helps increase task visibility and feeling of individual impact, both discussed in section 2.2 and section 2.1.
Lastly, some form of task management, including making new tasks, deleting tasks and taking over tasks from other users, should be included in order to increase the general usability of the system.
Figure 1: Tommy’s tasklist in a dummy-project
3.2
Interaction Overview
The proposed system is a web-application, to ensure that all stu-dents have access to the system. The system consists of several components in order to help prevent social loafing. An overview of these components can be found in figure 2. This section will outline figure 2 and roughly describe the different components of the system.
The core component is a task distribution system, which can be used to assign tasks to different group members, and set deadlines (figure 2a). Tasks are displayed in lists divided by member (figure 2b), which can be accessed through a tab-navigation. Each member can view the tasks of all members, since there is no role-hierarchy in group members. Each member represents their own behavior, and each member should be able to react to the behavior of the other members. This means that each member should have access to making new tasks, deleting tasks and other members’ task lists. Tasks can be completed by uploading evidence of the fact that the task has actually been done, such as a picture or a written piece. Tasks then are reviewed by other group members (figure 2c). When deadlines are not met, or a bad review is given, the group member who should have done their task correctly could get a warning message, urging them to contribute to the project (figure 2d). After several warnings, other project members and the lecturer are in-formed by the system.
An overview of the progress of the project can be accessed through the tab-navigation (figure 2e). It features a timeline that shows the progress divided by due dates, including the due tasks, and provides an overview of the project as a whole.
3.3
Use cases
This section outlines the use cases of the proposed system, and dives deeper into the reasoning behind the different components and interactions as mentioned in section 3.2. First, tasks will be outlined, since the tasks are a core aspect of the system. Then, warnings and lastly the overview are discussed.
3.3.1 A user wants to make a new task.
New tasks can be added by clicking on the ”New Task”-tab. A form is then shown where the user can fill in several details regarding the task, such as a name, a due date and a minimum amount of days the task should take to complete, which is used in copying the task after it has exceeded the deadline. This will be outlined in more detail later.
The task can be assigned to one person and a submit-button adds the task to the task-list of that person. As mentioned before, each group member has their own list of tasks, in which the tasks are sorted by deadline. These lists are available in the ”Tasks” tab and they are visible to all members to ensure transparency of the project process. Figure 1 shows an example of Tommy’s tasklist as seen by Tommy’s other group members. This transparency increases task visibility, by letting all members have insight into the collective contributions, and thus addresses the first and fifth requirement as discussed in section 3.1. It also lowers the feeling of lowered individual impact in the group members, because it allows each individual to clearly see their own contribution to the project, ad-dressing the fourth requirement as discussed in section 3.1.
(a) New Task form
(b) Task List
(c) Task review
(d) Warning pop-up
(e) Timeline-overview
Figure 2: Interaction Overview of the system prototype
3.3.2 A user wants to check off a task.
A group member’s own tasks can be checked off by uploading evidence of the fact that the task has actually been done. This is done by clicking a button on the task. The evidence could be a written piece, a picture, or anything else that shows other group members that the task has been completed. For a task where a chapter for a paper was to be completed, for example, the user could upload that chapter, as the written chapter itself serves as evidence that the task has been completed.
Uploading evidence serves two goals. Firstly, the uploaded evidence can be used as documentation for later reference serving the first requirement as mentioned in section 3.1.
Secondly, the evidence is used in the peer-reviewing of tasks, thus addressing the second requirement discussed in section 3.1. After the evidence is uploaded, the two group members with the least tasks in their task list will get assigned a review-task, which tells them to review the uploaded evidence. These review-tasks require no evidence to check off. Instead, a simple click is required to check off these type of tasks, after the review is done.
3.3.3 A user wants to review an uploaded task.
In order to review evidence of a task, the evidence must first be downloaded. Evidence of a task can be downloaded through a but-ton on the task (figure 1). After the evidence is downloaded, the review-button appears, through which users can review a task. Re-viewing is done by selecting either ”Good” or ”Bad” and confirming. Then, in the task list a ”Good” reviewed task turns green, whereas a ”Bad” reviewed task turns red, in order to provide clarity as to which tasks have been successfully completed and which have not. The binarity of the reviews, with regard to the ”Good”- and ”Bad”-options, is to ensure that low-effort submissions are filtered out, as opposed to providing meaningful feedback. For example, since the system itself is not able to assess the evidence, a person could submit a picture of a cat as supposed evidence for a task. These submissions should be filtered by peer-reviewing and is done by selecting ”Bad” during the review.
Peer reviewing serves one main goal in the proposed system: Since peer reviewing was shown to suppress social loafing, by using the proposed system it is expected that the amount of social loafing will decline, thus addressing the second requirement discussed in section 3.1.
3.3.4 A user does not complete a task.
A user has two ways to leave a regular task uncompleted. Firstly, a task can be uncompleted by not having submitted evidence for that task after the due date has expired. Secondly, a task can be given a ”Bad” review, which results in a task being uncompleted.
In case of a review-task, the user has one way of leaving that task uncompleted. If the user does not press the ”done” button on the review-task, the task will stay uncompleted.
An uncompleted task yields several consequences. Firstly, the task will be copied to a future date. The date is determined by the mini-mum amount of days needed for the task, which was filled in when adding the task. This copied task is called a ”retry”-task, and has a higher urgency in the system, which has some consequences that will be elaborated on later.
Secondly, since there is a new ”retry”-task, the old task will be
turned inactive, entailing that no evidence can be uploaded any-more to this task. However, because it can serve as documentation, the old task will not be deleted.
Thirdly, the user that left a task uncompleted will receive a warning. The warning will pop up on the screen the next time the user logs into the system, and will also be emailed to the user by the system (figure 2d).
The warning serves several purposes. First of all, the warning shows that there are consequences of not completing tasks. By clarifying this to the user, the warning aims to motivate the user to contribute to the project more consistently.
Secondly, the warning aims to increase task visibility by mentioning that the other group members, and eventually even the supervisor will be informed about the non-contribution. This addresses the third requirement as discussed in section 3.1.
In case a task, and the retry-task that follows are both not completed, a final retry-task is made, with a very high urgency. However, since this task has to be completed eventually, and this group member apparently is not able to reliably finish the task, a button appears on the new task through which the task can be taken over by another group member. After a group member has clicked the button and has confirmed, the task then is copied to their own task list. All group members will be informed of this action through email. After a user leaves more than two tasks incomplete, a second warn-ing is issued, as well as a notification to all other group members about this behavior. This serves as documentation of the incident, as well as an update to the rest of the group, causing common knowledge on the state of the project. The warning mentions that all group members have been informed, and that the supervisor will be informed after the next incompletion of a task, serving as a means to address the third requirement as mentioned in section 3.1.
After yet two more tasks have been left incomplete, the supervisor is informed through email by the system, and the group members receive a message updating them on the situation. The goal is to inform the supervisor objectively about the behavior. The conse-quences that the supervisor might implement are out of reach of the system.
The warnings are sent after one, three, and five tasks are left un-completed in order to prevent multiple warnings from being sent at once. If a user leaves two tasks uncompleted on the same day, only one warning should be issued, instead of two at once. Further-more, issuing warnings after every uncompleted task could result in perception of unfairly strictness in the system.
3.3.5 A user wants to delete a task.
Tasks can be deleted by clicking the red bin displayed on each task. However, in order to prevent users from deleting tasks for the sake of avoiding doing them without any consequences, a pop-up ap-pears when the bin is clicked, telling the user that if she continues, all group members will be notified of the action. Once confirmed, the task is deleted, and all group members are notified through email. This feature serves the fifth requirement discussed in section 3.1.
3.3.6 A user wants to hide all past tasks.
By default, all tasks, even past the due date, remain visible in the
Figure 3: Timeline of a project
task list as a means to providing an overview of the project and documentation of done and uncompleted task. A user can hide all tasks due in the past by clicking a button on the top side of the task list. The tasks reappear if the button is clicked again. This feature serves the fifth requirement discussed in section 3.1.
3.3.7 A user wants to see an overview of the project.
On the timeline-page, the timeline can be found. The timeline aims to provide an overview of the project as a whole, by showing the due dates of all tasks, represented as circles on a line (figure 3). The right-most circle represents the final deadline, which is visualized by a black edge on the circle.
When a circle is clicked, an overview is shown of the group mem-bers and the relative amount of tasks that are done (figure 4). A progress bar including the percentage of tasks done is shown for each individual group member. A task is considered ”done” when evidence is uploaded, and the evidence has been given a ”Good” review.
Furthermore, in order to provide clarity on the amount of tasks completed, each circle on the timeline is accompanied by a smaller red circle (as shown in figure 3). This circle indicates the total percentage of tasks that are due for that date and that are done.
Figure 4: The overview provided when clicked on a date
Additionally, the circles have a red indicator which shows one or more exclamation points (as shown in figure 3), when an urgent task, with regard to the ”retry”-tasks, is due on that date. The more urgent a task is, the more exclamation points will be shown on the circle.
The timeline serves the purpose of providing a clear overview of the scope of the project, as well as providing an overview of the progress of the project, and each member’s contribution to the progress.
In the bottom half of the screen the amount of warnings per group member can be seen, as a means to provide an overview on all of the group members and their overall contribution to the project (figure 2).
Both the timeline and the warning-overview also provide documen-tation on the project and the course of events. Both the overview of the warnings and the timeline implement the fourth requirement discussed in section 3.1. The reason that they are positioned on the same screen is that both the timeline and the warnings-overview provide documentation of the project as well as an overview of the project as a whole.
3.4
System Architecture
This section provides a concise description of the components used to build the system, including the software architecture.
Hardware
In order to keep the threshold to use the system as low as possible, the system is a web-application accessible through every major web-browser on any Windows6, Mac OS7, or Linux based system such as Ubuntu8. An active internet connection is needed in order to run the application. No additional software-download is needed to start and use the system. The system was implemented in Ubuntu 16.049and tested on Ubuntu 16.049and Windows 106.
Mobile browsers are not supported. The reasoning behind this is straightforward. The time frame for this study was too short to focus on developing a prototype for mobile systems. Given that the
6https://www.microsoft.com/en-us/windows 7https://www.apple.com/lae/macos/high-sierra/ 8https://www.ubuntu.com/
9http://releases.ubuntu.com/16.04/ 6
Figure 5: System architecture of the system prototype
system is focused around giving users a clear overview of a project, the system would have to be redesigned in order for it to convey its overview well enough.
Software Architecture
The architecture of the system can be separated in three parts: The application, the database, and the server (figure 5). These three parts are tied together through a data model.
3.4.1 The data-model.
The prototype of the system that was made for this study uses a specific model of group members, tasks, review-tasks, warnings and other concepts. This section outlines a general data model that includes all concepts needed to build a prototype such as the one used in this study.
Firstly, the concept of tasks should be present in the data model. Each task should have a deadline, and it should be marked whether a task is done and whether the task is reviewed. Also, it should be possible to store the contents of the review. Furthermore, in order to keep track of retry-tasks, a version number should be tracked
on tasks.
Since distributing tasks is a core concept of the system, also group members should be represented in the data model, including a link with tasks, so tasks can be assigned to group members. Further-more, warnings should be included in the data model. The data model should be able to assign warnings to group members. Also, the data model should keep track of the amount of warnings given to a group member.
3.4.2 The database.
In order for the different tasks, warnings, users and other concepts to be saved, a database, for example MongoDB10is to be set up and connected to the application. The application should continuously push new information, such as tasks that are completed, new tasks that are added, but also evidence-files to the database.
When users register, their details are saved in the database. Each time a user logs in after registering, the user details are verified, and the relevant information is retrieved from the database, and
10https://www.mongodb.com/ 7
shown in the application.
3.4.3 The server.
Both the application and the database are hosted on a server. This can be done on a cloud server which can be rented at a number of different companies, for example Amazon Web Services11. The server should be stable enough for both the web application and the database to run smoothly with minimum waiting times for retriev-ing or storretriev-ing data from and to the database. Therefore, specifically, the server should have a minimum of 2Gb internal memory and an SSD with at least 40Gb of memory. No specific processor or graphics card is required.
Furthermore, the server should be able to send emails about warn-ings and updates regarding the project, which is also an available service at Amazon12.
3.4.4 The application.
The application itself is written in VueJs13and NuxtJS14. VueJS is a JavaScript framework that is used mainly for building interfaces and web applications. It provides double binded variables and fairly simple state management. NuxtJS provides routing and other con-figurable options for VueJS applications.
In this study VueJS was used as the main JavaScript framework based on personal preference. Other frameworks, such as React15 or Aurelia16could have been used to achieve the same results re-garding building the user interface.
4
EVALUATION
In order to research whether digital technologies can help prevent social loafing in university project groups the prototype as described in section 3 was evaluated. Because no such system yet exists, no A/B testing can be done, and since this system is the first in its kind, it has to be evaluated whether the system would be accepted by university students in the first place. Therefore, the evaluation of the system was based on the first version of the Technology Acceptance Model [7], which is a model of the acceptance of new technology by people. The theory mentions two important factors: the perceived usefulness of a system, which entails a perception that the system could potentially increase one’s job performance, and the perceived ease-of-use of a system, which entails the believe that a system could be used easily and without much effort. If both factors of a system are deemed high by its users, the system is likely to be accepted and used by people [7].
The Technology Acceptance Model (TAM) was proposed by Davis in 1986 [7]. Since then, several revisions have been made with regards to the model, and a second version of the model [24], as well as the Unified Theory of Acceptance and Use of Technology (UTAUT) [25] have since been proposed as enhancements on the first model.
TAM 2 incorporates several more variables in the model than the
11https://aws.amazon.com/ 12https://aws.amazon.com/ses/ 13https://vuejs.org/ 14https://nuxtjs.org/ 15https://reactjs.org/ 16https://aurelia.io/
original TAM, such as experience and job relevancy [24], whereas the UTAUT uses many more variables in order to predict intention and behavior [25]. Subsequently, in this study the original TAM was chosen as a model to base the evaluation on, since time constraints would not allow for a more sophisticated model to be incorporated. This section describes the evaluation and the findings of the evalu-ation.
4.1
Experiment design
The evaluation of the prototype using the Technology Acceptance Model [7] is done through conduction of user tests. These tests consist of a usability-test, which tests the perceived ease-of-use, and an interview, which tests perceived usefulness.
The amount of participants needed for the evaluation of the per-ceived usefulness is based on theoretical saturation [6]. The goal is to reach saturation, after which there would be no use to test more participants, since they would yield no more information. The amount of participants needed for the evaluation of perceived ease-of-use, also known as usability, is based on usability testing research done by Nielsen [19]. He claims that between three and five experts could identify most usability issues in a system [19]. The evaluation is to be conducted individually and in a silent space in order to prevent distraction. Participants answer several ques-tions, asked in a short interview-setting, regarding the demograph-ics first. Then, they receive a written scenario, made to immerse them in a fictional character. The immersion was important to create, since the test-setup was not as realistic as possible. Ideally, this system would have been tested with actual project groups, during a real project of four weeks or longer. However, due to time constraints this was not achievable, so two scenarios to im-merse the participant in a fictional character, and let the participant identify with the fictional character, were made: one concerning a free-rider character, and one concerning a hard working character. Participants receive the scenario which fits them most, based on a question whether they had ever free-ridden in a project before themselves, asked during the demographic-questions.
The scenario contains actions for the participant to perform, using the prototype, which is provided on a laptop.
After the scenario is completed, the participants are asked to per-form several basic tasks using the prototype, such as deleting a task, adding a task, and conducting a review. These tasks are used to test the perceived ease-of-use and are the same for all participants. The tasks collectively cover the whole system and urge the participant to create an opinion on the usability of all parts of the system. Lastly, an interview is conducted with the participant concerning their thoughts of the system. Questions are the same for all partici-pants, but follow-up questions might differ. This interview is used to test perceived usefulness. The questions collectively cover all of the functionality of the system and urge the participant to create an opinion of the usefulness of the functionality.
The participants should meet several requirements. Firstly, in order to evaluate the system for university project groups, the partici-pants should be currently enrolled in a university. Furthermore, in order to have a wide enough sample, both male and female stu-dents should partake, as well as stustu-dents from different universities enrolled in different study programs.
Due to time constraints, mostly convenience sampling and snowball sampling [6] are used when gathering participants.
4.2
Experiment and demographics
Eight participants were gathered in total. All participants are 24 years old and currently enrolled in a master’s program. Four par-ticipants are registered at the University of Amsterdam, and four participants are registered at the University of Utrecht. Three par-ticipants are male, and five are female. All parpar-ticipants have en-countered group projects where social loafing took place. Four par-ticipants resolutely said that they had never free-ridden, whereas three participants thought they had done it, but not purposefully. One participant admitted to have free-ridden before.
The experiment took place in a silent room at the University of Amsterdam for four participants, whereas four participants were placed in a silent room at the University of Utrecht in order to conduct the experiment. After testing eight participants, both the-oretical saturation, and the minimum of three to five participants were reached. Therefore, the experiment was terminated.
4.3
Findings
This section discusses the themes that emerged from the test ses-sions. The themes were extracted from the coded interviews and test related remarks made by participants during the test session. The process of coding was done after each test session so the point of theoretical saturation could be determined.
4.3.1 Transparency of tasks.
This theme refers to the transparency that the task-lists bring forth, as discussed in section 3.3.1.
All eight participants were favorable towards the transparency that is created by the task-lists. Three participants liked the documenta-tion it provides. Participant 7 said ”It is nice that you can prove who’s done what, based on evidence”. This remark could be linked to the idea of documentation, discussed in section 2.2. Three participants mentioned that the overview it provides gives them ease of mind.
4.3.2 Reviews.
This theme refers to the peer-review feature, as discussed in section 3.3.3.
All eight participants found reviewing useful as a concept. How-ever, all eight participants had the same remark concerning the extensiveness of the reviewing. They would have wanted more extensive reviewing, instead of binary reviewing as it currently exists. Participant 6 said ”Maybe being able to place comments would be nice. It could also start discussions”.
Two participants mentioned Google Docs’ commenting-feature, which helps them create a clear overview of the feedback they give or get. Furthermore, four participants had trouble finding the but-ton in order to review a task, which was located on the task in the task list.
Concluding, Participants were positive with regard to the review functionality. However, the concept was perceived as too shallow and more in-depth reviewing was expected, as well as a more clear design of the review-button on tasks in the task-list.
4.3.3 Timeline.
This theme refers to the timeline, as discussed in section 3.3.7.
All participants were positive towards the timeline. Participant three said that the timeline ”causes insight about when a deadline is” and that the timeline ”makes it easier to plan”. However, usability-issues came to light during the tests. One issue was the percentage-indicator, which currently shows the amount of tasks that are done (both uploaded and reviewed). However, seven out of eight par-ticipants expected to see the amount of tasks that were uploaded, instead of amount of tasks that were both uploaded and reviewed. Moreover, the exclamation marks denoting the priority retry-tasks were not clear to five participants. This might be linked to the fact that retry-tasks as a concept were generally not understood immediately when asked to upload evidence into the retry-task. Two participants mentioned that they would have expected a dash-board containing statistics on how the project is going, as an addi-tion to the current timeline.
Generally, the timeline was perceived as useful. However, some us-ability issues in the design of the timeline as well as the retry-tasks were pointed out.
4.3.4 Warnings.
This theme refers to the warnings issued by the system after a group member has left one or more tasks uncompleted, as discussed in section 3.3.4.
Six participants were positive regarding the warnings that popped up on the system. Participant 3 said ”[…] for most people it will be a trigger, because people will see that you haven’t done anything”. However, there was critique too. Participant 2 noted that the mes-sage was a little harsh and two participants thought the warning could use more positive and motivating content.
Concluding, two thirds of the participants were positive towards the warnings, saying that the evidence and consequences the warnings brought to the project are desirable.
4.3.5 Usefulness.
This theme refers to the perception of usefulness of the system as discussed in section 4.
Six out of eight participants thought the system as a whole could help in preventing social loafing. Two participants found that the system could potentially help, but only if the group commits to using the application throughout the project, which, according to them, could be difficult to manage. Three participants mentioned that such problems could be solved if university would make the system a mandatory part of group projects.
Five participants would use the system themselves, if they had the chance.
4.3.6 Usability.
This theme refers to the perception of ease-of-use of the system as discussed in section 4.
All participants found the system easy to use as a whole and most of the features were found easily, such as the task-list, hiding tasks that were due in the past, and uploading evidence to a task, which were all usability-tasks that were completed easily by all partici-pants.
However, some parts of the system were more difficult, according to the participants. The aforementioned exclamation marks in the timeline were unclear to five participants, as well as the percent-ages, which were not correctly understood by three participants.
Furthermore, retry-tasks were not easily understood by six partici-pants. Four participants only understood the concept of retry-tasks after it was explained to them in person. Moreover, reviewing tasks appeared to be a little difficult for six participants, since the review-button was not easily found.
Concluding, the usability of the system was positively received by the participants. However, several issues came to light that could be improved.
5
DISCUSSION
The present study aimed to find whether digital systems could help prevent social loafing in universities. In order to research this, a prototype was built, and evaluated based on the Technology Accep-tance Model.
Findings show that participants were positive regarding the system. They thought the system could help prevent social loafing in group projects, albeit in some cases by making it a mandatory part of group projects. Specifically, participants liked the overview and documentation the system provides. Furthermore, peer evaluation of other group-members’ work was perceived as useful. Also, the perceived ease-of-use of the system was regarded positively by the test participants.
The proposed system has several limitations as well, as shown by the results. Firstly, the percentages shown on the timeline denote the percentage of tasks that are both uploaded and reviewed with a ”Good” grade. Results have shown that this caused confusion with participants, since participants expected to see uploaded tasks, no matter the state of the review. Secondly, the findings suggest that while the concept of peer reviewing was regarded positively, a more in depth reviewing system would be appreciated, in order to give more detailed feedback.
Thirdly, the findings suggest that more positivity in the system would be preferred by two participants. The warnings presented in the system were perceived by participant 1 and 2 as focused too much on negative consequences and these participants said that positivity in some way would be more effective for them. Fourthly, two participants commented on the potential of larger groups using the system and the expandability of the system. The scenarios used in the evaluation were based on a group size of 4, since this is the most used group size in university group projects. Also, larger groups have been shown to include more social loafing, as discussed in section 2.1. Therefore, the layout of the system works optimal with this number of group members. However, if larger groups would be using the system, the layout of the system, particularly the timeline, is unoptimized and could even become unfit for use.
Lastly, several usability issues, such as retry-tasks and positioning of the review-button, have been identified and could be improved upon.
5.1
Limitations
The present study is also limited in its evaluation, which will be elaborated on in this section.
The evaluation of the proposed system has several limitations. Firstly, due to time limitations, evaluation was done by interviews
and usability testing during a 30 minute session, instead of letting students use the system during a real project. Although the current evaluation has resulted in a rough understanding of the acceptance of the system, more reliable findings could be found through evalu-ation with students working on a real life project.
Secondly, the sample used to evaluate the proposed system was not wide enough to generalize conclusions. Because the sample consisted of master students only, the conclusion cannot be gener-alized to bachelor students, nor to group projects in non-university settings.
Thirdly, it is not clear whether the two scenario’s that were pre-sented to the participants in order to achieve immersion did actu-ally immerse the participant in a character. Participants using each scenario did not differ clearly in response during the interviews. This may indicate that the that the scenarios did not cause enough immersion, since some difference in response is expected from free-riders and from hard workers in group projects during the interviews. However, no way of testing the different scenarios is used, so no definitive remarks can be made regarding the scenarios. Lastly, since mostly convenience sampling was used, the partici-pants knew the conductor of the tests beforehand. Although before each interview the conductor has informed the participants that they should speak their minds freely, the knowledge of who the conductor of the tests was has likely had an influence in some way on the responses of the participants.
6
CONCLUSION AND FUTURE WORK
This study aimed to find whether digital systems can help prevent social loafing in university project groups.
Findings show that both the perceived ease-of-use and the per-ceived usefulness of the proposed system are positive.
Given the findings, it can be concluded that digital systems could potentially help in preventing social loafing in project groups with master students. However, as discussed in section 5.1, this con-clusion is made within the context of the evaluation. Given the limitations, a more generalized conclusion cannot be drawn from current study. Further research should be done before generalizing these conclusions.
Future research should focus on two aspects. The first aspect is regarding improving the evaluation of the system, whereas the second aspects focuses on improving the proposed system.
6.1
Evaluation Improvements
As discussed in section 5.1, the evaluation of the present study could be improved in several ways. Future work should look into ways to evaluate the proposed system in real life group project settings in order to strengthen the conclusion of the present study. Future work should also use use a wider sample than the present study, in order to provide more generalized findings than the present study.
6.2
System Improvements
6.2.1 Mobile Design.
Due to time constraints the system was not designed for mobile systems. However, since users might want to check into the system to regain the overview of a project while in public transport or otherwise away from home, a mobile design would be effective.
6.2.2 Dashboard.
A dashboard containing information on the group members could be implemented by future work. The dashboard could increase task visibility, positivity in the communication from the system to the users, and decrease the feeling of lowered individual impact. Fur-thermore, a dashboard containing data on the project and the the group members could increase the clarity of the project overview.
6.2.3 Automatic quality assessment.
By implementing an automatic quality assessor in the system, the tasks are automatically checked and reviewed by the system. This would potentially make use of text mining and text analysis meth-ods.
While this feature would take away work from the students, it would also take away peer reviewing as a means of countering social loafing. No research has yet been done on the effects of automatic quality assessment on social loafing.
REFERENCES
[1] Ahmad, M. O., Markkula, J., and Oivo, M. (2013). Kanban in software develop-ment: A systematic literature review. 2013 39th Euromicro Conference on Software Engineering and Advanced Applications, (September):9–16.
[2] Anderson, D., Concas, G., Lunesu, M., Marchesi, M., and Zhang, H. (2012). A Comparative Study of Scrum and Kanban Approaches on a Real Case Study Using Simulation. Agile Processes in Software Engineering and Extreme Programming SE - 9, 111:123–137.
[3] Brien, C. L. O. (2008). Social Loafing on Group ProjectsStructural Antecedents and Effect on Student Satisfaction. (October).
[4] Brooks, C. M. and Ammons, J. L. (2003). Free Riding in Group Projects and the Effects of Timing, Frequency, and Specificity of Criteria in Peer Assessments. Journal of Education for Business, 78(5):268–272.
[5] Brutus, S. and Donia, M. B. L. (2010). Improving the effectiveness of centralized peer evaluation. Academy of Management Learning and Education, 9(4):652–662. [6] Bryman, A. (2016). Social research methods. Oxford university press.
[7] Davis, F. D. (1986). A technology acceptance model for empirically testing new end-user information systems: Theory and results. Management, Ph.D.(April):291. [8] Earley, P. C. (1989). Social Loafing and Collectivism : A Comparison of the United
States and the People ’ s Republic of China Author ( s ): P . Christopher Earley Source : Administrative Science Quarterly , Vol . 34 , No . 4 ( Dec ., 1989 ), pp . 565-581 Published by : Sage P. 34(4):565–581.
[9] George, J. M. (1992). Extrinsic and Intrinsic Origins of Perceived Social Loafing in Organizations. The Academy of Management Journal, 35(1):191–202.
[10] Hall, D., Buzwell, S., Hall, D., and Buzwell, S. (2013). The problem of free-riding in group projects : Looking beyond social loafing as reason for The problem of free-riding in group projects : Looking beyond social loafing as reason for non-contribution. (March).
[11] Hansen, R. S. (2006). Benefits and Problems With Student Teams: Suggestions for Improving Team Projects. Journal of Education for Business, 82(1):11–19. [12] Ikonen, M., Pirinen, E., Fagerholm, F., Kettunen, P., and Abrahamsson, P. (2011).
On the impact of Kanban on software project work: An empirical case study in-vestigation. Proceedings - 2011 16th IEEE International Conference on Engineering of Complex Computer Systems, ICECCS 2011, (May):305–314.
[13] Karau, S. J. and Williams, K. D. (1997). The effects of group cohesiveness on social loafing and social compensation. Group Dynamics: Theory, Research, and Practice, 1(2):156.
[14] Kirschner, F., Paas, F., and Kirschner, P. A. (2009). A cognitive load approach to collaborative learning: United brains for complex tasks. Educational Psychology Review, 21(1):31–42.
[15] Laal, M. and Ghodsi, S. M. (2012). Benefits of collaborative learning. Procedia -Social and Behavioral Sciences, 31(2011):486–490.
[16] Latan´e, B., Williams, K., and Harkins, S. (1979). Many hands make light the work: The causes and consequences of social loafing. Small Groups: Key Readings, 37(6):822–832.
[17] Liden, R. C., Wayne, S. J., Jaworski, R. A., and Bennett, N. (2004). Social loafing: A field investigation. Journal of Management, 30(2):285–304.
[18] Mahnic, V. (2014). Improving software development through combination of scrum and Kanban. Recent Advances in Computer Engineering, Communications and Information Technology, Espanha, pages 281–288.
[19] Nielsen, J. and Molich, R. (1990). Heuristic Evaluation of user interfaces. CHI ’90 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (April):249–256.
[20] Poddar, A. (2010). Continuous Additive Peer Review: A New System to Control Social Loafing in Group Projects. Advancement of Marketing Education, 17:1–12. [21] Schwaber, K. (1997). Scrum development process. In Business object design and
implementation, pages 117–134. Springer.
[22] Sjøberg, D. I., Johnsen, A., and Solberg, J. (2012). Quantifying the effect of using Kanban vs . scrum : A case study. IEEE Software, 29(5):47–53.
[23] Sutherland, J., Viktorov, A., Blount, J., and Puntikov, N. (2007). Distributed scrum: Agile project management with outsourced development teams. In System Sciences, 2007. HICSS 2007. 40th Annual Hawaii International Conference on, pages 274a–274a. IEEE.
[24] Venkatesh, V. and Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management science, 46(2):186–204. [25] Venkatesh, V., Morris, M. G., Davis, G. B., and Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS quarterly, pages 425–478.
