Exploring the Requirements of Knowledge Sharing Among
Academic Staff Members: The UvA Case
Selai Anwary
Information Studies. HCM, University of Amsterdam. Student ID: 10000129
Amsterdam, The Netherlands
selai.anwary@student.uva.nl
T
HESISM
ASTERI
NFORMATIONS
TUDIES -H
UMANC
ENTEREDM
ULTIMEDIAF
ACULTYO
FS
CIENCE 16/02/2016Exploring the Requirements of Knowledge Sharing Among
Academic Staff Members: The UvA Case
Selai Anwary - 10000129
ABSTRACT
This paper presents a design-based approach to explore the requirements of knowledge sharing among the academic staff members at the University of Amsterdam. It describes the evaluation results of two iterations of this design-based methodology that assess usability for two prototypes of an app called Knowlety. A description is given for the lessons learned from these iterations. Additionally, these evaluations illustrate that some of the app’s functions are perceived as positive by staff members. However, further study is needed to assess the app more diagnostically.
General Terms
Documentation, Design, Experimentation.
Keywords
Knowledge Sharing, Time, Time-Management, Technical, Social Network, Location-Services, Location-Awareness, Interaction Design, Mobile, Collaboration, Incentives.
1. INTRODUCTION
“My favorite things in life don’t cost any money. It’s clear that the most precious resource we all have is time”, a proverb originally coined by Steve Jobs, exposes time as being a resource that is measured in value to organizations as well as individuals. It highlights the fact that the amount of time we have is naturally limited. We do all sorts of things that result in time becoming an economic trade-off, e.g. we pay housekeepers to clean our houses or we hire planners to organize our weddings. As a consequence, we can spend time on things we believe are more meaningful. Learning needs time and effort, but learning for an organization is still seen as a very important concept. The University of Amsterdam (UvA), like any other organization must keep learning because of its changing environment. Therefore, the employees, i.e. academic staff members need to keep learning from each other to handle these transformations that influence education. This means, that they also need to derive information from their changing environments, they must share this knowledge amongst the other organization members and construct a collective meaning of these changes. But, a survey by Michael Vliek1 has shown that time still forms a barrier.
The UvA has seven faculties in which education and research is taking place. At the head of each faculty a dean functions as a leading character. The education and research are housed at the education and research institutes. Each faculty has its own location in Amsterdam, where staff members have their bases in departments. In addition, each faculty has its own central service that ensures the management of the faculty. As a result, it has been found difficult amongst the UvA’s academic staff members to share knowledge about education with their colleagues outside of their departments or faculties. Although respondents indicate that they would like to share their knowledge and develop an interaction with
1This survey has not been published yet. Vliek is from the Psychology Research institute at the UvA. Tel. 020-5256892, email: vliek@uva.nl
peers, one major obstacle to this goal has been identified, namely the lack of time. Ideally, they would like to share knowledge in direct contact with colleagues e.g. by discussing in the same department or in different ones. In addition, prior use of Yammer and the survey have shown that fewer respondents would like to learn and share their knowledge and experiences in an online environment. But, paradoxically due to the distance between the departments and staff member, regular face-to-face interactions are difficult to maintain at the UvA.
Over the past few decades a myriad of technologies and methods have been developed to aid organizations in better managing of knowledge and knowledge sharing. These technologies and methods often focus on general aspects of knowledge management (e.g. forums, wiki-pages, time management, project planning tools, social aspects etc.). They seem to implement all of the things that might be of use for organizations as a whole. Therefore, these applications are often not aligned with the individual and organizational ethos and goals. So then what are the requirements for knowledge sharing among the UvA academic staff members?
System requirements are most likely developed in regular series parallel with the design. Therefore, to answer the previous question in this research the focus is on novel approaches to knowledge sharing via micro-blogging and task-management. Time management focusses on scheduling of time, which sometimes leads to overwhelm. While a task refers to a singular item, time refers to something much larger. Meaning, that managing a task is far more manageable than managing time. Therefore, users will end up taking tasks one at a time and become more successful by doing so. Of course, since time is an issue when it comes to knowledge sharing one should also understand how these tasks fit within time.
For knowledge sharing to take place there is also a need for connecting individuals with the right people and information. So this study will also explore the ways in which these types of connections can take place. At the heart of this is the conversation between the UvA academic staff members, or lack thereof. In addition, the research will examine the different ways in which agents can be motivated to participate and preform action i.e. share knowledge in an online environment. The contributions of this paper are as follows: (1) a description of the functional requirements i.e. actions and behavior; (2) nonfunctional requirements i.e. performance, design and user experience; and (3) constraints. The research is based on the previous study by Vliek at the UvA and prototype evaluations with academic staff members in various departments within the UvA. During the evaluation the users are presented with two prototype applications. Here the prototypes are tested on their usability and user satisfaction.
In the next section the related work is discussed. In section 3 the research methodology is explained, which is followed by the interaction design in the fourth section. Section 5 describes the
choices that are made with regards to what the system will do internally and how it will do this.
2. RELATED WORK
2.1 Knowledge Sharing Systems
Universities are environments that are knowledge intensive, which play a central role in the creation of knowledge through different studies and in knowledge distribution through research publications. In addition, they play a critical role in transferring knowledge through working together with other organizations to support innovation, as well as assisting learning through their education and guidance programs. Many organizations already have knowledge management (KM) technologies in use, which are in fact information technologies that can be used to facilitate knowledge management. Content management systems are relevant to knowledge management, because they are responsible for the creation, management and the distribution of content on the intranet, extranet or a web-based platform. These content management systems come in many different forms, which means that an organization needs to cautiously evaluate what its needs are. The Faculty of Science at the UvA has made a first attempt in using a knowledge sharing service. This knowledge – sharing platform is called Starfish. The service offers a question based knowledge network that focuses on education practices: didactics, education technologies and people with a fair amount of experience. In addition to its open source content the platform offers access to specific content for staff members that can login with their UvAnetID. In their research article for Starfish, Brouwer et al. describe the network as follows [1]:
Most of the listed lecturers are also researchers and many of them maintain a personal homepage or have a homepage on their institution’s server, LinkedIn, or other social network. It makes no sense to duplicate these sites, and certainly not to increase required maintenance time, thus a profile listed in the database can be directly linked to such personal homepages […] Members who don’t have a personal homepage can, instead, upload a cv into their profile.
There are many types of knowledge, but an important distinction can be made in implicit knowledge (the so called tacit knowledge) and explicit knowledge. According to Panahi et al. implicit knowledge is the most important part of human knowledge [2]. For this knowledge to be shared within the university it is important that this knowledge is made explicit. Relationships and their strengths are very important in making this possible. That is, weak ties provide greater opportunities to congregate new information. According to Hayhornthwait et al. people hang on to these weaker ties with knowledgeable people that they can trust, whereas strong ties guarantee deepening of information [3]. It is remarkable that Starfish doesn’t offer a more integrated social network for staff members, since that could provide the possibility to work on relationships online alongside face – to – face contact. This would also mean that the contact could continue online and present more availability for staff members.
2.2 Social Knowledge Sharing: Micro-blogs
Within the corporate software landscape micro-blogs have made their arrival [4]. Others have conducted a thorough review of past studies on micro-blogging features and concluded that these services differ from other computer mediated communication technologies in that they allow: brevity, mobility, broadcasting, autonomy and interoperability [5]. Although - as a similar tool - Twitter has been gaining popularity within the public space it has not yet implemented collaborative instruments for companies.
Yammer, which was already used by the UvA enables comparable micro-blogging functionalities that Twitter has for company employees by way of closed groups. Thus, in addition to the more general micro-blogging tools (following and posting messages), Yammer also provides the user with conversations, attachment of files and groups [6]. Compared to the micro-blogging platform Twitter, these qualities make the Yammer platform more useful to organization.
Further research has shown that informal communication aids by boosting trust and teamwork [7]. However, in these studies it is not yet defined if these benefits are the consequence of micro-blogging. They add that from an emotional perspective, people seem to use micro-blogging to achieve another level of cyberspace presence, namely the idea of being “out there” and to feel a different layer of connectivity with the world.
Additional study by Azab attempts to evaluate the role that these micro-blogs play in the increasing of teamwork and trust [8]. Although the study has been done among the staff in a corporate environment situated in a developing country, it does show that micro-blogging leads to significant improvement of communication. It also unveils that enterprise micro-blogging differs from public platforms because it focuses on shared work tasks and news, whereas public platforms are focused more on the user. Thus, it is less about individual behavior and activity. Zhang et al. [9] discovered that Yammer failed to establish significant increases in social awareness and ties. This is mostly due to its predominant usage as communication medium for outside news objects and information. Therefore, implementation of micro-blogging within enterprises should also consider and try to encourage greater and easier sharing of personal experiences and information. The research also highlights another key concern with Yammer, namely the fact that it is hosted outside of the corporate network. It suggests that regardless of privacy and security implementations, users tend to be more cautious about sharing information.
2.3 Main Findings from Literature
In summary, there is limited research on micro-blogging in enterprise context. However, there are several comparable studies, which highlight the potential nature and value of micro-blogging e.g. how it can contribute to more effective ways of collaboration in groups. The key issue seems to lie in the adoption and constant contribution of these technologies. According to the related work, sharing of personal information and experiences would increase team cohesion and collaboration. But, these researches forget the fact that in some organization, such as the UvA there is still a lack of time and real-life-conversation. The study by Zhang et al. seems to reveal that employees are not necessarily trusting of the cloud-based platforms or how the information is managed outside of the organization. So it is important to look into how these platforms can be hosted within the organization and how time-management can be a part of the platform. Understanding the specific user-group and their needs is also valuable, because it can assist in modeling of the interaction design and the user adoption.
Accordingly, the following sections will deal with the research methodology. There are two main parts to account for: (1) the interaction design and (2) the system description. These sections are used to address the research question that was expressed in the introduction.
3. RESEARCH METHODOLOGY
In this thesis a design-based research methodology has been applied to handle the research. The research relies mostly on digital
prototyping to evaluate ideas. Using previous research by ICTO, this approach enabled the collecting of quantitative data before the design process took place.
In the first iteration a low-fidelity prototype was developed based on the findings from Vliek’s survey. Here, the survey is used to generate a user-centered design process. Based on these findings, the envisioned human interaction is detailed out in the ensuing interaction design section. This prototype was tested with five teachers. In the final iteration, the evaluation of the lo-fi prototype functions as a source for the high-fidelity prototype Knowlety that is evaluated in interviews and user observations of 15-30 minutes using the application. The evaluation was conducted with twenty teachers, respectively. This enabled the gathering of qualitative data that are useful to collect additional requirements and to identify potential usability challenges with the interaction performances. Additionally, for a better understanding of what the prototype will do internally the system description section will state clearly the hardware requirements and what the input and output will be, built on the interaction design. These technical requirements are supported by literature.
4. INTERACTION DESIGN
The intended interaction between the human (user) and Knowlety is explained in this section. First, the different use cases should give a clear description and explanation of user objectives (Appendix A). These objectives are based on Vliek’s survey and were used for the design of the low-fidelity prototype. Secondly, the lo-fi prototype is used to get a better understanding of user expectations and needs.
4.1 The Envisioned Users & Principles
The Knowlety application will run on mobile phones so that the users have it within their reach. It is intended for every teacher at the University of Amsterdam regardless of any other demographic characteristic. Of course, keeping in mind that the main focus of
this research should be on the academic teachers as the main user group. The lo-fi prototype is the first step in the user interface design. It is an important step, because it provides this research with quick feedback and low costs in the design process.
4.2 Low-fidelity Prototype
To meet the user’s expectations and needs in a broad sense a lo-fi prototype is created following the previous principles and is evaluated in the following section. This prototype involves the creation of balsamiq-sketched drawings of interfaces that are used for testing. The design of the lo-fi prototype focusses on several key navigating scenarios: (1) navigating through the menu; (2) opening the map and viewing a social card; (3) going through the statistics; (4) viewing the chat option; (5) posting an image and (6) viewing what others have posted.
4.2.1 Design & Implementation
The first navigating scenario is addressed by using Tumblr as an example. On its application, users can follow other blogs as well as make their own blogs. Much of Tumblr’s features are accessible from the user "dashboard" interface, which also gives the option of posting content and displays posts of blogs that are followed by the user. The map and social card in Knowlety are used for finding other users and is based on the idea that the UvA staff members are more willing to interact face-to-face (fig. 1C and D). Therefore, location-awareness was incorporated in the prototype. Thus, the user is able to use their phone’s location to show potential matches with other users (fig. 1J). These matches are based on tags that the users have in common. The design for the chat option is based on already existing chat applications such as WhatsApp and Telegram. New frameworks or patterns might look nice, but familiarity is going to add more to the insurance of making the user feel at home (fig. 1L).
In the lo-fi prototype a first attempt is made to point out the utility of micro-blogging. Here, the system is positioned as an Figure 1. Lo-fi prototype that was used for evaluation.
A
A B B C C D D E E F F G G
H
efficient means of personal brand building, knowledge management and communication. In the login interface the user is given three options (fig. 1A): (1) signing in with their email and password; (2) signing in with LinkedIn and (3) signing up for the use of the application. Once the user has signed in with LinkedIn, their profile summary, name, photo, title and current positions will be embedded in their Knowlety profile (Fig. 1B).
The personal brand building is mainly done through the user profile, where the user gets a certain title. There are 3 three titles: lord (minimum of 200 posts), master (400 posts) and the untouchable (600 posts and above). These titles are represented with stars on the social card and the crossed paths section (fig. 1D and J). Which, in return should also function as an incentive for the user to participate in knowledge sharing. The social card is something that each registered user has that holds specific information about them. For instance, if user A and B crossed paths both of them will be listed in each other’s crossed paths section. This is where they can access the other’s profile (fig. 1K) by tapping on their image. The social card takes a more literal meaning in the map section (fig. 1D) where the social card can be accessed by the user’s map-marker (fig. 1C). Here, the user is also able to access the other’s profile by tapping their photo on the social card. Through the user’s profile, user A can navigate user B’s posts and followers, which they are also able to do in their own profile.
Task-management (fig. 1G) in the prototype refers to the management of actions that include posts and replies. The whole idea here is to focus on these actions as tasks that need to be completed somewhere in time, instead of focusing on time alone. The application allows the user to queue posts from the "make a post" menu. Here, the user can post links, images, videos and text. From the dashboard (fig. 1N) the user can either reply or queue replies to the things that others have posted. In doing so, these actions will become part of the user productivity Thus, making these actions deliberate instead of accidental or sporadic. Here the intention is to increase quality and to leave the user with a sense of accomplishment. Of course, it is fairly easy to cross things of a lists that have already been done, but the user should also be able to reflect on their progress. Therefore, the user can access statistics (fig. 1H and I) by clicking on "posts" next to the profile picture on the left. These statistics represent completed, planned and tasks that are overdue. This should function as a way to motivate the user to continue down their productivity path.
The search icon (magnifying glass) is applied in every screen from figure 1B till 1N so that the user can view specific results that are based on the typed syntax. The magnifying glass will open an overlay (fig. 1M) in which the user can type to search for specific users or posts with a certain tag. Of course, using the right syntax will lead to better results.
4.2.2 Evaluation Setup
The sessions for the usability-testing worked much like any other usability testing sessions that are held during a design process. First, a range of testers were approached and selected who represent the intended audience. In advance, different scenarios were prepared so that the users could preform certain tasks. The interface - that is presented on a computer screen - shows the available commands, and the user should be able to recognize the one that is needed to preform a specific task. As the participants were using the prototype they were encouraged to share their thoughts and ideas. The testing sessions were documented via Lookback.io on a Macbook. This allows for observations and the debriefing of users afterwards to measure the recognition performance, because Lookback.io enables screen capture and face recording. Thus, by
combining qualitative research with usability testing a better understanding is created of user perceptions and motivations in addition to their actions.
4.2.3 Evaluation Results & Discussion
4.2.3.1 Measuring Functional Requirements
Among the most popular measures for system usability are effectiveness, efficiency and satisfaction. In this research we will focus on satisfaction. Functional requirements specify the ability of Knowlety to do certain things. Here, these abilities are measured by satisfaction of the user, i.e. how well according to the user is Knowlety able to do certain things? Meaning, that usability is a function both in appearance and behavior.
Usability is not a trait of an application that exists in the absolute (objective) sense, it is best described as a general trait of a system that is appropriate to its purpose. While other measuring methods are technology agnostic, the SUPR-Q contains four elements: usability, trust, appearance and loyalty [10]. But, this method is used for websites. There is a measuring method for mobile usability testing that still has to prove itself and is specific to the usability of mobile applications [11]. For now, the focus will be on the subjective assessment of usability by using the System Usability Scale (SUS) and its questionnaire (Appendix B) to measure acceptability [12]. The questionnaire is used after the participants have finished all tasks.
The SUS provides solid indications of the overall user satisfaction, because it has thousands of previously documented uses to compare to. With success, researchers [13] have mapped adjectives to SUS. The mean SUS score is around 68-70.5. Meaning, that according to the researchers a score of 68-70.5 makes a system 50% better than half of all other systems that were tested with the SUS. Systems that score in the 90s are "exceptional", those that score in the 80s are "good" and others that score below the 50s are considered "unacceptable" (p. 115).
In order to calculate the SUS score, first the sum score contributions from each item is presented in table 1. For questions 1,3,5,7, and 9
Figure 2. Comparison of adjective ratings, acceptability scores, and school grading scales, in relation to average SUS
score based on [12]. Table 1. SUS scores per question.
the score contribution is the scale position minus 1. For questions 2,4,6,8 and 10, the contribution is 5 minus the scale position. To obtain the overall value of the system usability, the sum of scores is multiplied by 2,5. This leads to an overall average SUS of 51 for the lo-fi prototype. As seen in the next image a score of 51 means the application has a score that is worse than average; it has a low result that hovers between "poor" and "ok". Meaning that the prototype barely makes it to being marginal (fig. 2). Therefore, it is not yet on the right track. Concluding, the application still needs a lot of improvements to make it one that is "best imaginable" regarding usability.
4.2.3.2 Measuring Functional Requirement: Nonverbal
This part of the user experience presents the emotional experiences of the users. The previous perceptions of usability could be effected by the emotion and the usability of a product will most definitely have an effect on the user’s emotions.
For this research the Emocard tool by Desmet [14] was selected for the nonverbal element. This tool consists of sixteen different cartoon-like faces (female/male) that each represent sixteen distinct emotions that are based on Russell’s circumplex of emotion. Each of these faces includes a combination of two emotion dimensions, namely Pleasure and Arousal. Using this, the reactions are divided in four quadrants [16]: calm-pleasant (positive), calm-unpleasant (negative), excited-pleasant (positive) and excited-unpleasant (negative). Therefore, reactions that are more pleasant and high in arousal are considered desirable and therefore interpreted as positive emotions. This tool allows for and animates the uncertainty that is essential for accurate emotional measurement.
The video recordings from Lookback.io enabled evaluation while the users were observed during their interaction with the prototype. Emocards were used for each of the eleven tasks (Appendix C). Each Emocard was marked that best matched the participant’s reactions throughout a task. The total number of participants with a response on the particular “expression” is counted for each task across five participants (Appendix D). Then, the general results were calculated for the prototype. These results are illustrated in figure 3. This radar chart shows how values change from one emotional expression to the other. Reactions that are more pleasant and high in arousal are considered more desirable and therefore interpreted as positive emotions. Looking at the general results, this means 22 of the responses are neutral, 22 responses are pleasant (positive) and 11 responses are unpleasant (negative). Although some of the results are high for more positive responses, the excitement is somewhat low, namely 3.
Four out of five participants showed a positive change in emotion during the "login with LinkedIn" task, and the same thing happened during the third and fourth task. Although, only a limited amount showed excitement. The respondents showed some positive emotions during task 9, 10 and 11. But, neutrality is a response that is occurring in each task. Compared to the SUS this isn’t entirely unexpected. Three out of five participants showed negative change in emotion during tasks 8. More remarkably, four out of five participants showed negative responses to tasks 6 and 7.
4.2.3.3 Discussion
The questionnaire results from the usability test are summarized in figure 4 in which the arrow shows the desired scores. The y-axis represents the score contributions, from 1 that stands for "strongly disagree" till 5 that stands for "strongly agree". There are also suspected outliners represented with different icons. The results indicate that the teachers are more neutral towards a majority of the questions. However, the results show that teachers would feel confident using the application. While the teachers don’t think they need support from a technical person to use the application (question 4) they do feel that its functions are not very well integrated and that they can imagine that most people would not learn to use the application very quickly (question 5 and 7). Taking a look back at the recordings and the negative emotion during task 6, 7, 8 one can see where the integration problem may lie: (1) posting an image; (2) adding a person to connections and (3) viewing statistics (Appendix D).
Additional feedback from the participants pointed out that they expected a bit more space in the interface; i.e. the upper half of the interface seemed to have little purpose. They suggested to use the entire screen for the different pages so that the app becomes less bulky. Some of the participants were not too keen about the chat. They felt that the chat feature could become overwhelming. Although these types of communication are centralized and beneficial for university staff, they are not so much a boost for productivity and are more often than not confusing.
Although the SUS is not diagnostic and does not give any indications of where the system fails, it does show whether the Figure 3. Visualization of the gathered emotional responses
for the lo-fi prototype.
Figure 4. SUS scores divided into positively (a) and negatively (b) phrased questions for the lo-fi prototype.
De
si
re
system still needs improvement. In addition, the combination of the SUS, recordings, face readings and feedback has given more insight into the system's problem areas. Thus, going into the hi-fi prototype the focus will be on solving these issues as well as elaborating on more details and functionalities.
4.3 High-fidelity Prototype
The user experience of Knowlety is of course related to a specific set of users, so a small size of five people is unlikely to be representative of the total teacher population at the UvA. Nonetheless, the lo-fi prototype has given sufficient findings to reevaluate the prototype design. The design of this prototype focusses on several use case collections: (1) sharing, managing and editing content; (2) tagging content; (3) searching for tags, posts and people; (4) viewing statistics and (5) private messaging.
4.3.1 Design & Implementation
This final iteration focusses on solving the issues that were pointed out in the previous lo-fi iteration. This iteration will also try to bring more clarity and delight to the application. To add to the delight of the user and familiarity it seems logical to have some of the UvA "house-style" elements included; the UvA red, white, black with
the addition of UvA’s web blue. For more clarity the menu is discarded from the pages; i.e. it is now included as a side navigation that functions as an overlay. Because of the negative feedback for the chat, this was also discarded and replaced by a private messaging feature with priority ranges and privacy options. In this iteration most of the previous features are deployed in a different manner so that the interface reaches its full potential. Thus, more space becomes available for content and visualizations. For a better understanding of the application’s continuous stream a visual flowchart is illustrated in figure 5. The flowchart is a simple representation of the main scenarios that are evaluated during the test.
Solutions are given for (1) posting an image; (2) adding a person to connections and (3) viewing statistics, which were problematic in the first iteration. Firstly, for all three more space has become available in the interface that allows for an overall better integration (fig. 5). Secondly, posting content (e.g. an image) is made less complicated due to a better integration in the timeline and profile (fig. 5C and L) with the use of an overlay (fig. 5R). This overlay can be accessed by tapping "what are you working on" in the timeline and profile. Posting an image now shows the albums Figure 5. Visual flowchart for hi-fi prototype that was used for evaluation.
that are on the phone at the top of the page. Thus, the user can now browse through images in albums while thumbnails are shown (fig. 5S). The user simply selects images that they want to post. Advanced post options enable the user to plan such posts to be published on a certain time and date or they can queue posts (Fig 5X). This queue allows for a storage of 100 posts. In addition, adding another user to connections is represented with a larger plus icon with a black background so that it stands out more. This icon is implemented in the social cards on the map (fig. 5J), the received social cards in the connections (fig. 5M), on the timeline for users whom others reblogged from (Fig. 5L) and in the search overlay (for users as well as tags). By doing so, it is made easier for the user to get connected with other users. Finally, visual changes are made in the statistics page so that the graphs for actions that are planned, completed or overdue are layered (fig. 5H). Thus, the difference between the three during a certain month becomes more visible.
The interface for posts shows all previously posted content in an infinite scrollable screen with the most recent post shown first (fig. 5I). At the top of the interface the user can choose to filter these posts by their type: text, image, link or video. Here, the user can also access the planned posts and posts that they have liked. Each user has their own profile in which they can access several other pages. The user doesn’t always have to use the navigation menu to access tasks, recent activity, followed tags or the posting content overlay. They can just post content from their profile once they have accessed the application by signing in. Their previous posts and connections are also there for easy access (fig. 5C).
If users are already connected, profiles are accessed by tapping user’s pictures or names. But, if that is not the case - by tapping the picture or name - the user will be presented with their social card first (fig. 5Q). This enables the user to read brief information about the other user such as rank title, profession, popular tag and amount of posts before going to their profile. In the crossed paths section (fig. 5K) and timeline other users can be added directly by tapping the plus icon. If for instance user A adds user B to their connections, the system will send out user A’s social card to user B. As a result, user B will receive a social card in their connections page, they then need to accept by tapping the plus icon so that the connection is made i.e. the connection is only made if both user have added each other. Secondly, once the user accessed the other user’s profile they are presented with an interface that is almost similar to their own profile (fig. 5C), except for the lower half. This area shows the other user’s featured posts (fig. 5P). These posts are the ones that have the most notes i.e. reply-, reblog- and like-count. In this sense the social card and profiles have become more advanced compared to the first iteration.
Featured posts will also add to the personal brand building. But where the first lo-fi iteration gave a glimpse of this concept for Knowlety, this hi-fi iteration introduces more titles and a white or red laurel to motivate knowledge sharing. At first, these titles will take into account the amount of posts. It would make sense to add the amount of likes and reblogs received from others. Except, likes and reblogs are intrinsic rewards and making them part of the application’s reward system would result in Knowlety turning into a popularity contest very quick. In any case, at the first stage sharing is more important than the likes, reblogs or replies a post gets. In the beginning staff members should be motivated to share something personal or about education regardless of how many likes, replies or reblogs it gets. These elements will function as additional incentives for the user. In the first stage there are three different ranks with titles that are visible:
1. Newbie with no stars: lower than 20 posts
2. Learner with one stars: 20 posts and more 3. Guru with two stars: 100 posts and more
Stage two includes counts for posts, likes, reblogs and replies for these ranks. Counting will start when the user reaches a Guru ranking, and the titles are as follows:
4. Supreme with three stars: 200 posts and more, and a total of 500 likes, reblogs and replies included.
5. Master with four stars: 400 posts and more, and a total of 1000 likes, reblogs and replies.
6. Philosopher with five stars and red laurel: 700 posts and more, and a total of 2000 likes, reblogs and replies.
In this iteration task-management, the dashboard and map are slightly more improved. Firstly, the dashboard is now a timeline where all posts by other users - who are in the user’s connections - are gathered in an infinite scrollable interface with the most recent post at the top (fig. 5L). These results can be refined with the advanced filter overlay that can be accessed from the timeline (fig. 5U). The user’s activity center is also available from the timeline (fig. 5V). Here the notifications for messages, likes, reblogs and replies are gathered in one screen. Secondly, for the map is more spacious and has an added standard range for only UvA buildings (fig. 5J). This can be turned on with the UvA icon. Accordingly, the map will only show nearby people and crossed paths inside UvA buildings only.
The search overlay is made more detailed in this prototype so that the user experiences the several stages of search in Knowlety: (1) opening search and using the right syntax for tag search; (2) autocompletion and (3) filtering (fig. 5E). Autocompletion enables the user to quickly find and select from a pre-populated dropdown list of tags and people before any final results are shown (fig. 5F). Messaging has replaced the chat from the first iteration with added privacy options and priority range (fig. 5T). With each replied message the message is closed so that the messages don’t turn into chats that are overwhelming. For instance, if user A sends user B a message user B will be notified with "User A has sent you a message" in their notification center. This message will also show what range of priority user A has selected: high, medium or low priority and if the message should be private or not. The additional privacy options allow the users to send messages that can be posted so that other users can reply, like and/or reblog to create more collaboration among the user network. If as a result user B responds - either private or public, which is up to user B if the private option was not selected - the message is closed and user A is notified with "user B has replied to your message" in their activity center. This is to motivate face-to-face interaction and to set up meetings instead of messaging back and forth.
As for collaboration, the believe - in this last iteration as well as in this first one - is that teamwork is essential to collaboration, but too much of it can have lower productivity as a result. Research suggests that people are more creative when they enjoy privacy and have freedom from interruptions [16]. Collectivism may also lead to like-mindedness, which shuts out alternative viewpoints. Thus, individualism is encouraged with the assistance of collaboration. In the prototype this is mainly done through messaging and addressivity with the "@" icon (fig. 5W); i.e. any user can mention others in their text indicating an intended addressee. The addressee will be notified for any time they are mentioned. This type of interactions functions in a similar way as gazing does in face-to-face interaction when the directed gaze turns either to a person or group as a whole [17].
4.3.2 Evaluation Setup
The hi-fi prototype is designed in Sketch using the results from the lo-fi prototype usability test. The artboards in Sketch are made clickable in InVision. The interactions with the prototype is made possible on a mobile device (iPhone 5S), which allows the bridging of gaps in the articulation of interaction. The prototype is explained in the following section that is followed by an evaluation. For comparison, the setup for the hi-fi prototype testing was kept somewhat similar to the evaluation of the lo-fi prototype. But Loopback.io embedded in InVision viewer was used for recordings on the iPhone itself. Thus, the interface was presented on the iPhone 5S screen which showed the available interfaces and commands that were needed to perform certain tasks. While at the same time the iPhone recorded the participant’s face with the front camera and their touch interaction on the screen.
4.3.3 Evaluation Results & Discussion
4.3.3.1 Measuring Functional Requirements
To obtain the overall value of the system usability for the hi-fi prototype the SUS was used for this iteration as well. The sum of the score is presented in table 2. The SUS scores per question led
to an overall average SUS of 71,75 for this prototype. Whereas the previous iteration hovered between a result of "poor" and "ok" this one is now more acceptable (fig. 6). Although it barely makes it to being "acceptable", it does show that the applied procedure for the second iteration has resulted in a closer approximation to a solution. But, a more acceptable result would be anywhere between "good" and "most imaginable". Therefore, more iterations are necessary to get closer to the desired degree of usability.
4.3.3.2 Measuring Functional Requirement: Nonverbal
Emocards were also used for the nonverbal part of the evaluation of the hi-fi prototype. Here, emocards were marked that best matched the participant’s emotions during twenty tasks. This resulted in a total number of participants with a response on a particular “expression” for each task across twenty participants. In appendix E the total number of participants with a response on a particular “expression” is counted for each task across twenty participants. The general results are visualized in figure 7. This iteration gives a summarized result of, 165 neutral responses, 174 pleasant (positive), 30 excited neutral (positive) and 31 unpleasant (negative). Compared to the previous radar chart for the lo-fi prototype, this one shows more spread towards excitement and a relatively pleasant user experience with reduced negative responses.
Task 3 “find a person near you with whom you have the most tags in common and open their social card” and task 6 “see who you have crossed paths with” have the most excited responses. Other tasks have responses that are more calm and average pleasant. There are some tasks that show a spread of responses that are more positive, namely tasks, 5, 7, 9, 10 and 20.
4.3.3.3 Discussion
This iteration for the evaluation of Knowlety has overall better results than the first. Meaning, that the changes that were made actually paid off. The results for the SUS are summarized in the box plots in figure 8 on the next page. This shows more desired results compared to the lo-fi prototype. The results illustrate that the teachers would like to use the application more frequently and that they don’t think they need support from a technical person or that they need to learn a lot of things before using it.
Most of the participants agreed on question 2, they didn’t find the application unnecessary complex. But, the question does have some suspect outliners. A few teachers found the application more complex and had trouble using it. Two of which were having a negative experience using the prototype. Further dialogue indicated that they actually felt overwhelmed by the amount of options. This Table 2. SUS scores per question for hi-fi prototype.
Figure 6. Second comparison of adjective ratings, acceptability scores, and school grading scales, in relation to
average SUS score based on [12].
Figure 7. Visualization of the gathered emotional responses for the hi-fi prototype.
was mostly because of the extent of functionalities. Concluding that the application is doing a bit too much according to their preference.
Additional feedback was quite positive. The teachers like the way the application is presented. Although it was represented by a prototype, the teachers were impressed by the user interface. The most encouraging responses verbal and nonverbal were related to the crossed paths section and map. Not all functionalities appeared to be popular among the teachers e.g. replying to a post on the timeline. A few teachers suggested to narrow down the application to a few dominant functionalities so that it becomes less overwhelming. They also suggested to use other visualization for the statistics so that it is more “clear”. Concluding, that while Knowlety is more accepted by the participants it still needs improvement to match better the needs of the teachers.
5. SYSTEM DESCRIPTION
In the following section a description is given of the choices made with regards to the system’s software and hardware. These choices are based on the technical recommendations for Knowlety. The aim
of the system is to be used by a majority of UvA's academic staff members who are able to run the application on their mobile device. For now, the system is prototyped for iOS. But, this could later be extended to Android and other operating systems.
5.1 Connecting Users
5.1.1 Location-awareness
For some of the functionalities of Knowlety it is important that location is tracked. As GPS does not always work for tracking indoors, Wi-Fi is one of the most significant input technologies for location. GPS on the other hand is very power consuming. Wi-Fi localization has two methods, namely fingerprinting and triangulation. The latter can be used with off-the-shelf tools. Feng et al. [18] have described how triangulation works: (1) measuring the signal strength from the mobile system between three access points by the Log-distance path loss model. This is represented by a variable d and L(X, Y) defines the location of the mobile system (fig. 9).
5.1.2 Finding Others and the Right Information
These days, social (/friend) recommendation is the most popular feature of social media. But, to find useful information by way of applying this function is difficult. Zheng and Zhu describe that these - often inaccurate - social recommendation results are largely blocking the social interaction among micro-blog users [19]. Therefore, it is valued to combine certain interests from the user to their location. So that, these "friend" recommendations become location based. With Knowlety a location based "friend" recommendation system is proposed that recommends surrounding users with similar used tags to each other. The system will also show other users nearby so that these users don’t get filtered out; i.e. the users have less chance of getting inside a filter bubble [20].Knowlety provides users with effective ways of annotating collaboratively and the organization of items with their own vocabularies by using tags. However, due to the flexibility of such a system a large number of redundant tags may be the result. Wu et al. have introduced the Friend Recommendation algorithm by User Similarity Graph (FRUG) to find potential connection with the same interest in a social tagging system such as in Knowlety [21]. To lighten the problem of tag redundancy they have utilized Latent Dirichlet Allocation (LDA) to obtain user’s topic of interest. And to calculate similarity from users’s topic of interest, co-collected items and co-annotated tags, they proposed a multiview user’s similarity measure method. This approach is highly recommended since the experimental tests resulted in a good performance of FRUG in terms of precision and recall.
5.2 Search and Filtering
In the search interface the user can access recommended tags that are based on tags that are popular inside the user’s network. In addition, a search algorithm should perform searches for tags as well as user names. The most recent and relevant result will appear at the top of the page. This should allow for multiple tag search and the use of k-nearest neighbor for tag recommendations.
5.3 Architecture
There are plenty of aspects to an application’s performance. At the server side an important aspect is to get a page including all assets to the client as fast as possible. In this sense the data access needs to be fast while at the same time the architecture is scalable so that it doesn’t break down. Although Knowlety is currently non-deployed, it may increase in size overtime. Therefore, the MySQL rational database is recommended; such databases are able to grow Figure 8. SUS scores divided into positively (a) and
negatively (b) phrased questions for the hi-fi prototype.
De
si
re
d
largely before any scaling horizontally is needed. Thus, the database should have plenty of joints that are mostly centralized. And lastly, it should consist of a monolithic design that enables real-time monitoring and log analysis with no backend services.
The Knowlety architecture (Appendix F) consists of two primary components: the user interface and data. The data consists of (1) posts that are the largest among all data, (2) user statistics and (3) additional aggregated data such as counts for connections, posts, reblogs, replies and likes.
6. CONCLUSION & FUTURE WORK
In this research two prototypes were designed and evaluated. The first prototype contained incomplete and sketchy characteristics of the target iPhone application. It was kept simple in order to produce a prototype that is quickly made - so that the evaluation of broad concepts could take place. Concluding that the participants are neutral towards this first iteration and that it hasn’t come near the desired results with a SUS of 51. But, participants consider the map for finding others useful. The second iteration resulted in a high fidelity prototype with more details and functionalities. This iteration came closer to desired results with a SUS of 71,75. It also presented more positive reactions from the participants, which were mostly related to the crossed paths section and map. This approach enabled the evaluation of user behavior and its relation to the use of the final product. While not al details were subject to evaluation, this study helped identify certain design issues before large scale quantitative research can take place.
To understand the requirements for a system that fits best the needs of UvA’s academic staff members, this paper presented the first results of a case study. The challenge here was to present a solution for a complex problem that wasn’t easily defined. Vliek’s survey helped identifying the root cause for this problem, which resulted in a solution presented in the first iteration. Design thinking required this research to move beyond the logical and academic methods. The user requirements can feed into functional requirements as well as into the UI design. When usability is viewed as merely a nonfunctional requirement major issues might arise that are in the way of satisfying user objectives. Therefore, several key aspects of design thinking were used. Firstly, a deeper understanding of the user experience was created by the prototype evaluations using the SUS. Secondly, context and affect were used as tools to discover previously unexpressed user needs by watching, discussing and listening. This allowed for the research to get useful feedback. Additionally, the hi-fi prototype was deployed in order to tackle issues from the first iteration and to approach desired results.
Additional evaluations that are more diagnostic are needed to evaluate Knowlety in more detail. In these first two iterations general aspects of the application were tested. But, in the next versions it could also support collaborative tasks and secondary profiles for groups. While at the same time it is also important to discard functionalities that are problematic. To make stronger conclusions about how the user behavior relates to the use of the final product, a pilot study with a running application should be conducted to evaluate costs, feasibility, affect and performance on a larger scale.
7. ACKNOWLEDGMENTS
First, I would like to thank my supervisor Radboud Winkels for his hospitality, feedback and giving me the trust to work on this research independently. Furthermore, thanks to Frank Nack for supervising when Winkels was absent and for being second
reviewer to this research. Finally, special thanks to the people at InVision for their email feedback and conversations.
8. REFERENCES
[1] Brouwer, N., Byers, B., Maciejowska, I. (2014): A Platform for sharing Expertise in University Chemistry and Chemical Engineering Teaching, Special Edition Virtual Education Community, EC2E2N NewsLetter 2014, 15(5), ISSN, 2309-5911, pp. 1-6.
[2] Panahi, S., Watson, J., & Partridge, H. (2012). Social media and tacit knowledge sharing: developing a conceptual model. World academy of science, engineering and technology, (64), 1095-1102.
[3] Haythornthwaite, C. (2005). Social networks and Internet connectivity effects. Information, Community & Society, 8(2), 125-147.Riemer, K., Altenhofen, A., & Richter, A. (2011). What are you doing? -enterprise microblogging as context building. In ECIS.
[4] Zhao, L.and Lu, Y. (2010). Perceived Interactivity: Exploring Factors Affecting Micro-Blogging Service Satisfaction and Continuance Intention. Paper presented at the PACIS.
[5] Liu, Z., Economics, D., Min, Q., & Liu, Z. (2014). THE IMPACT OF PERCEIVED INTERACTIVITY ON
INDIVIDUAL PARTICIPATION IN MICRO-BLOGGING. [6] Riemer, K., & Richter, A. (2010). Tweet inside:
Microblogging in a corporate context. Proceedings of the 23rd Bled eConference, 1-17.
[7] Zhao, D., & Rosson, M. B. (2009, May). How and why people Twitter: the role that micro-blogging plays in informal communication at work. In Proceedings of the ACM 2009 international conference on Supporting group work (pp. 243-252). ACM.
[8] Azab, N. A. (Ed.). (2012). Cases on Web 2.0 in Developing Countries: Studies on Implementation, Application, and Use: Studies on Implementation, Application, and Use. IGI Global.
[9] Zhang, J., Qu, Y., Cody, J., & Wu, Y. (2010, April). A case study of micro-blogging in the enterprise: use, value, and related issues. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 123-132). ACM. [10] Sauro, J. (2015). SUPR-Q: A Comprehensive Measure of the
Quality of the Website User Experience. Journal of Usability Studies, 10(2).
[11] Hussain, A., & Ferneley, E. (2008, November). Usability metric for mobile application: a goal question metric (GQM) approach. In Proceedings of the 10th International
Conference on Information Integration and Web-based Applications & Services (pp. 567-570). ACM.
[12] Brooke, J. (1996). SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), 4-7.
[13] Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, 4(3), 114-123.
[14] Desmet, P., Overbeeke, K., & Tax, S. (2001). Designing products with added emotional value: Development and appllcation of an approach for research through design. The design journal, 4(1), 32-47.
[15] Russell, J. A. (1980). A circumplex model of affect. Journal of personality and social psychology, 39(6), 1161.
[16] Cain, S. (2012). The Rise of the New Groupthink.”. New York Times, 13.
[17] Bays, H. (1998). Framing and face in Internet exchanges: A socio-cognitive approach. Linguistik Online, 1(1). Re- trieved May 20, 2008 from http://www.linguistik- online.de/bays.htm
[18] Feng, J., & Liu, Y. (2012). Wifi-based indoor navigation with mobile GIS and speech recognition. Int. J. of Computer Science, (9), 6.
[19] Zhu, J. Q., & Zheng, M. (2015, July). Location Based Friend Recommendation for Online Social Network with
Hypercube. In 2015 International Conference on Artificial Intelligence and Industrial Engineering. Atlantis Press. [20] Pariser, E. (2011). The filter bubble: What the Internet is
hiding from you. Penguin UK.
[21] Wu, B. X., Xiao, J., & Chen, J. M. (2015). Friend
Recommendation by User Similarity Graph Based on Interest in Social Tagging Systems. In Advanced Intelligent
Computing Theories and Applications (pp. 375-386). Springer International Publishin
APPENDIX
A. USE CASES
B. SUS QUESTIONNAIRE
D. EMOCARD SCORE PER TASK: LO-FI PROTOTYPE
F. DATABASE ARCHITECTURE & SCHEMA
The app should be able to grow in complexity only as needed. But, we know where the ceiling is, since we know how many educational staff members the university has. Thus, we can gain insight into what the app may grow into. But, it is good to start with a simple app that is a relational database, which is mostly centralized in one database with some basic functional partitioning across multiple databases (fig. 1). So you might have the user’s table completely separated from the tag’s table. But, with the added joints and aggregated queries of off the core data whenever one wants to see counts.
One of the pressing issues might be too many posts for one database and aggregate and filtered lookups can take too long. On the posts themselves there are the additional aggregated data such as reblogs, likes and replies. These are actions that happen on a certain post. Since post are the biggest problem one should shard the post data by user and user ID (fig. 3). Meaning, that one loses the ability to do joints and so it moves to only primary key lookups. This results in the fact that joints and filtering happen in app. In addition, the post index must be sharded by time (fig. 2). Thus, every view of the first page of the timeline must go to the most recent post index database.
Figure 2. Real Time Ordering of User Posts in Timeline. Figure 1. Database Architecture.
