User-centered design: A qualitative case study on
improving interface design for transaction-based systems
Marius van der Knaap
University of Amsterdam Amsterdam, the Netherlandsmmvanderknaap@gmail.com
ABSTRACT
This study applies user-centered design methods to a digi-tal communication tool for construction projects. Methods include a literature review, heuristic evaluation, a competi-tive review, semi-structured interviews and usability testing. The research resulted in user personas and requirements and supported the development of a prototype. Preliminary user testing showed positive results towards the prototype. Im-plications for transaction-based systems are formulated from these results.
Keywords
User-centered design; interface design; user experience
1.
INTRODUCTION
The term user-centered design was first proposed by Donald Norman in 1986 [1]. It is a general term for ‘philosophy and methods which focus on designing for and involving users in the design of computerised systems’ [2]. Big organisations such as Google and the UK government are stressing the importance of user-centered design [3, 4].
Many software developers fail to involve users in the design process. They either think it is too time-consuming, are hesitant to show their unfinished products, or don’t know how to involve users [5]. The latter is the case for the com-pany Bakker&Spees (‘B&S’). B&S developed a communica-tion system for construccommunica-tion projects called VISI.
The system is based on the Dutch standard for creating con-ditions for the introduction of standardisation-ICT in the civil engineering sector (‘VISI standard’) [6].1 This open standard structures all communication in transactions. Us-ing the system ensures that all official project communica-tion is in one place.
1Note that the VISI system is named after the VISI stan-dard. From here on, the system is referred to as ‘VISI’ and the standard it is based on as ‘VISI standard’.
The system includes many features such as creating reports on past communications and filtering transactions. How-ever, VISI product specialists report that the system lacks in user-friendliness. Users often take months to get used to the software. Section 4.1 goes into more detail about the sys-tem’s current problems. Solving the problems is important because decreasing the learning curve and improving acces-sibility of the system’s features should improve the system’s success.
According to Abelein, Sharp and Paech [7], user participa-tion and involvement benefits user satisfacparticipa-tion and system success. Involving users in system development benefits us-ability, efficiency and motivation [8]. Poor interface design leads to frustration or even dangerous situations in the case of i.e. medical equipment [9].
This study researches whether user involvement in a user-centered design process during system design can help re-solve the problems. Because most problems exist in the system’s user interface (UI), the main focus of this study is the UI.
This leads to the following research question:
How can the VISI user interface be improved through user involvement in a user-centered design process?
Answering this leads to improved understanding of best prac-tices in the design of transaction-based systems. It helps in focusing the design process on a specific aspect of the system while applying proven user-centered design methods.
2.
DESIGN PROCESS
The user-centered design process is formalised in the Inter-national Organisation for Standardisation (ISO) standard 52075 [10] and can be seen in figure 1. User-centered design is an iterative process, involving users in many aspects of design.
The first phase provides a scope for the research, established in the research question. An analysis of the context of use provides context for the user requirements. This analysis is carried out using proven usability research methods. First is a review of the VISI standard on which VISI is based. The problems of the current system are examined in the problem statement. Then, the competitive review looks at similar products and their design decisions. A product spe-cialist interview and user interviews provide further context
of use. Personas and requirements can be derived from the context analysis, which provide for the construction of an interface prototype. Finally, the prototype is evaluated and implications for transaction-based systems are formulated.
Figure 1: User-centered design process according to ISO standard 52075 [10]. Design always start with a planning process, after which each phase is car-ried out several times until the system meets the requirements.
2.1
Usability testing
Usability testing has become a common approach in user-centered design. Usability tests were used extensively during this study to receive feedback about design decisions. Us-ability testing means observing people who are trying to use the software with the goal of making it more user friendly by observing people using the system [5]. This study car-ried out qualitative usability tests, meaning no quantitative measures such as task completion time or success rate are measured [5].
A standard testing protocol was established for the usability tests. The protocol ensured participants were treated as equally as possible during every testing session and are based on Steve Krug’s usability test script [11]. The test protocol, tasks, interview questions and consent form can be found in appendices F through I.
3.
LITERATURE REVIEW
The VISI standard is registered under ISO standard 29481 [12] and is based on Dynamic Essential Modelling of Organ-isation (DEMO) methodology [13]. DEMO is a design and engineering methodology for organisations developed by Jan Dietz [14]. DEMO structures all communication in transac-tions.
Figure 2 illustrates the three phases of a transaction pro-cess: order phase, execution phase and result phase. Each transaction has an initiator and an executor, seen in figure 3. VISI is used most during the execution phase.
VISI users are assigned roles and may be first-, second- or third-line users as seen in figure 4. First line users are at
Figure 2: The transaction process for any type of production process according to DEMO methodol-ogy [14].
Figure 3: Product states from a DEMO-perspective. A product iterates through the cycle of requested, promised, stated and accepted, until the final ac-cepted state is achieved [14].
the top of the communication chain and usually carry the end responsibility for a project. Second and third line users may be advisers or contract managers.
The types of transaction a user can send depend on their role in the project. Transactions are either sent to the other company (i.e. from client to contractor), or to other roles within the same company. Not all transactions within a project are visible to all users. Users can only see transac-tions that they have permission to see. One company can never see internal transactions from the other company.
Each role has different responsibilities, and therefore has different user goals. For example, first-line users often carry the end responsibility but send few messages themselves.
Instead, they oversee communication done by second-line users. These are often the people sending the most messages. Some persons supervise different projects but are not heavily involved in a single project. There is a special supervisor role for these types of users.
Figure 4: Roles users may have during a construc-tion product. A role may only communicate one level up or down. Supervisors can see all communi-cation that the company is allowed to see.
4.
PROBLEM STATEMENT
4.1
Existing VISI
Users and developers often compare the VISI interface to Mi-crosoft programs such as Outlook. The left menu consists of folders which unfold when clicked on. A new transaction is started by clicking the button on the top left of the screen. Clicking the parent transaction opens the most recent sub-transaction in the reader frame at the bottom. Unfolding the parent transaction shows its sub-transactions, ordered newest first. Clicking on a subtransaction opens that sub-transaction in reader frame. There are many more features which will not be discussed in detail here.
Van Soest, VISI product specialist, named some of the prob-lems users currently have with the interface. Users struggle with the vocabulary used in the software. Terminology is not always known to the user even though it is part of the contract. It is difficult to see what happened during a action and what possibilities a user has to continue the trans-action. Sometimes the VISI projects are too complicated for first-time users. Furthermore, users report poor integration across different software platforms. There are likely to be more problems not uncovered by the product specialist.
4.2
Heuristic evaluation
A heuristic evaluation was conducted to reveal additional problems with the current VISI system. Heuristic evalua-tion is an empirical evaluaevalua-tion method by Nielsen [15]. It is an informal method requiring usability experts to judge whether a system follows the principal design guidelines (the ‘heuristics’). Nielsen formulated ten heuristics as seen in ap-pendix D [16].
The current VISI system was tested by two usability ex-perts. The evaluations were conducted conform to the pro-tocol by Nielsen [17]. First, the evaluators explored the in-terface freely. They also completed the B&S VISI training designed for VISI users to learn about the system. The
eval-Figure 5: The current VISI UI. Common actions are displayed at the top. Different folders and filters can be chosen at the left side menu, which is then displayed on the right side. Clicking on a transaction opens the corresponding transaction at the bottom.
uators were then asked to judge the interface based on the ten heuristics.
Both evaluators agreed problems existed in the interface. The most critical problems according to the evaluators are discussed here. The complete heuristic evaluations can be seen in appendix D.
Evaluators reported that there were too many scrollbars, text was too small and menus unfolded into too many op-tions. One evaluator missed ‘breadcrumbs’, an indicator of where he was in the system. This evaluator also reported that he was unable to see the content of the message he was responding to when formulating a response.
Evaluators found that the relationship between labels and actual functionality of buttons were unclear. For example, the document export button (‘print’) did not match the eval-uators’ expectation of the functionality. Below is a sum-marised list of most critical problems of the current VISI system according to the evaluators and product specialist.
• Unintuitive terminology, metaphors and icons • Difficult to understand what happened during a
trans-action
• Projects are sometimes too complicated for first-time users
• Unnecessary screen clutter • Often unclear where the user is at
• Bad legibility in some parts • No appropriate help function • Bad integration across platforms
5.
COMPETITIVE REVIEW
This study intents to solve these issues by creating an al-ternative UI design. Looking at similar applications helps gain insight into requirements [18] and prevents reinventing the wheel [19]. Therefore, this section performs a compet-itive review of similar systems to inform subsequent design activities [18, 20, 19].
The competitive review looks at four different systems: Google Inbox, Basecamp, DocStream, and Infostrait’s Alfamail. While all the systems offer many features, only relevant elements such as similar features or interesting design decisions will be discussed here.
5.1
Inbox
Inbox by Google incorporates Material Design into their email system. Material Design is a visual language inspired by the study of paper and ink. The goal is to create a sin-gle underlying system that allows for a unifying experience across platforms. The full set of design guidelines can be found on Google’s Material Design web page [21].
One of the problems of the current VISI system is that it is difficult to understand what happened in a transaction. Selecting a conversation larger than four messages illustrates Google’s solution to this problem, as seen in figure 7.
Figure 6: Inbox by Gmail. The figure shows the result when selecting the ‘Promos’ label. These are filtered by Google’s algorithms as ‘deals, offers, and other marketing and promotional messages’ [22]. Messages are automatically grouped by date.
Figure 7: The last message in a conversation is shown in its entirety. The first message is shown summarised. Messages in between are collapsed. Clicking on the counter unfolds these messages in a similar fashion to the way the first message is dis-played.
5.2
Alfamail
There are currently two systems available based on the VISI standard: VISI and Alfamail. Figure 8 shows Alfamail. Alfamail contains similar features to VISI. It has filtering by transaction status and a report feature. The transaction list shows fewer details than VISI’s transaction list. Only subject, sender and response deadline are shown.
Users have reported they found Alfamail easier on the eye than VISI [23], because it has more space between lines and better legibility of text items.
Figure 8: Alfamail by Infostrait [24]. Alfamail im-plements a three-window interface. The left dis-plays the top-level menu. Next to is it the message overview, where the user can select a specific trans-action. This is displayed in the final frame on the right.
5.3
Basecamp
Basecamp is a project management platform. Basecamp offers six tools ‘that every group needs to do any kind of work together’ [25]. These include a message board, a shared to-do list, document exchange, check-ins, a schedule and an informal ‘campfire’ chat for off-topic conversation. These tools may be applicable to VISI as well.
5.4
Docstream
Docstream is a document management system often used alongside VISI. Users often manage their documents inter-nally on Docstream which they transfer through VISI when necessary. Docstream contains several document manage-ment features such as version managemanage-ment and commanage-menting. Documents are shared throughout the project. Managing all the documents in one central location prevents people from using older versions of documents.
5.5
Conclusions from competitive review
Following design conventions makes sure the user knows in advance what is going to happen when performing an ac-tion [26]. Design lessons can be learned by looking at similar products. The competitive review reveals that VISI is essen-tially a type of messaging and project management system. Users have less freedom in choosing their message content in VISI compared to other messaging systems, because VISI messages must always follow a predefined structure.
Inbox shows how multiple messages in a conversation may be organised and the most essential elements required for a conversation overview. It reveals that Inbox prioritises the last and first messages in any conversation by collapsing the messages in between.
Most other project management systems have document management. Documents often have version control and comments. Docstream allows users in a project access to the same project folders. This ensures that people will al-ways use the latest version of a document. Because Doc-stream and other platforms are often used alongside VISI, integration across platforms is important.
Because of the widespread use of Material Design guidelines in Google’s products, many VISI users will already be famil-iar with it. Therefore, the VISI prototype will use Material Design guidelines.
6.
USER RESEARCH
User research is ‘an early phase of user-centered design dur-ing which potential users are studied in order to understand usage context and user needs relating to the designed prod-uct’ [27]. In this study, the goal of user research is to reveal who the people using the system are. This facilitates persona creation and requirements elicitation. User research is done through user interviews and a product specialist interview.
6.1
Product specialist interview
One method to gain insight into users is by interviewing peo-ple in an organisation who deal with users [28]. B&S’s prod-uct specialists know most about VISI and its users. Prodprod-uct specialists meet users during the trainings they give to new customers. The full interview with Joyce van Soest, VISI product specialist, can be found in appendix E, while the most important findings are discussed here [29].
VISI users are predominantly male in the age range of 30 and 60 years. Van Soest explains there are three distinguishing variables between users: experience with the software, com-puter skills and function within the company. People with little computer skills have more difficulty using the system and are more likely to dislike working with the software.
In most cases, use of VISI is stipulated by the initiator (client) in the building contract. Any executor (usually a contractor) who takes on the contract is then required to use VISI for communication with the client. People work-ing for the client side are often civil servants. Civil servants are generally more focused on contracts and organising ev-erything according to the paperwork. Contractors prefer taking action over working with documents and generally dislike VISI more than civil servants. Users often have little motivation to use the system if it is imposed by superiors. Sometimes teams have become used to VISI and request use of VISI themselves.
Experience with the software determines how easily users adapt to new features. In general, users fall under one of two groups when solving problems in the interface: ‘clickers’ or ‘askers’. ‘Clickers’ start clicking around to find answers, while ‘askers’ prefer asking questions to solve their problems.
6.2
User interviews
In addition to the product specialist interview, five users with different roles have been interviewed. Three users from the client side on a VISI project have been interviewed, and two users from the contractor side. The list of questions can be found in appendix H, and the most important findings are discussed below.
6.2.1
Who are the users?
When asked, all users answered that what they liked most about their jobs was cooperating with different people to create a beautiful project. The cooperation is also the major point of frustration for many users when one of the parties does not fulfill their duties.
On average, users spend a lot of time behind their desks. Their desks are located in the office or at the site office. The time spent behind a computer varied from four to eight hours per day. Other activities are meetings and visiting construction sites. A lot of communication still goes through email. Document management is often done in other soft-ware programs as well, such as Sharepoint, GroupWise or Docstream.
6.2.2
Why and how do they use VISI?
About one hour or less per day is spent on VISI. Some users have the VISI software open all the time, some process their VISI transactions once on a daily basis. Users are triggered to open the software when receiving an automated email from the system about an incoming message.
Most users lack inherent motivation to learn or use the sys-tem. Some see the benefits of the system and stated that it helped them organise their documents and communication better. Seeing the benefits contributed to the motivation to use the system.
When asked about the limitations of the software, users could only state minor usability issues. However, most users mentioned it took them a long time to get accustomed to the system. Some worried about the fact that only they knew which transactions to use. They feared substitutes would not be able to do their work in VISI in the case of unex-pected leave or illness.
6.3
Lessons from user research
VISI is not the users’ motivation to come to work every day. Users are likely to have little motivation to use the system. Use of VISI is often imposed by superiors. This means VISI must be an easy-to-use system, as users are likely to get frustrated quickly. The system must support both ‘clicker’ and ‘asker’ types. There must be easy recovery from erroneous clicks, and a help function must always be visible and show relevant help information.
Because of the broad range of technological expertise, new features must be appropriately introduced so that the com-puter illiterate can get to know them. However, the duction can also be skipped if the user requires no intro-duction. Finally, integration between software packages is helpful to users. Using the document metadata such as cre-ation time may help improve integrcre-ation. Other lessons from user research are incorporated in the personas.
7.
PERSONAS
According to Mulder [30], ‘a persona is a realistic charac-ter sketch representing one segment of a Web site’s targeted audience’. Personas summarise findings from user research and help development staff make decisions based on these instead of assumptions, Mulder argues. Personas focus de-sign activities by helping the team prioritise system features and content that best support the audience [20]. Personas were developed because VISI developers rarely get in touch with users. Personas help VISI developers understand the target audience to help drive design [20].
Because there are no software analytics in VISI, personas are based on qualitative data. Some assumptions have to be made in qualitative data, because findings from interviews cannot be grounded in quantitative evidence.
Two personas were created based on the two distinct user types: clients and contractors. Usually, a first name is suffi-cient for a persona [30]. More personal information includes job and company, age, location, personality, home/family life and hobbies. The personality is illustrated by three character traits. The basic information is concluded with a photo [30].
The profile is a story describing the persona. It provides the context of who the personas are. A quote captures the personas essence and is one of the first things people read. Other information includes technology experience, person-ality, goals and frustrations, and motivators. The result is a one-page document describing the personas attributes and goals. The final personas can be found in appendix A, and will be briefly introduced here.
The two personas are named Steven Boogaard and Gerard Scholten. Both are middle-aged men working in the civil engineering sector. Steven works for the contractor BAM Infra. His main goals are creating beautiful projects and co-operating with people He does not like working with VISI, because he already uses another document management sys-tem. What he likes about VISI is the fact that he can or-ganise his transactions and create overviews. Every week, he creates an overview of tasks for his colleagues.
Gerard works for the port of Rotterdam. He is not as well organised as Steven. One of the reasons he likes VISI is because it helps him keep all his documents in one place. He is not very good at keeping documents organised himself. Gerard’s main motivations are the realisation of projects and acting as the link between contractor and client. He gets frustrated when contractors try meddling with their contracts or try to deliver less work than agreed on.
7.1
Taxonomy
Creating the taxonomy aimed to solve the problem of incon-sistent and unintuitive terminology in the current interface by formalising the use of language in the system. A focus group consisting of product specialists was organised to cre-ate the system taxonomy. During the brainstorm sessions, each concept was discussed individually until a new term was agreed on. Icons were chosen to accompany the new terminology.2
The most important concept is transaction. The term is cur-rently used because it is introduced in the DEMO method-ology. However, users found it difficult to explain exactly what they expected from it. Especially people who were new to VISI could not describe what a transaction was. Af-ter evaluating different concepts, it was decided transactions would be now be named ‘conversations’.
Conversations can have different statuses. A conversation is made up of individual messages. A message can contain documents (previously: attachments or documents). A col-lection of conversations is called an overview (previously: report). The initiator is always called client and the ex-ecutor is always called contractor. Previously, these were sometimes abbreviated.
Besides labels, data and time notations have been formalised. Previously, these were always written out fully in DD-MM-YYYY HH:MM format. For a more personalised user expe-rience, a dynamic timestamp format was created, as seen in table 7.1.
Figure 9: Basic VISI taxonomy tree illustrating the system’s most fundamental concepts.
2
Note that VISI is Dutch and terms discussed below are translated. There may be slight nuance differences between the actual terms and the translated terms discussed here.
Timestamp When to use
Just now Between now and one minute
Today, hh:mm Between now and 0:01 today
Yesterday, hh:mm Between 0:01 and 23:59 yesterday
DD month Standard timestamp
DD month YYYY HH:MM Extended timestamp for hover-overs etc.
Tomorrow With a planned item tomorrow
Today With a planned item today
Table 1: Timestamps for VISI. Time is always writ-ten with two digits, even if not explicitly required (i.e. one past nine is still 09:01).
8.
REQUIREMENTS
Requirements for the VISI software can now be established. Wiegers and Beatty [31] discuss various ways to elicit soft-ware requirements. This study used interviews and obser-vation methods in requirements gathering [31].
Requirements are divided into two categories: functional and non-functional requirements. Functional requirements specify the behaviours the product will exhibit under specific conditions [31], i.e. the things a user can do. Non-functional requirements describe the product’s characteristics that are important to either users or developers such as performance, safety and availability [31]. Non-functional requirements es-tablish necessary background processes. Requirements are defined in a tree structure. Top-level requirements are dis-cussed here. Appendix C contains the full system require-ments.
Functional requirements
• Conversation list– Columns:
Status, users involved, subject, documents, date – Filters
– Refresh – Search • Conversation view
– Information per message:
Type, response deadline, notes, date, documents, to, from, subject
– When possible: respond/forward – Export
– Print • New conversation
– Process indicator – Screen 1: Select recipient – Screen 2: Select type – Screen 3: Input form • Document management
Use document metadata.
Shows all documents exchanged in the project. Allows filtering of documents to fine-tune results. Folders allows manual document grouping.
• To-do list
Shows all active tasks the user is responsible for. Tasks can be deleted.
• Overview
Shows statistics about the current project. Overview selector for the report feature. • Projects
Allows changing the current project. Persons, roles, organisations management. • Current project
Clicking opens quick project change. • Notifications
• User info and log off • Hotkeys for expert shortcuts
Non-functional requirements
• Intuitive and modern interface using Material Design • Speak the users’ language
• Support transition period from old to new system • Remember user settings across sessions
9.
PROTOTYPE DEVELOPMENT
The requirements form the basis for the prototype develop-ment. The interaction design started with low-fidelity wire-frames sketched on paper. A wireframe is a blueprint of a system with a simplified view of what content will appear on each screen, usually devoid of color, typography and style [20]. VISI wireframes served the purpose of starting the in-teraction design process.
Paper prototyping is a variation of usability testing where representative users perform realistic tasks by interacting with a paper version of the interface [32]. The interface is manipulated by the researcher, responding to user actions by presenting them the resulting screens. Paper prototyp-ing allows quick iterations and improvprototyp-ing a design rapidly without having to write any code [32]. Several B&S em-ployees tested the paper prototype in a number of iterations until they stopped giving new insights. The testing goals were:
Testing the taxonomy
Users were asked what they expected buttons to do. If their answer did not correspond to the intended functional-ity, other terms and icons were tested until a suitable label and icon were found.
Testing the interaction
The paper prototype supported the most common VISI in-teractions, such as creating a new conversation and opening a conversation overview. Users carried out the tasks during testing. Interaction schemes were changed in subsequent it-eration when users took too long or were unsuccessful in completing the task.
Testing the placement of components
Users were asked whether they thought the placement of components was correct. If the answer was negative, the component was moved until a satisfactory position was found.
Testing the conversation view
One of the problems of the current VISI system is the diffi-culty understanding what happened during a conversation. Conversations are more complicated than conventional mes-saging systems. This has several causes, for which the solu-tions found during testing will be described here.
• Multiple people in a conversation.
The first name and last name of the sender is displayed in every message.
• There can be different recipients of a message. The first and last name of the recipient is displayed in every message.
• Users can belong to two different parties.
There is a visual distinction between external and in-ternal messages.
• Users cannot see all messages in a conversation. Messages are hidden if a user does not have permission to see it.
• Users have a ranking (first, second, or third line). Second and third line messages are indented.
Figure 10: Paper prototypes for VISI. In a usability test, people were ask to interact with the prototype as if it was a digital screen. After pressing a but-ton, the interviewer presented them with the next screen.
The prototyping tool Justinmind [33] was used to create the digital prototype. The first iteration was based on the latest version of the paper prototype. This was tested in usability tests in accordance with the usability testing protocol.
Intermediate prototype versions were tested with partici-pants selected through convenience sampling. This means
that participants were not actual users but rather persons available that represented users in some way. Often encoun-tered problems and their solutions were:
• Finding the right button.
The button’s position or visuals were changed.
• Understanding the meaning of icons and labels. Icons were moved closer to their corresponding fields or the label was changed.
• Experiencing screen clutter.
Margins were expanded or unnecessary details removed. • Meaning of items in to-do list was unclear.
The visual representation and content of items was al-tered.
• The intention of the overview screen was unclear. Further user testing with an actual implementation of the system is required to assess whether the overview displays relevant statistics.
Figure 11: The final hi-fi VISI prototype. The left side displays the main menu. To the right is the content frame. The conversation list shows all the conversations within the project and their status. Clicking on a conversation opens that conversation [34].
9.1
Evaluation
The final prototype was evaluated with actual VISI users. Screenshots of the final prototype can be found in appendix B. No quantitative measures such as task-completion-time have been used during the evaluation. Therefore, results from the evaluation are qualitative and may be subjective.
The users mental model of the system was mostly correct. Users understood that transactions were now named conver-sations and they understood the system underwent a visual change but the underlying system was still the same.
Overall, users completed the tasks successfully. The final prototype was tested on two users. Because of the small sample size, there are likely to be more issues. The problems the testers encountered were:
• Finding the ‘new conversation’ button.
• Attaching a document, likely caused due to the limited functionality of the prototype.
• Understanding the to-do-list.
Users expected the list to show all messages they had to respond to, while the prototype only showed mes-sages that had an expired response deadline.
10.
IMPLICATIONS FOR
TRANSACTION-BASED SYSTEMS
Findings from this research can be generalised towards transaction-based systems. Developers of transaction-based systems may learn from design decisions made in this study. This section describes the most important lessons learned.
Transaction-based systems are essentially messaging and project management systems. However, the complicated message structure requires careful design of the conversation overview. Note that users are often not inherently motivated to use the system. Ensure that they have a smooth experience because they are more likely to get frustrated quickly.
Users often manage a multitude of different software pack-ages. Integration with other software packages is important. Additionally, there must be a uniform standard for the data exchanged between systems. Data exchange between plat-forms is sometimes difficult because one system requires a different data format than the other. One solution is using document metadata. This is a uniform standard across op-erating systems. Using the document metadata allows uni-form data exchange and decreases users effort. Metadata can also be used to fill in parts of a form automatically, further decreasing user effort.
Systems may be based on an underlying methodology. For example, VISI is based on DEMO methodology. In VISI, the term transaction comes from DEMO methodology but research revealed the term does not make sense to the user. Always research whether the terminology from underlying methodology is suitable for your product.
11.
CONCLUSION
The VISI standard imposes the use of a predefined structure in communication between persons working on a construc-tion project. Doing so stimulates users to communicate in the right manner. Bakker&Spees is a company that creates software based on the VISI standard. The software called VISI helps prevent mistakes and helps keep the overview of communication within a project.
However, product developers stated that users often had trouble learning to use the system. The heuristic evalua-tion showed the interface is cluttered, metaphors were un-clear and system status was not always visible. Additionally, there was no appropriate help screen, the system sometimes had bad legibility of text and gave little guidance when fill-ing in forms. Problems were likely caused due to a lack of user involvement during system design. The personas were intended to learn VISI developers about the user. A VISI de-veloper can now ask himself ‘does this design decision make sense for my personas?’.
To increase the intuitiveness of the system, a taxonomy was created formalising the systems terminology. The taxonomy creates more user-friendly and intuitive terminology allow-ing the user to understand what somethallow-ing is without learn-ing about it.
The prototype was developed through several iterations. Each iteration was evaluated with usability tests. The resulting prototype is a non-functional prototype demonstrating the systems core functionalities and can be seen in appendix B. The next step for VISI is actual implementation. After creating the functional system using the prototype and re-quirements, more usability tests may be used to evaluate the resulting system.
Involving users proved invaluable during the design. Users revealed problems during testing that would otherwise not have been found. Users provided ideas for new features and helped prioritise existing ideas. The new interface designed with user involvement shows promising results during eval-uation.
12.
DISCUSSION & FUTURE WORK
More user testing may be required in the future. While the prototype was developed with cross-platform compatibility in mind, it is currently only designed for desktop PCs. A mobile and tablet friendly version will have to be developed and tested. Furthermore, only the most essential features were included in the prototype. The competitive review re-vealed several other tools that may be beneficial to VISI’s project management, such as Basecamp’s ‘campfire’ chat. These may be investigated at a later stadium.
13.
REFERENCES
[1] Donald A. Norman and Stephen W. Draper. User centered system design. New Perspectives on
Human-Computer Interaction, L. Erlbaum Associates Inc., Hillsdale, NJ, 1986.
[2] Chadia Abras, Diane Maloney-krichmar, and Jenny Preece. User-centered design. In In Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications. Publications, 2004. [3] Google. Google design. https://design.google.com/,
referenced 23 May 2016.
[4] User research community. User research. https: //www.gov.uk/service-manual/user-research/, referenced 19 May 2016.
[5] Steve Krug. Rocket Surgery Made Easy. Pearson Education, 2011.
[6] Niek Pluijmert. DEMO Awareness course, 2016. [7] Ulrike Abelein, Helen Sharp, and Barbara Paech. Does
involving users in software development really
influence system success? IEEE Software, 30(6):17–23, 2013.
[8] World Wide Web Consortium. Involving users in web projects for better, easier accessibility.
https://www.w3.org/WAI/users/involving, referenced 13 July 2016.
[9] Joanna Lumsden, editor. Handbook of Research on User Interface Design and Evaluation for Mobile Technology. IGI Global, February 2008.
[10] ISO 9241-210:2010. http://www.iso.org/iso/ catalogue_detail.htm?csnumber=52075, referenced 11 July 2017.
[11] Steve Krug. Usability test script.
http://www.sensible.com/downloads-rsme.html, 04 july 2016.
[12] ISO 29481-1:2010. http://www.iso.org/iso/ catalogue_detail.htm?csnumber=45501, referenced 11 July 2017.
[13] Jan L.G. Dietz. DEMO: Towards a discipline of organisation engineering. European Journal of Operational Research, 128(2):351 – 363, 2001. Complex Societal Problems.
[14] Jan L.G. Dietz. DEMO Awareness course, 20 January 2016.
[15] Jakob Nielsen. Usability inspection methods. In Conference Companion on Human Factors in Computing Systems, CHI ’95, pages 377–378, New York, NY, USA, 1995. ACM.
[16] Jakob Nielsen. 10 Usability Heuristics for User Interface Design. https://www.nngroup.com/ articles/ten-usability-heuristics/, 1995. [17] Jakob Nielsen. How to Conduct a Heuristic
Evaluation. https://www.nngroup.com/articles/ how-to-conduct-a-heuristic-evaluation/, 1995. [18] Jenny Preece, Yvonne Rogers, and Helen Sharp.
Interaction Design. John Wiley & Sons, Inc., New York, NY, USA, 1st edition, 2002.
[19] Andrea Soverini. UX project checklist.
https://uxchecklist.github.io/, 19 may 2016. [20] Dan M. Brown. Communicating Design: Developing
Web Site Documentation for Design and Planning. Voices that matter. New Riders, 2011.
[21] Google. Material Design – Google design guidelines. https://material.google.com/, referenced 04 July 2016.
[22] Google. Clean up your inbox with bundles. https:// support.google.com/inbox/answer/6050237?hl=en, referenced 12 July 2016.
[23] Arne Bruinse. VISI:verbetering uiterlijk frontend (internal communication document).
https://intern.bakkerspees.nl/wiki/, References 12 July 2016.
[24] Alfamail. Alfamail visi.
https://www.alfamail.nl/wp-content/uploads/ 2015/04/alfamail-visi-standaard.png, 2016. [25] Basecamp. The tools of basecamp 3.
https://basecamp.com/3/tools, 04 july 2016. [26] Jakob Nielsen. Top 10 Mistakes in Web Designn.
https://www.nngroup.com/articles/ top-10-mistakes-web-design/, 2011. [27] Petri Mannonen. Technology cultures. In E.G.
Blanchard, editor, Handbook of Research on Culturally-Aware Information Technology:
Perspectives and Models: Perspectives and Models, pages 94–113. Information Science Reference, 2010. [28] User research community. User research.
https://www.gov.uk/service-manual/
user-research/start-by-learning-user-needs, referenced 19 May 2016.
[29] Joyce van Soest. Product specialist interview, 08 May 2016.
[30] Steve Mulder and Ziv Yaar. The User is Always Right: A Practical Guide to Creating and Using Personas for the Web. New Riders Publishing, Thousand Oaks, CA, USA, first edition, 2007.
[31] K. Wiegers and J. Beatty. Software Requirements. Developer Best Practices. Pearson Education, 2013. [32] C. Snyder. Paper Prototyping: The Fast and Easy
Way to Design and Refine User Interfaces. ITPro collection. Morgan Kaufmann, 2003.
[33] Justinmind. Prototyping platform for web and mobile apps - Justinmind. http://www.justinmind.com/, referenced 04 July 2016.
[34] Webalys. Icons from Streamline icons. http://streamlineicons.com/, 13 July 2016.
Conversaties
o Kolommen: Status
Ontvangen
Ontvangen, te laat gereageerd Verstuurd
Verstuurd, te laat gereageerd Afgerond
Concept
Ongelezen: vetgedrukt datum laatste bericht, type,
documenten, o Sorteren op kolommen o Paginavigatie
o Filters:
Traditionele filters al aanwezig in de VISI software
Onderzoek welke filters gebruikt worden en display alleen die o Rechtermuisknop
Markeer als gelezen Reageer
Conversatie
o Informatie per bericht: Transactietype Reactietermijn
Alleen als relevant Opmerkingen
Eerste ### letters voor zover interface dat toelaat Datum verstuurd
Bijlagen met downloadfunctionaliteit o Opslaan als PDF
o Afdrukken
o Sorteer: oudste/nieuwste eerst o Concept opslaan
o Wanneer mogelijk bij een bericht: Stuur door
Reageer
o Terug naar conversatieoverzicht knop
o Klik op bericht voor uitklappen
Zie alle informatie individuele bericht Afdrukken bericht
Opslaan bericht
o Creeer indent bij doorsturen 2e/3e lijnen o Visuele weergave in/uit doorsturen in/uit
Jouw taken
o Alle berichten die door de gebruiker beantwoord dienen te worden o Alle documenten die binnenkort aangeleverd moeten worden Nieuwe conversatie
o Scherm 1:
“Kies een type conversatie:” Maak selectie uit types conversatie o Scherm 2:
“Kies conversatie:”
Maak sub-selectie uit selectie type conversatie o Scherm 3:
Transactierelevant invoerscherm
Toon altijd uploadscherm op dezelfde plek Documentbeheer
Overzichten
o Overzicht (oud: rapportage) kiezen
o Snelle cijfers om inzicht in huidige project te geven Projecten
o Wissel tussen actieve projecten van de gebruikers o Projectbeheer
Rollen, personen, organisaties Snelle acties
o Snelle acties kunnen toegevoegd worden via rechtermuisknop op een bepaalde actie of voorgeprogrammeerde snelle acties in het instellingenmenu
Hoofdscherm functionaliteiten (altijd zichtbaar)
o Huidige projectnaam
Klik op projectnaam voor snel wisselen project o Verversen o Zoeken o Welkomsttekst Naam Inlognaam Jouw rol Aantal projecten
Profiel wijzigen (shortcut naar instellingen) Uitloggen
o Notificaties
Global notificaties (dus ook voor nieuwe berichten in ander project) Nieuwe notificatie bij bijwerken conversatie
Klik op notificatie icoon om notificatie te zien Klik op notifcatie om naar bericht te gaan
Kan ook gebruikt worden om push berichten naar alle gebruikers te sturen van B&S (bijvoorbeeld bij ingeplande reboot van server)
o Help
Geeft relevante informatie weer voor huidige scherm Start tutorial
Telefoonnummer B&S
o Instellingen Hotkeys
Linker menu-items 1-7 Nieuw C/N
Ctrl + L logoff
Snelle acties toevoegen aan hoofdmenu Versie-instellingen (terug naar v4) Profielfoto
Standaard profielfoto is eerste letter voornaam + eerste letter achternaam (“MK”)
Favorieten ja/nee weergeven Thema’s
Margeselector (desktop/tablet) o Rechtermuisknop
Voeg snelknoppeling van deze actie toe aan hoofdmenu o Inloggen
Ga naar laatste actieve project
o Hover-over geeft hint functionaliteit en sneltoets weer Bv. “Maak nieuwe conversatie (N)”
Niet-functionele requirements
Spreek de taal van de gebruikero Intuitieve labels
o Communiceer binnen de software op dezelfde manier zoals we dat zouden doen buiten de software (informele manier van communiceren)
Verbeter herkenbaarheid van het merk B&S
o Door logo van huidige software altijd weer te geven Ondersteun transitieperiode
o Maak terugschakelen oude interface mogelijk o Geef een tutorial bij eerste gebruik
Tutorial kan opnieuw opgevraagd worden in helpmenu Veranderingen in raamwerkeditor
Onthoud instellingen gebruiker Sidemenu in-uitklappen Weggeklikte taken
+-knop vouwt uit in favoriete conversaties net als in Google Inbox Verberg helpteksten in instellingen bij nieuwe conversatie
Bijlagen op achtergrond versturen maar popup weergeven met status ergens Animaties
Instellingen leesvenster onderin of in nieuw scherm voor conversatie openen Sneltoetsen
Instelbare view detaillevel conversatie (mini, normaal, alles uitgeklapt) Icons
Bakker&Spees – Entrepotdok 63A – Amsterdam.
Heuristic Evaluations B&S VISI v4
11-04-2016
1. Visibility of system status
No breadcrumbs/highlights of current action
Can’t see relevant document where the transaction is about Titles of transaction all start with the same word, making it unclear Send may have been better placed on the screen
2. Match between system and the real world
A status icon is better than colours Functionality of reports/files unclear Projectmanagement = settings? Files is a weird metaphor Files unfolds too long
Filters fit better in message screen instead of in their own folder Avoid scrollbars
Message should fill entire screen rather than being displayed in a sideframe
3. User control and freedom
Unclear what to fill in (lacking domain knowledge
Adding attachments or working documents only has to unfold when adding them + and – for adding attachments is an unclear metaphor
4. Consistency and standards
Messages/transactions/documents are named different each time Handle unclear label -> respond would be better
Header could display breadcrumbs
5. Error prevention
Fields need a clearer indication when they are required to fill in
6. Recognition rather than recall
Can’t see what I’m responding to when formulating a response
7. Flexibility and efficiency of use
Respond with document would be handy
The most common functionalities must need the least clicks Unnecessary screens
Hide non-editable values
There are no hotkeys for expert users
8. Aesthetic and minimalist design
Too many scrollbars everywhere
Bakker&Spees – Entrepotdok 63A – Amsterdam. Menu is too long
Message is too small, can only read header on smaller screens Many fields in a form. Are they all necessary?
Things rarely used don’t need to be in the left menu
I didn’t use files for any of the task, so maybe it’s unnecessary
9. Help users recognize, diagnose, and recover from errors
Maybe make a concept message feature?
10. Help and documentation
Ensure learning curve is as small as possible Help does not supply relevant help information A tutorial would help
Bakker&Spees – Entrepotdok 63A – Amsterdam.
Product specialist interview B&S VISI v4
12-04-2016
Translated from Dutch
1. Who are the users?
In principal, there are three main distinguishing variables between users. Users are either experienced or inexperienced with the software. Another variable is computer skills, which differ between users. The final variable is whether they work for the contractor or client side. Clients want to work safer, they want to have their responsibilities well written down, are more precise.
Contractors often have fear they can’t deviate from the contract, prefer just doing things than writing them down. Older users often have a lot of difficulty in using the system. Secretaries are not meant to use the system but sometimes their bosses tell them to use VISI because they themselves don’t feel like they have the time to use VISI. Often, the younger users are, the easier it is to use VISI. First line managers are often dominant. During trainings, I see two types of users: clickers and askers. Clickers start clicking around to find what they are looking for, while askers prefer asking me for a solution.
2. How do you experience how users use VISI?
What makes sense to use does not always make sense to the users. Vocabulary is often unclear. Just because it is stated in the contract does not mean the users know what it means.
3. What types of vocabulary is unclear?
Transaction is a typical VISI-term4. What do you think of user acceptance?
After a while, users see the benefits of VISI. However, an often heard phrase is “I still need to get some experience with the system before I feel comfortable with it”.
5. What do you think is the goal of the user?
Capturing communication. Managing a project. After a while users start enjoying VISI more.
6. Why do people use VISI?
Sometimes just because their bosses tell them to. Sometimes, motivation comes from the team itself. This is often when use of VISI became a habit.
7. What do you think are the problems with the interface?
It is difficult to see what happened in a transaction. Especially with larger transactions, it is difficult for people to see who did what and what happened with all the messages. A problem people encounter is that they got a message, they want to do something with it, but don’t know exactly what they have to choose. They don’t know who it goes to, where it came from. Some projects are too complicated, especially for first-time users. A project with three layers (lines) is too complicated for first-time users. Sometimes, knowledge about contract kinds is missing with users. VISI assumes people know about how these contracts work. Preventing problems requires a pattern, but it is difficult if people don’t know about the pattern. A to-do list would be useful for future VISI.
Bakker&Spees – Entrepotdok 63A – Amsterdam.
Usability test protocol
Translated from Dutch Steps
1. Introduction 2. User interview 3. Usability test Introduction usabilitytest
We are going to do a usabilitytest now. I have some information here that I am going to read form this paper so that I’m sure you will get all the information.
Today we are going to evaluate the Bakker&Spees VISI software, so that we can see if it works the way it is supposed to work. This session will take approximately 30 minutes.
The most important thing to know is that we are testing the software, not you. You can do nothing wrong.
While you are using the software I am going to ask you to think aloud as much as possible, to say what you are looking at and what you are thinking.
Don’t be afraid to hurt our feelings, we are only doing this to improve our software.
If you have any questions, don’t be afraid to ask them. Maybe I won’t answer them immediately because we are trying to find out how people will use the system without someone watching over their shoulder. In that case I will answer you questions after this session. If you need a break, please let me know.
With your consent I will record this session, including what happens on the screen and our conversation. The recordings will only be used to help us improve the system. That helps me because I have less notes to make.
Would you mind signing this form? It says we have your permission to record this session and that the recordings will only be used in our team.
Sign form Start recording Do you have any questions?
Open the software
Thank you. I will now ask you to perform some tasks. Like I said, try thinking aloud as much as possible.
Give participant the first scenario
Continue until nothing new is found or participant gets tired
Bakker&Spees – Entrepotdok 63A – Amsterdam.
Usability test tasks
Translated from Dutch. Heavily summarised in translation for convenience.
1. Send a new message to the client 2. Deal with to a message
a. File an internal request b. Respond to the internal request c. Complete the transaction 3. Find a specific transaction 4. Edit your display settings 5. Print a message
6. Create a report of specific document type
Translated from Dutch 1. What’s your name? 2. How old are you? 3. Where do you live?
4. What kind of work do you do?
5. What do you like about your work?
6. What are you frustrations during your work?
7. On average, how much time do you spend behind a computer per day?
8. What kind of programs do you use the most?
9. What do you think of the VISI-software?
10. Why do you use VISI?
11. How much do you use VISI?
12. Where do you use VISI most?
13. What are your biggest annoyances using VISI?
Bakker&Spees – Entrepotdok 63A – Amsterdam.
Toestemmingsverklaring
Ik ga ermee akkoord te participeren in het onderzoek uitgevoerd en opgenomen door Bakker&Spees.
Ik ga ermee akkoord dat de opnames gemaakt door Bakker&Spees gebruikt worden voor onderzoeksdoeleinden. Ik begrijp dat de informatie en opnames alleen gebruikt worden voor onderzoek en dat de opnames niet gebruikt worden voor andere doeleinden. Ik ontbind mijn rechten op de opnames en begrijp dat de opnames zonder mijn toestemming gebruikt mogen worden.
Ik begrijp dat deelname vrijwillig is en zal direct bezorgdheid of ongemakken melden aan de onderzoekers.
Teken de overeenkomst om aan te geven dat je de informatie op dit formulier gelezen hebt en dat vragen die je nog hebt beantwoord zijn.
Datum: ___________
Naam: ____________________________________________________
Handtekening: ___________________________
Bedankt!
Je deelname wordt zeer op prijs gesteld.
