Guidelines for increasing the Usability of Mobile Applications

(1)

Guidelines for increasing the

Usability of Mobile

Applications

A Case Study

Suzanne van der Tweel

Studentnumber: 10253025

Universiteit van Amsterdam

Supervisor: Dick Heinhuis

(2)

Abstract

The objective of this research was to find a framework which can increase usability in mobile applications during the development process. A framework is presented based on a literature review of more than 100 mobile usability studies, defined by Coursaris and Kim (2011). Satisfaction, learnability, efficiency and effectiveness are the four usability dimensions conducted from a literature review. With these dimensions numerous practical usability guidelines were found and categorised. These guidelines were tested with a case study of a mobile application, during the development process. For each dimension, one guideline was chosen and for this guideline a solution was applied to the existing application. A new version of the application was created and was tested with a questionnaire in order to validate the usability model. Only for the dimension effectiveness a significant difference was found. The three other dimensions did not have a significant difference, and could therefore not be used to increase the usability of mobile applications. In further research more usability guidelines can be validated by different case studies.

(3)

1 Introduction

Computer technology is one of the most powerful developments of the last decade. Com-puters, tablets and mobile phones are of great importance in our lives. Everybody is using mobile devices almost everyday with 1.91 billion smartphone users in 2014 and 80% of the internet users owning a smartphone (Statista, 2015b; Chaffey, 2015). Smartphones are de-fined by the Nielsen Company in 2012 as full-screen phones which has a device-sized touch screen and a graphical user interface driven by touch gestures. A lot of things have changed since the release of the smartphone. According to Nielsen and Budiu (2013) users are much more familiar with their mobile phones. More and more people are using their smartphones for day to day tasks. In 2008, Mary Meeker made a prediction that mobile will overtake the fixed internet access by 2014 (Bosomworth, 2015). This shows the importance of the shift from Desktop to Mobile devices.

The amount of mobile app downloads is increasingly growing. From 2.52 billion down-loads in 2009 to 268.69 billion downdown-loads expected in 2017 as shown in figure 1. The competitive behaviour of mobile applications has also increased extensively (Tunguz, 2014). The average application swings 10-30 ranks per month, compared to 1-5 ranks 18 months ago. This shows the enormous competition inside the mobile application market. Also mobile users have a lot of choices between different applications. It is very important that companies are able to differentiate in order to become successful.

Figure 1: Number of mobile app downloads worldwide from 2009 to 2017 in millions (Statista, 2015a)

However, smartphones are not always a bed of roses. In table 1 the conversion rates of 100 million site visits across e-commerce clients are shown. Much fewer mobile users bought with their mobile phone instead of desktop computer users. According to Nielsen and Budiu (2013) there is one conclusion about the huge difference in conversion rates between desktop computers and mobile phones: The mobile user experience must be horrible.

(5)

1 INTRODUCTION

Device Conversion Rate Desktop Computer 3.5% Mobile Phone 1.4% Tablet 3.2 %

Table 1: Conversion Rates of E-commerce clients of 100 million websites (Nielsen & Budiu, 2013).

It is much harder for mobile devices to maintain a good user experience compared to desktop computers. There are a lot of limitations and challenges for mobile interfaces due to the size of the devices. Coursaris and Kim (2011); Nielsen and Budiu (2013); Harrison, Flood, and Duce (2013) defined the following limitations:

• Small versus big screens • On the move versus stationary • Touch versus mouse

• Wireless connectivity versus faster wired internet • Low resolution screens

• Less processing and power capability • Navigational issues

• Less precision in pointing

Apparently these limitations are not sufficiently considered. The limitations of mobile de-vices require more attention compared to desktop computers. Nevertheless, there is a dif-ference between interface design and usability design (Juristo, Moreno, & Sanchez-Segura, 2007). Usability is about the whole user-interaction with the system and not just the in-terface. The interface is just the visible part of the system and interaction is a much wider perception. Interaction is the coordination of information exchange between the user and the system (Juristo et al., 2007, p.1508). Therefore this problem lies under a much wider concept, named Usability.

Usability studies done by Nielsen and Budiu (2013), show that users perform much better with mobile devices with mobile applications instead of mobile websites. They measured a success rate of 74% for mobile applications and 64% for users that used mobile websites. Mobile applications are more usable, because of the limited optimisation of mobile websites. ”An app can target the specific limitations and abilities of each device much better than a website can while running inside a browser”, Nielsen and Budiu (2013, p.34) concluded from their research. Next to that applications are simpler than websites, which can be denoted as the success of mobile applications. Mobile applications are designed especially for smartphones, websites are not.

Weiss (2003) says that usability is defined as the ease of use. When people encounter usability problems that is due to a bad design and not because the user is stupid. He denotes that happy customers are loyal customers and usability can be a important factor. He also states that there is lack of usability on most handheld devices. It seems that usability becomes less important in the development of mobile applications. It is striking, because

(6)

1 INTRODUCTION

earlier usability was a key factor in the development of the personal computer, which will also be explained in section 1.2 (Campbell-Kelly, Aspray, Ensmenger, & Yost, 2009). This indicates that usability is actually a very important factor for developing applications.

However usability is a product that many products have, but many, many more lack according to Rubin and Chisnell (2008). It is hard to know when something is usable or not, because usability is only a problem when it is lacking or absent (Rubin & Chisnell, 2008, p.3). Usability is invisible, only when something is missing or not working properly, people will notice it. For example, when the temperature in a room is fine for everyone, people will not complain about it. When it becomes too cold or warm, people will notice it. Rubin and Chisnell (2008) define usability as the absence of frustration in using the product. The user can do whatever he or she wants to do in the way he or she expects to do it, without hesitation or questions.

With developers uploading more than 1000 applications to the Apple store per day, it is of great importance that the usability of mobile applications will be increased (Richter, 2015). Are there ways to increase usability, which make the user experience less horrible than described earlier?

In a research done by Gould and Lewis (1985), several principles were recommended for the role of usability in the design process. They said that developers should be focused on the users and should involve them in the design process. Users are normally involved at the end of the design process and are only used to validate or reject the design. Developers should have an interactive design, which includes a group of experts that consists of typical users of the product. These potential users should become part of the design team from the beginning and should have the most influence. They should be able to reject and validate the design before it is actually coded. It is important that designers are aware of human characteristics, but they must go beyond this and understand cognitive and emotional factors of the users that are related to the designed system (Gould & Lewis, 1985). Usability was not taken into consideration in the late 80s, because designers found usability obvious and were not included in the development process. This also demonstrates the shift in the importance of usability in software design.

This indicates the role of the user for the measurement of usability. D. A. Norman (1986) recommends a user-centered design which starts with the needs of the user. The interface is the system, from the user’s point of view. The user does not know how the system works in the background and should be able to gain knowledge from the interface design. Because of this the user should drive the design of the interface for the interactions with the system. The purpose of the system should serve the user and not to use a piece of programmed technology. ”The needs of the users should dominate the design of the interface, and the needs of the interface should dominate the design of the rest of the system” (D. A. Norman, 1986, p.61).

As the business of mobile applications is growing everyday, usability becomes more and more invisible. People will adapt to the use of applications or they will choose another one. For companies it is of great importance that the mobile applications they develop are usable. As described above, usability is tested at the end of the development process. Mobile devices have a lot of limitations, which make usability harder to accomplish. It would be of great value to increase the amount of usability inside the development process of mobile applications. The role of the user is very important in this process and could be responsible for a higher usability. Therefore the research question is defined in the next paragraph.

(7)

1.1 Research Questions 1 INTRODUCTION

1.1 Research Questions

Research question: Is it possible to develop a framework that can increase the usability of mobile applications during the development process?

The goal of this research is to find a framework to increase the usability inside mobile applications. This framework will be tested in a case-study of a mobile application which is still in the development process. In this way companies can gain a lot of advantages to create a user-centered design. The following subquestions can be defined in order to answer the research question:

1. Subquestion: Which frameworks can be used to define usability in mobile applica-tions?

2. Subquestion: Which practical guidelines can be determined from these usability frameworks?

3. Subquestion: Can the relevance of the founded practical guidelines be validated?

In order to answer the research question properly, it is important to understand what usability is exactly. In the paragraph below this will be explained.

1.2 The definition of usability

In the previous paragraph usability is described already. Because of this wide concept it is important to first explain the definition of usability, before continuing with the literature review. According to the ISO 9241-11 standards the definition of usability can be denoted as followed:

”Usability: the extent to which a product can be used by specified users to achieve speci-fied goals with effectiveness, efficiency and satisfaction in a specispeci-fied context of use”(Abran, Khelifi, Suryn, & Seffah, 2003).

In the beginning of this century the focus on mobile usability has increased in the Hu-man Computer Interaction field (Coursaris & Kim, 2006). The role of usability has changed rapidly since the development of the personal computer. From the beginning of the devel-opment of the personal computer, it was important that it became more usable (Campbell-Kelly et al., 2009).

Back in the 80’s, computers were complicated and hard to use for computer users that had practically no experience. With only basic training in software applications, the average user was unable to interact with computer systems. However, software designers assumed that the average user had more knowledge, which was quite a misunderstanding. Computer systems were associated with frustrations and were actually not used by most users (Soegaard, 2012). A computer could be controlled with the Microsoft Disk Operating System(MS-DOS), this was a command-line interface which was used to execute programs on computers (Microsoft, 2015). This language was not user-friendly at all and the command had to be exactly correct to execute the program. If one letter was missing the line had to be entered again. People without a technical background found this way of communicating with a computer difficult. In the 80s, when Apple released the first personal computer, usability was one of the main factors that contributed to the success of the Macintosh. Computers could be controlled

(8)

1.2 The definition of usability 1 INTRODUCTION

with icons and images and became much more user-friendly for people without a technical background. Steve Jobs was inspired by the Graphical User Interface that was designed at Xerox for the Xerox Alto. He used this for the development of the Macintosh and it was one of the first developments in computer usability. To solve the problems of the command-line interface, the Graphical User Interface was introduced by Apple (Campbell-Kelly et al., 2009).

Landauer (1995) describes in his book, Trouble with Computers, that there is a differ-ence between usability and usefulness. Most of the time, when people have problems with interactive systems it is because the system has lack of usability. For example, if you enter the letter ”K” inside a software program and in some cases it can mean ”Kill” or ”Keep” for the same program, there is lack of usability. No matter how usable the system is, because the functionalities work just fine, there is a usability problem. This is because programs are often written by different programmers, at different times and sometimes even by different companies (Landauer, 1995).

Usability became a key factor in the development process of interactive systems, which were not used by people with a technical background. The assumptions from the soft-ware designers became unacceptable, because people simply would not use their computer systems. In 1970, usability was defined as software psychology (Coursaris & Kim, 2011). According to Carroll (1997): ”Human-computer interaction (HCI) study is the region of in-tersection between psychology and the social sciences, on the one hand, and computer science and technology, on the other”. In the early days, psychology and social sciences were not taken into account in the development process. The goal of software psychology back then was to let developers consider the human characteristics, in order to create understandable software and interactive systems. It took over a decade for the computer industry to realise a user-centered development process was of great importance. Finally usability became a primary goal inside interactive systems and usability specifications are now standard in hu-man computer interaction development (Carroll, 1997).

In the next section an answer will be found to the subquestions defined in section 1.1. A literature review will be done and a usability framework will be presented. According to this framework some usability dimensions can be defined in order to increase usability in mobile applications. With these dimensions, different usability guidelines are found to include these dimensions in a real application and the guidelines are validated in a case study.

(9)

2 LITERATURE REVIEW

2 Literature Review

In this section a usability framework will be presented in order to answer to the following two subquestions, based on an literature review.

1. Subquestion: Are there frameworks that can be used to define usability in mobile applications?

2. Subquestion: Are there practical guidelines that can be determined from these us-ability frameworks?

Based on this literature review, a lot of different usability frameworks are found. Abran et al. (2003) compare different models with the ISO standards, but denote that the stan-dards these model provide can be confusing. People outside the process can have valuable understanding on the measurement of usability, but fail to apply them. Therefore this model is too complex to use in practice. Harrison et al. (2013) conducted a new framework to capture the complexities of mobile applications. This framework was based on the ISO standards and the model proposed by Nielsen (1994), which describes the general aspects of usability.

Coursaris and Kim (2006) presented a mobile usability framework, conducted by a review of more than 100 empirical mobile usability studies. This framework is used in other research as well, like Baharuddin, Singh, and Razali (2013), and can therefore be defined as the most relevant framework for mobile usability. To stay in the scope of this study, only this framework will be presented in the next section. In addition, several usability guidelines are presented which are in concordance with the framework.

2.1 Usability framework

Coursaris and Kim (2006) suggested a usability framework to the context of a mobile envi-ronment. By using this framework they conducted a qualitative meta-analytical review of more than 100 empirical mobile usability studies in 2011, which was based on their research back in 2006 (Coursaris & Kim, 2011, 2006). Multiple databases were used to find literature about mobile usability from the year 2000 until 2010. They defined four contextual vari-ables that they used for the presentation of the founded research that were related to the usability of mobile applications or devices. Han, Yun, Kwahk, and Hong (2001) proposed these four variables as the principle components of human-computer interaction and these variables will be explained in section 2.1.1. Based on these four variables Coursaris and Kim (2006) proposed a usability framework for a mobile environment. This framework is shown in figure 2 and consists of three different sections.

(10)

2.1 Usability framework 2 LITERATURE REVIEW

Figure 2: The Mobile Usability Framework by Coursaris and Kim (2006)

The outer layer shows the four contextual variables that were used the review to 100 empirical mobile usability studies. The inner circle shows the usability dimensions that were found in the reviewed literature. Finally there is a box presented which describes the consequences of usability which they found in the reviewed literature as well. These consequences show the importance of usability in mobile development.

2.1.1 Contextual variables

According to the analysis of the collected literature, different kinds of usability dimensions were defined, which will be described in the next section. The four contextual variables were used to identify these usability dimensions. The four contextual variables that were used were:

User Demographics/culture

Knowlegde/experience/self-efficacy Perception/cognitions

Emotional/psychological context

Environment Physical like auditory, visual, location or experiment type Psychosocial or social conditions

Technology Device type

Interface - Input mode

Task/Activity Realism, open(user defines outcome) vs closed (pre-defined outcome or goal) Task descriptions (open/unstructured)

(11)

2.1 Usability framework 2 LITERATURE REVIEW

It is important to take context characteristics into consideration. These characteristics can impact usability, if one of these changes (Abran et al., 2003). For example, the envi-ronment of the user's office can impact usability, like noise or other distractions. According to Coursaris and Kim (2011) the limitations of the laboratory must be considered in order to measure usability in a relevant way.

The results of the analysis show that the mobile usability studies are mostly focused on investigation task characteristics (47%), followed by technology (46%), environment (14%) and user characteristics (14%). The results exceed 100%, because multiple areas were stud-ied. Notable is that most of the reviewed articles are focused on task and technology in contrast with user and environment. Nevertheless the human factors in usability are very important in informatics (Shackel & Richardson, 1991). Therefore it would be of greater value if the user itself is more involved in the Human-Computer Interaction. The experience of the user was studied the most in reviewed literature and none of the studies included the role of gender and age (Coursaris & Kim, 2011).

2.1.2 Usability dimensions

Several usability dimensions were defined by the review of the 100 empirical mobile usability studies in table 2. Notable in this table is that there are three main dimensions that can be used to measure usability which are also in accordance with the ISO-9241 standards mentioned before (Coursaris & Kim, 2011).

Measures Count % Efficiency 61 33 Effectiveness 49 27 Satisfaction 18 10 Accessibility 15 8 Learnability 8 4 Workload 7 4 Enjoyment 4 2 Acceptability 3 2 Quality 3 2 Security 3 2 Aesthetics 4 2 Utility 2 1 Memorability 2 1 Content 2 1 Flexibility 1 1 Playfulness 1 1

Table 2: Frequency of Usability Measures (Coursaris & Kim, 2011)

According to Abran et al. (2003), the measurement of usability consists of the same three variables. Nevertheless, in several studies learnability is one of the most important characteristics of usability (Abran et al., 2003; Rubin & Chisnell, 2008; Nielsen, 1994; Harrison et al., 2013). Rubin and Chisnell (2008) define learnability as the ease of using the application and how to use the interface, based on their previous experience. This is why in this thesis, four usability dimensions will be defined as followed:

(12)

2.2 Practical Usability Guidelines 2 LITERATURE REVIEW

• Efficiency: degree to which the product is enabling the performed tasks. How well are the users able to perform their tasks using the system?

• Effectiveness: degree to which the specified goals is achieved by the user. How accurate and complete can the goals be achieved?

• Satisfaction: degree to which the user is satisfied. How do the users feel about using of the system?

• Learnability: the time to learn the software in order to understand the system. Is the system easy to learn?

Based on these usability characteristics a new usability model can be defined. In figure 3 this model is presented, which shows the meaning of these four usability dimensions.

Figure 3: Usability model

2.2 Practical Usability Guidelines

In various studies, usability guidelines are defined based on literature reviews, but these guidelines were not actually used in real life applications. The suggested guidelines were solutions to different problems that occurred in mobile usability, but were not used in a case-study to actually validate them. These guidelines should ensure the increase of us-ability in mobile applications. Therefore several usus-ability guidelines, conducted from these literature studies were divided in the four usability dimensions as described above. Only guidelines that were relevant for the four usability dimensions and for mobile applications are included. The cohesion of these guidelines should increase the usability of mobile appli-cations. With distinction of the guidelines, the usability framework can be combined with practical usability measures. In this way the academic literature can be used in practice. In table 3 these practical usability guidelines are presented.

(13)

2.2 Practical Usability Guidelines 2 LITERATURE REVIEW

Dimensions Usability Guidelines

Efficiency - Enlarge interface elements to accommodate the fat finger syndrome

1_{(Nielsen & Budiu, 2013).}

- Smaller displays require larger buttons for gestural interaction (Hoober & Berkman, 2011).

- Enable frequent users to use shortcuts (Gong & Tarasewich, 2004). - Flexibility and efficiency of use (Nielsen, 2005).

Effectiveness - Users want simplicity, power, features and enough options according to their specific needs (Nielsen & Budiu, 2013).

- Less is more, information overload confuses the user (Seong, 2006). - Consider using personalisation to meet the user's individual needs (Häkkilä & Mäntyjärvi, 2006; Gong & Tarasewich, 2004).

Satisfaction - Use feedback to interact with the user (Hoober & Berkman, 2011). - Help users to recover from errors and suggest solutions (Seong, 2006; Nielsen, 2005).

- Offer informative feedback, which is understandable by the user (Gong & Tarasewich, 2004).

- Error prevention and simple error handling (Gong & Tarasewich, 2004; Nielsen, 2005).

- Users should be given the satisfaction of accomplishment and comple-tion (Gong & Tarasewich, 2004).

Learnability - The best learning method is by doing, not memorisation (Nielsen & Budiu, 2013).

- Consistency is one of the most essential factors in interface design (Seong, 2006; Gong & Tarasewich, 2004).

- Reduce short-term memory, very little memorisation should be required (Gong & Tarasewich, 2004; Nielsen, 2005; Seong, 2006).

- Avoid information overflow, do not use irrelevant information (Häkkilä & Mäntyjärvi, 2006; Nielsen, 2005; Seong, 2006).

Table 3: Practical usability guidelines in conjunction with the four usability dimensions.

Based on this literature review, there is a possibility to measure usability in mobile applications. By combining academic literature with usability guidelines, usability can be measured in practice. One framework was presented with a review of more than 100 em-pirical mobile usability studies. From this analysis four contextual variables were defined in order to identify several usability dimensions. Efficiency, effectiveness, satisfaction and learnability were determined as the four most important dimensions for measuring usability in mobile applications. These dimensions were used to classify different usability guide-lines gathered from distinct studies. Only the guideguide-lines that were in accordance with the usability dimensions were used.

In the next section these guidelines will be validated in a real application with the use of a case study. First, an analysis will be done, to see which of these guidelines are already included in the application. Second, a recommendation application will be created based on this analysis. Finally, these two application will be examined and compared with each other.

1_{The fat finger syndrome = the failure of tapping areas that are smaller than your fingers (Nielsen &} Budiu, 2013).

(14)

3 RESEARCH METHOD

3 Research Method

In the literature review a theoretical basis is found for increasing mobile usability. In this study a post-test-only experiment will be done to validate the usability guidelines in a real mobile application. In this way an answer will be found to:

3. Subquestion: Can the relevance of the founded practical guidelines be validated?

First an analysis will be done to determine which of the defined guidelines are included in the chosen application. Next, for each dimension, one guideline will be chosen and a related solution will be added to the mobile application. In this way two different applications are designed and will be compared with each other. For both applications a questionnaire is distributed in order to measure the usability of the application. The defined questions are also divided into the four usability dimensions in order to validate the practical usability model.

3.1 The Mobile Application

The mobile application that is used in this study is a camera surveillance solution. This application is about a surveillance camera used to protect your belongings. You are able to see the video images live from your smartphone, playback recordings and get notified when there is motion detected. This application is still in development and therefore it is of great value to see if the usability guidelines can increase the usability of this application during the development process. Therefore the application is limited to one feature and only this feature will be examined in the experiment. The camera has four different modes and each mode has the following settings:

• Armed: Record events, live streaming and get notified • Mute: Record events and live streaming

• Disarmed: Only live streaming • Private: All turned off

When the application is opened for the first time two welcome screens are shown with basic information about the armed/disarmed feature. These screens are shown in figure 4. The application has four different interfaces for each mode. These interfaces are shown in figure 5 and they will change according to the pressed button in the screen. When pressing the purple lock button, the camera will change to the red disarmed screen and the button will change to an unlocked button. When pressing this unlocked button the screen will change back to the purple armed screen.

(15)

3.1 The Mobile Application 3 RESEARCH METHOD

Figure 4: Welcome screens

(16)

3.2 The Mobile Application Analysis 3 RESEARCH METHOD

3.2 The Mobile Application Analysis

In table 4, for each dimension, the practical usability guidelines are shown that are included in the mobile application.

Dimensions Usability Guidelines Included? Efficiency - Enlarge interface elements to accommodate the fat finger

syndrome2_{(Nielsen & Budiu, 2013).}

No

- Smaller displays require larger buttons for gestural inter-action (Hoober & Berkman, 2011).

No

- Enable frequent users to use shortcuts (Gong & Tarasewich, 2004).

No

- Flexibility and efficiency of use (Nielsen, 2005). No Effectiveness - Users want simplicity, power, features and enough options

according to their specific needs (Nielsen & Budiu, 2013). No

- Less is more, information overload confuses the user (Seong, 2006).

Yes

- Consider using personalisation to meet the user's in-dividual needs (Häkkilä & Mäntyjärvi, 2006; Gong & Tarasewich, 2004).

No

Satisfaction - Use feedback to interact with the user (Hoober & Berk-man, 2011).

No

- Help users to recover from errors and suggest solutions (Seong, 2006; Nielsen, 2005).

No

- Offer informative feedback, which is understandable by the user (Gong & Tarasewich, 2004).

No

- Error prevention and simple error handling (Gong & Tarasewich, 2004; Nielsen, 2005).

No

- Users should be given the satisfaction of accomplishment and completion (Gong & Tarasewich, 2004).

No

Learnability - The best learning method is by doing, not memorisation (Nielsen & Budiu, 2013).

No

- Consistency is one of the most essential factors in interface design (Seong, 2006; Gong & Tarasewich, 2004).

Yes

- Reduce short-term memory, very little memorisation should be required (Gong & Tarasewich, 2004; Nielsen, 2005; Seong, 2006).

No

- Avoid information overload, do not use irrelevant infor-mation (Häkkilä & Mäntyjärvi, 2006; Nielsen, 2005; Seong, 2006).

Yes

Table 4: Analysis of the Mobile Application

Notable from this analysis is that only a few of the guidelines are already included in the application. This means that most of the usability guidelines can be applied to the mobile application.

2_{The fat finger syndrome = the failure of tapping areas that are smaller than your fingers (Nielsen &} Budiu, 2013).

(17)

3.3 The Increased Usability Application 3 RESEARCH METHOD

3.3 The Increased Usability Application

For each of the four usability dimensions one solution will be applied according to the analysis done in the previous section. This solution will be applied in a new version of the mobile application added to the one that already exists. Each solution is based on one of the missing guidelines. In table 5 the used guidelines are presented together with the applied solutions in the application. These solutions were chosen, because they were applicable to the application, although the application is still in development. In this study the solutions are limited to one solution for each dimension, because otherwise the boundaries of the scope will be exceeded. Next to that, if too many changes are implemented to the application, the usability model cannot be validated due to too many changes. It is hard to measure what caused the increase or decrease of the usability if too many solutions are applied. This means only the four chosen guidelines can be validated or rejected.

Dimensions Usability Guideline Usability solution Efficiency Enlarge interface elements to

ac-commodate the fat finger prob-lem.

Enlarge small buttons.

Effectiveness Users want simplicity, power, features and enough options ac-cording to their specific needs.

Add information button.

Satisfaction Offer informative feedback, which is understandable by the user.

Add text of functionalities inside the screen.

Learnability The best learning method is by doing, not memorisation.

Notification which makes the user per-form a task to learn the feature.

Table 5: Solutions to the usability problems found in the analysis

Figure 6: Applied solutions

The applied changes are shown in figure 6. The buttons of the private and mute func-tionalities are enlarged and an information button is added to the screen. When clicking on

(18)

3.4 Sample 3 RESEARCH METHOD

the information button the overview of the functionalities is shown, just like in the welcome screen that is used in figure 4. In this way the user can choose enough options according to their specific needs and the application will keep its simplicity. Below the lock/unlock button a text is added, which shows the status of the camera according to the current mode. For example, when the camera is set to disarmed, the text ”Live view” is shown. This means, in this mode, only live streaming is possible and the user receives informative feedback which is understandable by the user. Finally, a notification is added when the user opens the application for the first time with the text ”Now first disarm you camera by clicking on the lock button”. This notification lets the user perform a task in order to learn the feature instead of memorising what the feature was. In the current version of the application the user should memorise the meaning of the functionalities from the welcome screen and is not able to recall this later. In the appendix all the changes to the application are included in screenshots.

3.4 Sample

In this thesis a post-test-only randomised experiment is chosen, because this kind of experi-ment is often used to randomly compare two different groups were one group has a different variable and than the second group (Campbell, Stanley, & Gage, 1963). In this case the guidelines that are applied are the independent variables and these are compared with a control group. In order to execute the experiment, a quantitative research method is used for this study, carried out with an online questionnaire. This research method was chosen, because it was important that the respondents were in their own environment. Coursaris and Kim (2006) already remarked the limitations of laboratory research, so this is why there is chosen for an online questionnaire. Both applications as described above were examined, in order to validate the usability guidelines. The experiment was based on previous research done by Sandberg, Maris, and de Geus (2011). They used a quasi-experiment for mobile English learning with children in the fifth grade with two different test groups and also ques-tionnaires were used to measure the differences. This is similar to this case-study, in which the usability is measured in two different groups with questionnaires, but only a post-test will be done in this thesis. Therefore the following hypothesis can be defined:

Hypothesis: The increased usability application will be more usable than the standard application.

3.4.1 Respondents

In this research 96 respondents were questioned, divided into two groups. One group was used to measure the usability of the standard mobile application, defined as the control group, and the other group was used to measure the usability of the increased usability application, defined as the treatment group. All respondents were unique and had never seen the application before. Two different groups were chosen, because of the usability dimension learnability. If the respondent was first asked to review the increased usability application and the standard application afterwards, he or she could have already learned how the application works and was therefore not reliable anymore. Miles and Banyard (2007) defines this as practice effects. These occur when a respondent gets better at a task over time.

(19)

3.4.2 Materials

For both applications a prototype was used, created with InVision (InVision, 2015). This is a prototyping tool for mobile devices and uses screenshots to create a prototype of the real application. The prototype applications could be opened from every kind of smartphone, even the operating system didn’t matter, with an external link. The screenshots that were used for the prototype were created with an iPhone 6.

3.4.3 Questionnaire

Several questions were obtained from online usability test questionnaires, which are used in practice (Ryu & Smith-Jackson, 2006; Garcia, 2013; Borysowich, 2007). The questions are also divided into the four usability dimensions and the contextual variables in order to validate the usability framework. For each dimension two questions are used to measure the usability for each dimension. In table 3.4.3 these questions are summarized. Next to the questions presented in this table, some demographic questions were asked and for each mode, described in section 3.2, one question was asked. This question was about if they understood what happened by, for example, arming the camera. The respondents were asked to fill in the online questionnaire in Google Forms. The questionnaire is included in the appendix.

Category Question

User What is your age? What is your gender?

Do you have any knowledge about surveillance cameras?

Environment What is your current location?

Technology What mobile operating system are you currently using? How often do you use your smartphone?

Task/Activity Do you think you will be using the product?

Efficiency I was able to achieve my goals by using the application It was easy to use the application

Effectiveness I feel like I successfully completed all my tasks It was easy to find the information I needed

Satisfaction I was satisfied using the application I found the application frustrating to use

Learnability The application is easy to learn

I immediately understood the functionality of the buttons

All of the questions were measured with a interval Likert scale of 1 until 10. An interval scale is used to rank categories with an equal distance between two points, the Likert scale

(20)

is most commonly used (Heinhuis, 2013). An interval of 10 points was used, because the user is forced to choose and cannot choose the middle of the interval.

3.4.4 Procedure

The respondents were asked to fill in the questionnaire on their laptop, tablet or desktop computer and use their smartphone next to it. During the questionnaire the respondents were asked to open a link on their mobile devices by, for example, emailing it to themselves. Next a scenario of the application was described and it was explained that the application was still in development and only the armed/disarmed feature was working. Respondents were asked to take this into consideration when answering the usability questions. The user was asked to perform several tasks inside the installed application and filled in the ques-tionnaire.

In the next section the results of the research will be presented. According to G. Norman (2010), parametric statistics can be used with Likert scale data, with small sample sizes. Therefore an independent Samples T Test is done for comparing the increased usability application with the standard application.

(21)

4 RESULTS

4 Results

The results are divided into two different categories. First the results of the usability dimen-sions are presented together with the total usability score. Next the contextual variables and demographic data are displayed. By analysing the results, an answer can be found to the research question:

Research question: Is it possible to develop a framework that can increase the us-ability of mobile applications during the development process?

An independent Samples T Test is used to analyse the results of the usability dimensions. Therefore the following null hypothesis and alternative hypotheses are defined:

H0: There is no difference between the increased usability application and the standard application in the average usability score3_.

HA: There is a difference between the increased usability application and the standard application in the average usability score.

4.1 Usability dimensions

For each dimension, two questions were asked to the respondents as shown in table 3.4.3. To measure each dimension, the average score of these two questions were calculated. For the total usability score, an average was taken from the four usability dimensions. An independent Samples T Test was used to compare the means of these dimensions, together with the total usability score. The results of this test are shown in figure 7 and in figure 8. The increased usability application is defined as version 1.0 and the standard application is defined as version .0.

Figure 7: Group Statistics per dimension and total usability

The results of the group statistics show that for all the dimensions a difference is found. For each dimensions the increased usability application has higher mean values than the standard application. Therefore the increased usability application has a higher total us-ability score compared to the standard application.

3_{The average usability score consists of the sum of the four average usability dimension scores, that were} measured with the questionnaire.

(22)

4.1 Usability dimensions 4 RESULTS

Figure 8: Independent Sample T Test per dimension and total usability

Nevertheless, only the dimension Effectiveness has a significant difference between the increased usability application and the standard application. All the other dimensions do not have a significant difference and therefore the total usability score is also not significant. The results were not sufficiently convincing in order to reject the H0 and to adopt the HA. There is no significant difference between the increased usability application and the standard application in the average total usability score.

Figure 9: Means per dimension and total usability

In figure 9 the means of the increased usability applications are compared with the means of the standard application. Although the results indicate that there is no significant difference between the two versions, the average score of the dimensions and total usability score show that the increased usability application has higher scores on any field.

(23)

4.2 Contextual variables and demographic data 4 RESULTS

4.2 Contextual variables and demographic data

The results of the contextual variables, defined in section 2.1.1, are presented in this section. In figure 10, several demographic data and contextual variables are shown. The first chart represents the division between the mobile operating systems the respondents are using on their personal smartphone. Also the smartphone usage is shown in the second chart, which is divided into four different categories. These categories were based on the average mobile usage in hours per day (Sale, 2014). According to these results, almost all of the respondents were frequent smartphone users. Finally, in the third chart, the division between male (43%) and female (57%) is shown.

Figure 10: Piecharts of the contextual variables and demographic data

The average age of the respondents was 27 years old and 79 of the respondents had previous knowledge about camera surveillance. Most of the respondents (63) were at home or at school (24). Nevertheless only 15 respondents thought they would be using the product by themselves. For the contextual variables another Independent Samples T Test was done in order to check if there is a significant difference in gender, smartphone usage, previous knowledge and operating system. There was no significant difference found for male and female groups. Next to that the smartphone usage was divided into three different groups, low (<2 hours per day), medium (3-4 hours per day) and high (5>hours per day). In these groups no significant differences were found either. Another T Test was performed based on the mobile operating system, but in this case also no significant difference was found. Finally, the previous knowledge about camera surveillance was used as a grouping variable. Compared to the other results, a significant difference was found in the total usability score of the respondents who had previous knowledge about camera surveillance. However, the sample size of the groups who did have previous knowledge was very small.

(24)

4.3 Conclusion 4 RESULTS

4.3 Conclusion

In this thesis a framework was found which could measure the usability of mobile applica-tions. This framework was used to find practical usability guidelines that could be imple-mented in a mobile application. These guidelines were divided in four usability dimensions. In addition, an analysis was done with a real mobile application to see if these guidelines could be validated. For each dimension, one guideline was chosen and for this guideline a solution was applied to the existing application. A new version of the application was created and was tested with a questionnaire. The existing application was tested likewise.

The results of this research indicate that the dimension Effectiveness has a significant difference between the increased usability application and the standard application. There-fore the guideline that correlates with this solution can be validated and can be used to increase the usability of a mobile application in this area. Nevertheless this can be only stated for this dimension. The guideline that is validated by this research is: Users want simplicity, power, features and enough options according to their specific needs.

The other guidelines cannot be validated which were in conjunction with the other three dimensions: learnability, satisfaction and efficiency. Therefore these three guidelines could not be used to increase the usability of mobile applications. This means for these dimensions the framework could not be used to increase the usability of mobile applications during the development process. The results were not significant enough in this thesis in order to prove that statement. Nevertheless, the other guidelines found with the literature review can be validated with further research and this will be discussed in the next section.

(25)

5 DISCUSSION

5 Discussion

Notable from the literature review was that the dimension satisfaction can be disputed. Some people say satisfaction is the outcome of usability and can therefore not be used to measure usability. In section 1, Weiss (2003) denoted usability as the ease of use and Delone and McLean (2003) believe system usage causes success. They say that higher system quality will lead to higher user satisfaction. Seddon (1997) claims that use is a behaviour and it precedes instead of causing the success. Nevertheless satisfaction is included in the ISO standards of usability and has an important role. This is why it is important to include satisfaction in the usability framework but contradicts what other researcher think.

In contrast to the conclusion of this research, a valuable difference was found between the increased usability application and the standard application. Although these results were not significant, all of the mean values were higher for the increased usability application. This is an interesting observation for further research.

5.1 Limitations

5.1.1 G*Power Analysis

Because there are no significant differences for most of the dimensions and the total usability, a power analysis is done. The power of a test is the probability of rejecting the null hypothesis when the specific alternative hypothesis is true (Bruin, 2011). In this case a Post Hoc Power Analysis is chosen, because the study already has been done. This makes it possible to estimate if the test has a fair chance of rejecting a incorrect null hypothesis (Faul, Erdfelder, Lang, & Buchner, 2007). This can be determined from the effect size and power. These values are calculated with the G*Power Analysis and are presented in table 6 (Bruin, 2011).

Value Effect size Power Usability 0,3101168 0.4456 Efficiency 0,2727024 0,3750 Effectiveness 0,4195581 0,9505 Learnability 0,225035 0,2910 Satisfaction 0,210481 0,2673

Table 6: Power analysis for insignificant differences.

The Cohen’s d effect size is used and he defines 0.2, 0.5 and 0.8 as small, medium and large effects separately (Faul et al., 2007). Generally a power of 0.8 or higher is acceptable (Picardi & Masick, 2013). When the power is above 0.8, the researcher has a greater likelihood to find a significant difference in the experiment. Resulting from table 6, the effect sizes of the dimensions are very low and the effect of the differences will be small. Next to that, a priori power analysis was done to compute the required sample size for the total usability and the corresponding effect size. A total sample size of 452 respondents was estimated. This means, assuming the same differences between the two groups, with this sample size a significant difference will be found in the total usability. However, it will keep the same low effect size.

(26)

5.2 Future work 5 DISCUSSION

5.2 Future work

Although no significant difference was found in the total usability score, it indicates this area can be explored further. Only four usability guidelines are tested and with a greater sample size, significant differences can be found as described in the previous section. This estimated sample size was too big for the scope of this thesis, but can be used to examine other usability guidelines. Besides, it was not achievable to test all the guidelines, and multiple case studies should be done to validate all the guidelines. If too many changes are made to one application it is unclear what exactly increased the usability, or even it could decrease the usability. However there were some guidelines already in the existing application, but these could not be tested because these guidelines could not be compared with another application.

It is important to have further research in practical usability measurement. Besides, one significant difference was found in the group of respondents that had previous knowledge about camera surveillance. This means, if a different target audience is chosen, there might be better results. There is a greater chance of finding significant results, if the target audience has previous knowledge about surveillance cameras. Lots of the literature found are literature studies and most of the guidelines found were not actually tested in practice. Although this research did not have a significant difference, it shows that better results can be achieved by implementing usability guidelines inside the development process of mobile applications. Hopefully this will lead to further deployment and usability testing.

(27)

References References

References

Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpreta-tions in iso standards. Software Quality Journal , 11 (4), 325–338.

Baharuddin, R., Singh, D., & Razali, R. (2013). Usability dimensions for mobile applica-tionsa review. Res. J. Appl. Sci. Eng. Technol , 5 , 2225–2231.

Borysowich, C. (2007, July). Sample application usability test questionnaire. (Last checked 8 June 2015: http://it.toolbox.com/blogs/enterprise-solutions/sample-application-usability-test-questionnaire-17827)

Bosomworth, D. (2015, January). Mobile marketing statistics 2015: Statistics on mo-bile usage and adoption to inform your momo-bile marketing strategy. (Last checked 28 April 2015: http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/)

Bruin, J. (2011, Februari). G*power data analysis examples: Power anal-ysis for two-group independent sample t-test. (Last checked 4 July 2015:http://www.ats.ucla.edu/stat/gpower/indepsamps.htm)

Campbell, D. T., Stanley, J. C., & Gage, N. L. (1963). Experimental and quasi-experimental designs for research (Tech. Rep.). Houghton Mifflin Boston.

Campbell-Kelly, M., Aspray, W., Ensmenger, N., & Yost, F. (2009). Computer. Westview Press.

Carroll, J. M. (1997). Human-computer interaction: psychology as a science of design. Annual review of psychology , 48 (1), 61–83.

Chaffey, D. (2015, January). Digital marketing statistics 2015:the top 10 free sources for global, european, uk, us, asia and latin america digital marketing statis-tics. (Last checked 15 May 2015: http://www.smartinsights.com/marketplace-analysis/customer-analysis/digital-marketing-statistics-sources/)

Coursaris, C., & Kim, D. (2006). A qualitative review of empirical mobile usability studies. AMCIS 2006 Proceedings, 352.

Coursaris, C., & Kim, D. (2011). A meta-analytical review of empirical mobile usability studies. Journal of usability studies, 6 (3), 117–171.

Delone, W. H., & McLean, E. R. (2003). The delone and mclean model of information systems success: a ten-year update. Journal of management information systems, 19 (4), 9–30.

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G* power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39 (2), 175–191.

Garcia, A. (2013, November). Ux research — standardized usability questionnaires. (Last checked 8 June 2015: http://chaione.com/ux-research-standardizing-usability-questionnaires/)

Gong, J., & Tarasewich, P. (2004). Guidelines for handheld mobile device interface design. In Proceedings of dsi 2004 annual meeting (pp. 3751–3756).

Gould, J. D., & Lewis, C. (1985). Designing for usability: key principles and what designers think. Communications of the ACM , 28 (3), 300–311.

Häkkilä, J., & Mäntyjärvi, J. (2006). Developing design guidelines for context-aware mobile applications. In Proceedings of the 3rd international conference on mobile technology, applications & systems (p. 24).

Han, S. H., Yun, M. H., Kwahk, J., & Hong, S. W. (2001). Usability of consumer electronic products. International Journal of Industrial Ergonomics, 28 (3), 143–151.

(28)

Harrison, R., Flood, D., & Duce, D. (2013). Usability of mobile applications: literature review and rationale for a new usability model. Journal of Interaction Science, 1 (1), 1–16.

Heinhuis, D. (2013). Modeling customer behavior in multichannel service distribution: A rational approach.

Hoober, S., & Berkman, E. (2011). Designing mobile interfaces. ” O’Reilly Media, Inc.”. InVision. (2015). Free web & mobile prototyping. (Last checked 5 June:

http://www.invisionapp.com/)

Juristo, N., Moreno, A. M., & Sanchez-Segura, M.-I. (2007). Analysing the impact of usability on software design. Journal of Systems and Software, 80 (9), 1506–1516. Landauer, T. K. (1995). The trouble with computers: Usefulness, usability, and productivity

(Vol. 21). Taylor & Francis.

Microsoft. (2015). Ms-dos overview. (Last checked 7 May 2015 https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/) Miles, J., & Banyard, P. (2007). Understanding and using statistics in psychology: A

practical introduction. Sage.

Nielsen, J. (1994). Usability engineering. Elsevier.

Nielsen, J. (2005). Ten usability heuristics. http://www. nngroup. com/articles/ten-usability-heuristics/(acc-essed December 19, 2013).

Nielsen, J., & Budiu, R. (2013). Mobile usability. MITP-Verlags GmbH & Co. KG. Norman, D. A. (1986). Cognitive engineering. Hillsdale, NJ: Lawrence Erlbaum.

Norman, G. (2010). Likert scales, levels of measurement and the laws of statistics. Advances in health sciences education, 15 (5), 625–632.

Picardi, C. A., & Masick, K. D. (2013). Research methods: Designing and conducting research with a real-world focus. SAGE Publications.

Richter, F. (2015, June). Apple’s app store is growing by 1,000+ apps a day. (Last checked 10 June 2015: http://www.statista.com/chart/3530/app-store-growth/)

Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley & Sons.

Ryu, Y. S., & Smith-Jackson, T. L. (2006). Reliability and validity of the mobile phone usability questionnaire (mpuq).

Sale, S. (2014, May). Consumers use smartphones for 195 minutes per day, but spend only 25% of that time on communications. (Last checked 8 June: http://www.analysysmason.com/About-Us/News/Insight/consumers-smartphone-usage-May2014-RDMV0/)

Sandberg, J., Maris, M., & de Geus, K. (2011). Mobile english learning: An evidence-based study with fifth graders. Computers & Education, 57 (1), 1334–1347.

Seddon, P. B. (1997). A respecification and extension of the delone and mclean model of is success. Information systems research, 8 (3), 240–253.

Seong, D. S. K. (2006). Usability guidelines for designing mobile learning portals. In Proceedings of the 3rd international conference on mobile technology, applications & systems (p. 25).

Shackel, B., & Richardson, S. J. (1991). Human factors for informatics usability. Cambridge University Press.

Soegaard, M. (2012, Mei). The history of usability: From simplicity to complexity. (Last checked 4 May 2015 www.smashingmagazine.com/2012/05/23/the-history-of-usability-from-simplicity-to-complexity/)

(29)

Statista. (2015a, -). Number of mobile app downloads worldwide from 2009 to 2017 (in millions). (Last checked 7 May 2015: http://www.statista.com/statistics/266488/forecast-of-mobile-app-downloads/) Statista. (2015b). Number of smartphone users worldwide from 2012 to 2018. (Last checked 1

May 2015: http://www.statista.com/statistics/330695/number-of-smartphone-users-worldwide/)

Tunguz, T. (2014). 4 major competitive trends in mobile app stores. (Last checked 7 May 2015: http://tomtunguz.com/app-store-competition/)

(30)

A APPENDIX

A

Appendix

(31)

(32)

(33)

(34)

A.2 Application screenshots A APPENDIX

(35)

Guidelines for increasing the Usability of Mobile Applications