Predicting the learnability of interactive health technology interfaces based on cognitive load theory

(1)

PREDICTING THE LEARNABILITY OF INTERACTIVE HEALTH

TECHNOLOGY INTERFACES BASED ON COGNITIVE LOAD

THEORY

(2)

(3)

3

Predicting the learnability of Interactive Health Technology interfaces based on

Cognitive Load Theory

Master Dissertation Medical Informatics

Student

F.H. Ruiter BSc. Student number – 10338101 E-mail: f.h.ruiter@amc.uva.nl

Mentor

A. Blankendaal

Team leader Research and Development, Financial department ChipSoft

Tutor

Dr. L.W.P. Dusseljee-Peute Assistant professor

Amsterdam University Medical Centers (UMC) Location Academic Medical Center (AMC) Dept. of Medical Informatics

Traineeship address

ChipSoft

Dept. Research and Development Orlyplein 10

1043 DP Amsterdam The Netherlands

Traineeship period

(4)

(5)

5

ACKNOWLEDGEMENTS

Performing a scientific research project and writing a thesis does no go without some struggles and demotivation. However, some guidance, support and motivation helped me to finish my master thesis. Therefore, I would like to express my gratitude to all who helped me finish my master thesis.

In particular I would like to Linda for her guidance, feedback, and provided insights both in the educational and social context. Again, it has been a pleasure working with you, and thank you for challenging me now, and the possible future. From ChipSoft; I want to thank Alex Blankendaal for his adequacy and sympathy, and Lucienne Temperilli-Klein for the help in contacting and recruiting the participants.

Thank you, papa and mama for the financial support over the past five years, and for being a sympathetic ear. Even when half of the times you think I speak Russian, and understand half of what I am telling, it has been a great support knowing that you are very proud of my accomplishments. In addition, an honorable mention for oma; she would have been the proudest of all. Unfortunately, she passed away during the internship period of my bachelor thesis. I would have wanted that you could see what I have become. Nawa and Sicco, thank you for just being you. For the past five years we shared the good and the bad. Had a lot of laughs, some tears, enormous amount of study stress, and made a lot of great memories together. Could not have survived university life without you two sweethearts.

(6)

(7)

7

SUMMARY

Introduction: Hospitals and care organizations implement Health Information Technologies (HIT) to improve care effectiveness and

efficiency [1]. Educating/training the end-users to use the HIT are a key factor for the successful implementation [1, 2]. By increasing the learnability of a HIT training costs can be reduced. According to the cognitive load theory (CLT) learning can be optimized by reducing the extraneous cognitive load [11]. In the CLT the cognitive load is due to instructional design and in the field of Human Computer Interaction (HCI) cognitive load is induced by the using a software tool (or HIT) [6]. However, it in HCI it is unclear if the learnability of a HIT increases when the cognitive load is minimized, and what the relation is to its usability. The aim is to provide insight in the current cognitive load theories and measurement methods, and to assess the cognitive load of two HITs and the impact on the usability.

Methods: A scoping review is performed guided by the framework of Levac et al. (2010). Based on the scoping review a theoretical

framework on the current cognitive load theories, provided definitions and described and/or used measurement methods is developed, with an emphasis on the cognitive load in the context of an interactive HIT and how to assess its usability. The developed theoretical framework informed the methods/materials selection and the study design of the main study; a cohort study. The perceived and measured mental workload and usability of two Hospital Information Systems (HIS); Ezis and HiX is assessed. Participants had to perform three identical task-based scenarios is both systems. The mental workload is assessed using the Single Ease Question (SEQ) for perceived task difficulty, task competition time, and the NASA – Task Load Index (TLX) for the perceived mental workload. The task satisfaction is assessed using the After-Scenario Questionnaire (ASQ) and the task efficiency is measured by the task completion time and number of mouse clicks. The satisfaction of the interface quality is assessed using the Post Study System Usability Questionnaire (PSSUQ). In addition, a retrospective think-aloud protocol is performed, and the general scores for the systems are collected. The mental work of both systems is analyzed in relation to the measured usability, taken the specific system experience into consideration.

Results: In the scoping review thirteen different theories are identified; ten of the theories have and interrelationship with the working

memory model. The terminologies cognitive load and mental workload are mostly commonly used by the authors. The terminologies are arbitrary and the terminology used is mostly determined by the research discipline. A plethora of definitions exists for the used terminologies and are based on an underlying theory. For the field of HCI an comprehensive definition is provided. Three distinct cognitive load measurement method categories are identified; subjective, performance, and physiological. For the mental workload assessment of an IHT system it is recommended to use a combination of a subjective and performance measure. There are no clear recommendations on the use of a usability method. The results of the main study suggest that when the task is perceived as less difficult in comparison to the previous task, the usability satisfaction increases. In addition, overall scored HiX better in comparison to Ezis in both specific system experience groups (Ezis and HiX experienced, and HiX experienced). The discrepancy in scores and the scores for both systems are higher for the HiX experienced group.

Discussion: An overview on current cognitive load theories, definitions, and measurement methods is provided. The theoretical

framework and the main study provide a guideline for the assessment of mental workload and usability of a HIT. Future research should validate the provided guideline and results of the main study. In addition, besides system specific experience other user characteristics should be taken into consideration in the data analyzation. Based on the identified factors/features a tool can be developed that determines the mental workload induced by a HIT. The tool can be used to predict the learnability of a HIT. Since, when the mental workload is decreased the learning process can be optimized.

(10)

10

SAMENVATTING

Introductie: Ziekenhuizen en zorgorganisaties implementeren zorg informatiesystemen (ZIS) om de effectiviteit en efficiëntie van

de zorg te verbeteren [1]. Het opleiden en trainen van eindgebruikers in het gebruik van een ZIS is een belangrijke factor voor een succesvolle implementatie [1,2]. Door het bevorderen van de leerbaarheid van een ZIS kunnen de trainingskosten afnemen. Volgens de cognitieve load theorie (CLT) kan het leren werken met een nieuw systeem geoptimaliseerd worden door het verminderden van irrelevante cognitieve belasting [11]. In de CLT wordt de cognitieve belasting gekenmerkt tijdens het leren of tijdens het uitvoeren van (leer)activiteiten en in het domein van Humane Factoren in mens-systeem engineering (HFE) wordt de cognitieve belasting veroorzaakt door het gebruik van een software tool (of ZIS). Het is echter onduidelijk of de leerbaarheid van een ZIS toeneemt als de cognitieve belasting geminimaliseerd wordt en wat de relatie is van de cognitieve belasting tot de ervaren bruikbaarheid van een systeem. Het doel van deze scriptie is om inzicht te bieden in de huidige cognitieve belasting theorieën en meetmethoden voor het opstellen van een onderzoeksdesign om de cognitieve belasting van twee ZIS te vergelijken en de impact van de cognitieve belasting op de bruikbaarheid te bepalen.

Methoden: Een kader onderzoek is uitgevoerd gestructureerd aan de hand van Levac e.a. (2010). Gebaseerd op het kader onder

is een theoretisch raamwerk over de huidige cognitieve belasting theorieën, verstrekte definities en beschreven en/of toegepaste meetmethodes, uitgewerkt. De nadruk ligt hierin op de cognitieve belasting in de context van een interactieve ZIS en hoe de bruikbaarheid daarvan bepaald kan worden. Het uitgewerkte theoretische raamwerk heeft geleid tot een studieontwerp van de hoofdstudie; een cohortstudie. De ervaren en gemeten mentale belasting en de bruikbaarheid is bepaald voor twee Ziekenhuis Informatie Systems; Ezis en HiX. Deelnemers moesten drie identieke taak gebaseerde taken uitvoeren in beide systemen. De mentale belasting is bepaald door het toepassen van de Single Ease Question (SEQ) voor de ervaren moeilijkheid van de taak, de voltooiingstijd van de taak en de NASA - Taak Belasting Index (TBI) voor de ervaren mentale werkbelasting. De taak-tevredenheid is bepaald door het gebruik van de After Scenario Questionaire (ASQ) en de efficiëntie van de taak is gemeten door de voltooiingstijd en het aantal muisklikken. De tevredenheid over de interface kwaliteit is bepaald aan de hand van de Post Study System Usability Questionnaire (PSSUQ). Als toevoeging is een retrospectieve hardop denken protocol uitgevoerd en zijn de algemene scores voor de systemen verzameld. De mentale belasting is geanalyseerd in relatie tot de gemeten bruikbaarheid, waarbij eerdere systeem ervaring in de analyse is meegenomen.

Resultaten: In het kader onderzoek zijn dertien verschillende theorieën geïdentificeerd; tien van deze theorieën hebben een

onderliggend verband met het werkend geheugen model. De terminologieën cognitieve belasting en mentale belasting wordt het meest gebruikt bij de auteurs. De terminologieën zijn arbitrair en gebruik van een terminologie is meestal bepaald door de onderzoeksdiscipline. Status quo is er een overvloed aan definities voor de gebruikte terminologieën welke zijn gebaseerd op de onderliggende theorie. Voor HFE is in deze studie een omvattende definitie gegeven. Drie onderscheidende cognitieve belasting categorieën voor meetmethoden zijn geïdentificeerd; subjectief, uitvoering, en fysieke meetmaten. Voor de bepaling van de mentale belasting van een ZIS wordt de toepassing van een combinatie van subjectieve en uitvoeringsmaten aangeraden. Er zijn geen duidelijke aanbevelingen voor een bepalingsmethode van de bruikbaarheid. De resultaten van de hoofdstudie suggereren dat wanneer de ervaren moeilijkheid van een taak toeneemt de tevredenheid afneemt. Voor alle uitkomsten scoorde HiX in deze studie beter in vergelijking tot Ezis voor beide specifieke groepen (Ervaren met Ezis en HiX, en ervaren met HiX). De discrepantie in scores en de scores voor beide systemen zijn hoger voor de ervaren HiX groep.

Discussie: Deze studie biedt een vernieuwend overzicht van huidige cognitieve belasting theorieën, definities en meetmethodes.

Het theoretisch raamwerk en de hoofdstudie voorzien in een opzet voor een studie voor het bepalen van de mentale werkbelasting en bruikbaarheid van een ZIS. In toekomstig onderzoek dienen de resultaten en de voorziene studieopzet gevalideerd te worden. Verder moeten er naast de specifieke systeem ervaring andere gebruikers karakteristieken in overweging genomen worden in de analyse van de data. Gebaseerd op de geïdentificeerd factoren/eigenschappen kan een tool ontwikkeld worden, dat de geïnduceerde mentale werkbelasting van een ZIS bepaald. Een dergelijke tool kan gebruikt worden om de leerbaarheid van een ZIS te voorspellen.

(11)

11

1 INTRODUCTION

In the last decade hospitals and care organizations have started implementing Health Information Technologies (HIT) on a wide scale to improve care effectiveness and efficiency [1]. Successful implementations, however, have been a challenge. Research on failed implementations have shown that a variety of key factors need to be considered during implementation [2]. A key factor is educating/training the end-users to use the information technology after implementation [1, 2] and its relation to this systems’ usability. Where usability is viewed as the ease of which a system can be used in practice, and its match with the user needs [1]. Training end-users can maximize user’s performance with a system [1], which indirectly increases user acceptance and satisfaction [1, 3]. However, training costs and tight implementation budgets often result in limited training prior to the actual usage of the information technology [1, 4]. In addition, continuous user training and adequate support overtime remains necessary to further learn to use the system and extend the user’s knowledge and effective use of the system [1]. However, if a health information technology, and foremost its interface is intuitive to use, and therefore learning to work with a system required minimal effort, then less support and training is needed to learn to use the system. But when is an interactive health technology interface intuitive and easy to learn?

There are several factors that could make learning a system easier or harder. It is easier to learn new information when there is already ‘some’ knowledge present. Such as in the case of a health information system where a physician has earlier experience with a similar system [5]. On the other hand, learning can be more difficult when new information conflicts with the already existing knowledge, such as a completely different navigation structure of a similar system [5]. Overall, when new information is clearly related to existing ‘information schemas’, learning to use a system will be most successful [5]. This effect of prior knowledge with regard to learning new system interactions has already been investigated and placed into context to some extent in the cognitive load theory (CLT) research [6]; an instructional design theory with the aim of assisting instructional designers to reduce the cognitive load caused by poor design of learning materials [5]. Cognitive load in this theory refers to the amount of extraneous load due to instructional design [6]. However, in the field of Human Computer Interaction (HCI), a systems learnability is also placed in relation to cognitive (work)load of an interface. Where the extraneous load is induced by using a software tool [6]. HCI focuses on software issues that may for example cause undue cognitive demands on the user, as a result of several aspects, for example: being poorly designed with low usability, such that it is unclear how to use the application; the data or settings to be remembered are spread across multiple screens, or; the way the software works conflicts with existing knowledge about the domain. These artefacts will make it more difficult or even impossible for a user to effectively learn to interact with a system [5]. As a result, the usability, learnability and satisfaction of users with the system may be influenced negatively.

It is important to acknowledge and understand the discrepancy in focus and perspective on cognitive load in both fields [7-10]. In HCI cognitive load is induced by using a software tool (or HIT), in CLT cognitive load is due to instructional design [6]. However, both fields have a strong focus on minimizing extraneous cognitive load [6]. The literature on CLT describes that reducing working memory load optimizes learning [11]. However, in HCI it is unclear what effect minimizing extraneous load has on a systems’ learnability in relation to its usability. Additionally, numerous ways of reducing cognitive load (mainly in the context of educational software systems) are described in both fields, but not many attempts have been made to actually measure cognitive load in HIT and assess its impact on the usability of a HIT. Therefore, the aim of this thesis is to provide insight on how cognitive load of an interactive HIT relates to HIT usability. To do so, the need exists for a theoretical framework that summarizes the current knowledge and applied definitions within existing theories on cognitive load and applied measurements to assess cognitive load. The resulting overview of these theories and measurement methods will be applied to assess the cognitive load of two interface designs of a financial administration screen for Diagnosis Treatment Combinations (DBC) and care trajectories of a Hospital Information Systems (HIS). The system under study was recently given an update in which the interface of the system changed. End-users with differing background experience and thus differing cognitive ‘information schema’s’ (experienced with old versus experienced with new system) will evaluate both system interfaces in relation to their experienced usability. In doing so, insights will be gained into the methods for the design and evaluation of interactive HIT that take both CLT and usability concepts into account. In addition, the impact of user characteristics in relation to cognitive load by taking into account prior knowledge with regard to software can be assessed.

(12)

12

1.1 Outline and research questions

Two main research questions are addressed in this thesis. In order two answer the main research questions two related studies are performed. First a scoping review is performed and a theoretical framework is established. The literature is searched on existing cognitive load theories and reported cognitive load measurement methods. During the literature search use of different terminologies and a plethora of definitions for those terminologies are noticed. Therefore, these terminologies and definitions are extracted to provide a universal definition on cognitive load to augment the definition used in the field of HCI. In addition, identified studies inspected the use of a usability assessment method. The existing cognitive load theories and current knowledge on the relationship between cognitive load and usability assessment of user interface designs will be mapped in a theoretical framework.

Research question 1: What theoretical insights have been reported on in the literature on how can cognitive load of an Interactive Health Technology interface adequately be measured to assess its learnability in terms of usability?

To answer this question, the following sub-questions will be answered:

1.1 Which cognitive load theories are currently being reported on in literature?

1.2 What definitions on cognitive load and related terminologies are applied by the authors?

1.3 What cognitive load measurement methods are reported on and how can these measurement methods be applied to ideally assess the cognitive load of an interface design?

1.4 What usability methods are reported on and are most appropriate in the context of cognitive load assessment? The theoretical framework and the identified cognitive workload measurement methods will be used as input for the second study; a cohort study. The cohort is a convenient sample of end-users of users and former users of the two versions. In the main study two versions of an interactive health technology interface will be assessed. The main outcomes of the cohort study are: perception on usability and perceived and measured cognitive workload of the end-users. The study design and the measurement methods used in the main study are informed by the current knowledge on how the cognitive load of an Interactive Health Technology interface can be adequately measured to assess its learnability in terms of usability.

The aim of the main study is to answer the second main research question and sub questions;

Research question 2: What is the relative effect of the perceived and/or measured cognitive workload of participants interacting with two versions of an Interactive Health Technology (IHT) interface on the IHTs usability?

To answer this question, the following sub-questions will be answered:

2.1 What is the perceived versus measured cognitive workload of the users of the old (Ezis) versus the new (HiX) interface design?

2.2 How do users assess the usability of the old (Ezis) versus the new (HiX) interface design in terms of efficiency and task satisfaction?

Then the relative effect of the cognitive workload on the IHTs usability of a user interface is determined, by combining the perceived usability and cognitive workload. To finally conclude whether and how the learnability of an IHT interfaces could be predicted based on the cognitive load.

(13)

13

1.2 Study design

The study comprehends a scoping review and the main study; cohort study (See figure 1). A scoping review will be performed to answer the first research question. Based on the scoping review a theoretical framework will be developed on current cognitive load theories, provide definitions, measurement methods, and usability testing methods in the context of the cognitive load assessment of a HIT. The theoretical framework will inform the method selection and study procedures of the main study. In the main study the cognitive load and usability of two interfaces of a HIS will be compared; Ezis and HiX. During the data analyzation the specific system experience of the participants will be taken into consideration.

Figure 1 - Flowchart of the study design

1.3 Chapter organization

The next chapter contains context information of the main study. In the context information the background and relevance of the main study, and the two version of the interactive health technology interface are described. In chapter three the research methodology of the scoping review is described, the theoretical framework is presented and concluded with a discussion. The study procedure and methodologies of the main study (informed by the scoping review) are described and clarified in chapter four. The results of the perceived and measured cognitive load and assessed perception on usability of the two versions of an interactive health technology are presented and discussed in chapter five. The thesis is finalized with an overall discussion on the theoretical framework, the results of the main study, strength and limitations of the studies, and recommendations and implications for further research.

(14)

14

2 CONTEXT OF MAIN STUDY

2.1 Background

Information technologies and the underlying software evolve over time; it is the process of developing, maintaining and reengineering software [12]. Software change is inevitable, as the business environment changes, errors must be repaired, the performance or the reliability of the system may have to be improved and add or modify system functionalities [12]. For many organizations the key problem is implementing and managing changes to their existing software systems. One of the challenges for implementing new software or a new version of the software is concerned with user acceptance. Due to the software evolvement the new version can strongly differ from the old version. When the new software functions or processes are not sufficiently communicated to the end users, and if the new software interface is not intuitive the user acceptance is in jeopardy [12]. As information technologies are changing over time so do Hospital Information Systems (HIS). A large part of the Dutch Hospitals and Care Centers implemented CS-Ezis.Net/HiX (ChipSoft) as a HIS. In 2013, ChipSoft presented version HiX as the successor of CS-Ezis.Net (Ezis). The version update from HiX to Ezis also concerned the hospitals financial administration. Accompanied with the update, the user interface of the “Episodeoverzicht patient” changed. The “Episodeoverzicht patient”, is an overview that presents all Diagnosis Treatment Combinations (DBC) and care trajectories of a patient; historically, closed, parallel, signal periods and DBC’s that are open for more than 120 days (or 90 for outpatient non-operative treatment). A DBC represents the complete treatment process, from diagnosis to potential treatment and tests or follow-ups (care activities) [13]. After a DBC is closed, the financial administration will prepare the invoice, and send and declare the invoice by the healthcare insurer. The registration of the DBC, care activities and the declaration are a complex but highly important task. Due to the update of Ezis to HiX, the financial administrators (end users) perceive the overview as more complicated and are less satisfied.

User satisfaction is directly affected by the overall perceptions of performance or cognition, and usability [6, 14, 15]. Previous studies in CLT found that self-reported mental effort (cognition) was positively correlated with task difficulty and negatively correlated with task familiarity [11, 16]. Still, it is unclear whether the decreased satisfaction is due to decreased usability and user performance, or due to possible overload of users working memory induced by task complexity and irrelevant cognitive load. In other words, there is a lack of understanding of factors influencing the usability of a HIT and their interrelations with respect to the CLT. Therefore, the cognitive load and the usability of the interface of interest (“Episodeoverzicht patient”) in both Ezis and HiX will be assessed.

2.2 Description of the HiX and Ezis interfaces

In Ezis the care trajectories and DBC’s are displayed in one view (See figure 2). The columns that are presented (from left to right) in the interface are the specialisms of the care trajectory and DBC, type code, segment code, action needed status, status of invoicing, the begin date and end date of both care trajectory and DBC, description of the care trajectory and DBC including the diagnosis, the reason the care trajectory and DBC are closed, type of care, the care demand code, diagnosis code, treatment code, number of the care trajectory and DBC, and care flow. In Ezis five colors are used to provide information about the care trajectory and DBC; blue indicates that it is a care trajectory, black is a DBC, red indicates that the DBC is open too long, orange is the signaling period, and green indicates that there is a parallel DBC. The color legend is presented in the bottom left corner of the view. The view includes six buttons with accompanied actions/functionalities; ‘Wijzigen’ allows the user to edit the DBC, ‘Toevoegen’ to add/register a new care trajectory/DBC, ‘Verwijderen’ to delete a care trajectory/DBC, ‘Uitbehandeld’ to close the selected care trajectory and DBC, ‘Afsluiten’ closes the selected DBC, ‘Patient’ to change the patient for which the care trajectories and DBC’s are presented, and ‘Sluiten’ to close the interface. By selecting a DBC the information on the DBC will be presented under the header ‘Verwijzing’. The interface provides the user with the possibility to filter the displayed care trajectories on the date (‘Datum vanaf’), and whether the care trajectory/DBC is not closed (‘Openstaand’). The care trajectories and related DBC’s are presented using a tree-graph representation.

(15)

15

Figure 2 - Screenshot Ezis Interface evaluated in this study

In HiX the care trajectories and DBC’s are also presented in one view (See figure 3). In addition to the care trajectories and DBC’s the care activities are presented. The data of the care trajectory is displayed using columns; specialism, location code, type code, segment code, status of invoicing, status of the DBC, parallel (whether there are parallel DBC’s), start date care trajectory, end date care trajectory, care type code, care demand code, diagnosis description, ICD-10 code, care product, start date DBC, end date DBC. When a care trajectory is selected, the DBC’s belonging to the care trajectory are presented in the groupbox below (‘DBC’s) the care trajectories. The data displayed for the care trajectory and DBC has overlapping columns. In addition, the diagnosis code (‘Diagn.’) and the DBC number and care trajectory number (two right columns) are displayed. When a DBC’s is selected the linked care activities, are presented in the groupbox below (‘Zorgactiviteiten’). The care activities are organized based on care class, foreach care class the code, description, number of care activities in that care class, units, and the care activities codes are displayed. For each care activity the number of the linked DBC, the date, code, amount, unit, description, applicant code, performer code, location, and cost center are presented. In HiX the view contains ten buttons; ‘Wijzigen’, ‘Toevoegen’ and, ‘Verwijderen’ to edit, add, and delete a DBC. ‘DBC Afsluiten’ to close the DBC and ‘Zorgtraject uitbehandeld’ to close the care trajectory and the belonging DBC’s. ‘Valideren DBC’ to check and validate the DBC, and two buttons to open other functionalities; ‘Toewijzen van verrichtingen’ to link the care activity to the DBC and ‘Ambulant verrichtingen’ to add a care activity to the DBC. A button to perform other actions and ‘Sluiten’ to close the view. HiX also allows the user to filter the care trajectories on the end date, and whether the care trajectory is open or closed. Other filter options are specialism, whether or not show all DBC’s, whether or not to show other care products of the first line diagnostics, and prepayments.

(16)

16

(17)

17

THEORETICAL FRAMEWORK ON CURRENT KNOWLEDGE ON COGNITIVE LOAD

THEORIES AND MEASUREMENT MODELS BASED ON A SCOPING REVIEW

(18)

(19)

19

3.1 Research methodology

In order to answer the first research question, a scoping review is used as research methodology. Colquhoun et al. (2014) define a scoping review as: “a form of knowledge synthesis that addresses an exploratory research question aimed at mapping key concepts, types of evidence, and gaps in research related to a defined area or field by systematically searching, selecting, and synthesizing existing knowledge” [17]. It should be noted that the knowledge synthesis is frequently qualitative and rarely quantitative. Since, the aim of scoping reviews is to synthesize evidence from studies with different study designs and to review identified studies without critically appraising them and provide a descriptive overview of the identified key concepts [18]. The framework outlined by Arksey and O’Malley (2005) and further developed by Levac et al. (2010) is used as guidance for carrying out the scoping review. The framework describes the scoping review as a five-phases iterative process with an optional sixth phase ‘consultation exercise’. The scoping review of this study encompasses the following five phases: 1) identifying the research question, 2) identifying relevant studies, 3) study selection, 4) charting the data, and 5) collating, summarizing, and reporting the results [17, 18].

3.1.1 Research question

The aim is to perform the scoping review to identify relevant studies and to synthesize existing knowledge to answer the main research question, ‘What theoretical insights have been reported in the literature on how the cognitive load of an Interactive Health Technology interface can be adequately measured to assess its learnability in terms of usability?’. The main research question encompasses the following sub research questions:

1. Which cognitive load theories are currently being reported on in literature?

2. What are the definitions on cognitive load and related terminologies applied by the authors?

3. What cognitive load measurement methods are reported on in literature and how can these measurement methods be applied to ideally assess the cognitive load of an interface design?

4. What usability methods are used by the authors and are most appropriate in the context of cognitive load assessment?

3.1.2 Identifying relevant studies

Before the initial search, search terms/keywords were collected, by first identifying, the three key terms in the research question, ‘Cognitive load’, ‘measurements’, and ‘Interactive Health Technology interface’. Followed by determining relevant associated terms and alternative synonyms. Resulting in 12 terms for the key term ‘cognitive load’, 7 terms (without derivatives) for ‘measurement’, and 5 alternative terms for ‘Interactive Health Technology interface’ (See appendix A1). In addition, a data collection search template was made including the database/platform name, date of search, search options, search query and number of articles (See appendix A2). The initial search was performed in three electronic databases: PubMed, SCOPUS, and IEEE Explore on title and abstract. The searches were not limited on date but were limited to language; English. For each database the search query was tailored to the specific requirements. The electronic database IEEE Explore is limited to a maximum of 15 search terms. Using apprais*(derivatives) or comparable terms (See appendix A1), increases the number of search terms extensively, therefore the measurement terms are not used. In addition, is the search in IEEE Explore divided in two searches. The ‘snowball’ technique was applied to identify and extract possible relevant citations within articles. The ‘snowball’ technique was also applied to extract relevant citations from identified conferences.

(20)

20

3.1.3 Citation management

All the citations were imported into the web-based systematic review software Covidence. “The Covidence online software product improves healthcare evidence synthesis by improving the efficiency and experience of creating and maintaining Systematic Reviews” [19]. When new citations are imported, Covidence will automatically detect and remove duplicates. Using the import managing screen the duplicates are reviewed, to determine whether the duplicates were removed correctly. In addition, the web-based version of EndNote, MyEndNoteWeb, was used to manage references identified using the ‘snowball’ technique. MyEndNoteWeb was also used to import the citations found during the divided search of IEEE Explore. To remove duplicates prior to the import in Covidence and avoid a skewed number of duplicates.

3.1.4 Title and abstract screening

The title and abstract screening is preformed using Covidence. To enhance the title and abstract screening, inclusion and exclusion keywords were highlighted by using the Covidence setting. Exclusion keywords were highlighted red and encompassed terms associated with the wrong population, ‘patient’, ‘disabilities’, and ‘disability’. During the title and abstract screening, the keywords ‘elderly’ and ‘impairments’ were also set as exclusion criteria. Title and/or abstracts either describing a cognitive load measurement method or cognitive load model/theory in a relevant context or intervention were included for subsequent review of the full-text article. Contexts or interventions that were labelled as irrelevant are; virtual/augmented reality, 3D, adaptive user interface, automotive/in-vehicle, table top user interface, multimodal user interface, tangible user interface, games, robots/robotics, nuclear power plants, and air traffic management.

3.1.5 Full text screening and data extraction

For each citation deemed relevant after title and abstract screening the full-text was searched through institutional holdings and imported in Covidence. For some articles the full texts could not be obtained and were excluded. The full-text were screened using the same criteria described in the section title and abstract screening. After completion of the full-text screening the reference list with relevant articles was exported to an Excel sheet and imported in Microsoft Excel 2016. The Excel sheet was transformed to a data extraction form by adding eight columns. For the extraction of cognitive load related data four columns were added, terminologies used, provided definition, cognitive theory described, cognitive load measurement method used/described (See appendix A3). Two columns were added for HCI/Usability, to extract whether a usability method was used and which theory is described for the application of a cognitive load measurement in HCI or the used usability testing method. The other columns were added to collect the overall/key findings described in the article and the publication year.

3.1.6 Data summary and synthesis

The data was extracted by copying relevant sentences/paragraphs in the data extraction form. The extracted data was summarized in the data extraction form by adding a row for each article and summarizing the extracted data. Descriptive statistics were calculated by using the filter functionality of excel to summarize the data. Each unique measurement method and cognitive load theory/model will be described in the results section.

(21)

21

3.2 Results

The results section is structured in five subsections; in the first section the search and selection of the articles identified during the scoping review are presented. The second section provides an overview of the general characteristics of the include articles. In section three the different cognitive load theories reported on in the included literature are described. The definitions provided by the authors are presented in section four. The reported cognitive load measurement methods are described in section five. Additionally, the advantages, disadvantages and comparisons of the measurement methods are discussed. In section 5 the applied usability methods are described and the main findings on usability and cognitive load are discussed.

3.2.1 Search and selection of articles

In December 2017 three electronic databases (PubMed, Scopus and IEEE Explore) were searched and returned 295 potentially relevant citations. By applying the snowballing technique (reference list screening) and web search another 18 records were identified. After deduplication, 278 articles were screened against title and abstract, 75 studies were assessed for full-text eligibility. However, after unsuccessfully attempting to obtain full-text, seven studies were excluded. In addition, three studies were identified as duplicates and were removed. Another 35 studies were excluded since they were not relevant, resulting in 44 exclude full-text articles. Studies that were excluded and categorized as irrelevant were due to, wrong study design, wrong setting, wrong outcomes, wrong intervention, and wrong population. In total, 30 articles reporting on a cognitive load theory, or either describing or using one or more cognitive load measurement methods were included. Figure 4 represents the flow of articles trough identification, screening, eligibility to final inclusion.

Figure 4 - PRISMA flowchart of study selection process

The data from the articles was extracted into excel using a data extraction form. The data that needed to be extracted from the articles was; authors, year of publication, used terminologies to define cognitive load, definition for the used terminology, used cognitive load measurement method, described cognitive load theory, usability method used, usability theory described and overall/key findings on the usability (See appendix A4).

(22)

22

3.3.2 General characteristics of included articles

The general characteristics of the articles included in this review are the author, year of publication, whether the author(s) described a cognitive load theory, the terminologies used, whether a definition is provided, if the authors used or described one or more cognitive load measurement methods, and if a usability method is applied in the study. The general characteristics of the included articles are presented in table 1.

Table 1 - General characteristics of included articles

Author Year Terminologies Definition Theories Measurement _method Usability _method

Meshkati [20] 1995 Mental workload No No Yes* No

Hu [21] 1999 Cognitive load Yes Yes Yes Yes

Feinberg [22] 2000 Cognitive load, Working memory Yes Yes No No

Wang [23] 2002 Cognitive load, Mental workload No No Yes Yes

Horsky [24] 2003 Cognitive complexity No Yes Yes Yes

Iqbal [25] 2004 Cognitive load, Mental workload No No Yes No

Cain [26] 2007 Mental workload Yes No Yes* No

Saleem [27] 2007 Mental workload No No Yes No

Wästlund [28] 2008 Mental workload Yes Yes Yes No

Saleem [29] 2009 Mental workload Yes Yes Yes No

Sheehan [30] 2009 Cognitive load, Cognitive effort Yes Yes Yes Yes

Block [31] 2010 Cognitive load No Yes Yes No

Gwizdka [32] 2010 Cognitive load, Mental effort Yes Yes Yes No

Saitwal [33] 2010 Mental workload No Yes Yes No

Bandlow [34] 2011 Cognitive load, Working memory Yes No Yes No

Chen [35] 2011 Cognitive load Yes No Yes No

Cinaz [36] 2011 Cognitive load No No Yes No

Jovanovic [37] 2012 Cognitive load, Working memory No Yes No No

Longo [38] 2012 Cognitive workload, Mental workload Yes No Yes Yes

Reis [39] 2012 Cognitive load, Working memory Yes Yes Yes Yes

Maior [40] 2013 Cognitive workload, Working memory Yes Yes Yes No

Roy [41] 2013 Mental workload, Working memory Yes No Yes No

Suebnukarn [42] 2013 Mental workload, Mental effort No No Yes No

Pike [43] 2014 Mental workload, Working memory Yes Yes Yes Yes

Zhang [44] 2014 Cognitive workload, Mental workload No No Yes No

Ariza [45] 2015 Cognitive workload, Mental workload Yes No Yes No

Fukuzumi [46] 2015 Mental workload No No Yes No

Chakraborty [47] 2016 Cognitive load No No Yes No

Lukanov [48] 2016 Mental workload, Working memory Yes Yes Yes No

Shah [49] 2016 Mental workload No Yes Yes Yes

All included articles were published between 1995 and 2016, sixty-three-point three percent (19/30) published after 2009. The terminologies most commonly used are cognitive load and mental workload, either combined or with another terminology (See figure 5). Figure 5 presents the terminologies used, and the number of articles that used the terminologies either alone or in combination. Fifty-six-point seven percent (16/30) of the articles provided a definition for the used terminology. For the included articles the terminology mental workload was introduced in 1995 by Meshkati and cognitive load in 1999 by Hu et al.

(23)

23 Figure 5 - Terminologies used

Fifty percent (15/30) of the articles reported on one or more cognitive load theories. However, thirty percent (9/30) of the articles did not report on a cognitive load theory or provided a definition. Thirty-three-point three percent (10/30) of the articles reported on a theory and provided a definition, twenty percent (6/30) only provided a definition, and sixteen-point seven percent (5/30) only reported on a cognitive load theory. Cognitive load measurement methods are described in ninety-three-point three percent (28/30) of the articles. Where twenty-six-point seven percent (8/30) also used a usability measurement method.

The snowballing technique demonstrates that sixty-six-point seven percent (20/30) of the articles refer to one or more of the included articles (See figure 6). Lukanov et al. (2016) refers to four included articles [26, 28, 38, 43]. The used terminologies of Lukanov et al. (2016) is mental workload and working memory, which corresponds with the used terminology mental workload in the four articles, and the terminology working memory by the references Pike et al. (2014). Longo et al. (2012) also refers to Cain, and they both use mental workload as a terminology. Cain (2007) refers to Meshkati et al. (1995) which also uses mental workload as a terminology. The use of terminology in an article and the corresponding use of terminology in the referred article(s) emerges in eighty-five-point five percent (17/20) of the reference linkage. Saleem et al. (2007) and Saitwal et al. (2010) both used mental workload as a terminology and refer to Horsky et al. (2003) using cognitive complexity as a terminology. Sheehan et al. (2009) refers to Saitwal et al. (2010) while respectively using mental workload and, cognitive load/cognitive effort. The use of terminologies and references suggest that the used terminologies are informed by the referred article.

Iqbal et al. (2004) used the terminologies mental workload and cognitive load and Wastlünd et al. (2008) used the terminology mental workload. However, Gwizdka (2010) refers to Iqbal et al. (2004) and Wastlünd et al. (2008) and used the terminologies cognitive load and mental effort. So principally, the use of terminologies in the article of Gwizdka (2010) is not informed by the reference Iqbal et al. (2004). However, Gwizdka (2010) provides a composite definition of workload; a definition for both mental workload and cognitive load (See section ‘Definitions used’). The use and description of terminology is also reflected in the three articles that refer to Gwizdka (2010). Since, Maior et al. (2013), Zhang et al. (2014), and Longo et al. (2012) all used the terminologies cognitive load and mental workload.

(24)

24

Figure 6 - References connections of the included articles

Gwizdka (2010) underlines that cognitive load is a concept of interest in several disciplines, including cognitive and educational psychology, human factors, and engineering psychology. In the disciplines human factors and engineering psychology cognitive load is referred to as mental workload [32]. Indicating that the terminologies cognitive load and mental workload are arbitrary. However, even within a singlediscipline no universally definition is provided for cognitive load/mental workload. Therefore, the definitions provided by the authors will be presented and discussed (See section ‘Definitions used’).

3.3.3 Reported theories

In fifteen articles one or more theories on cognitive load are described, resulting in thirteen different theories. Between those theories underlying relationships can be created. Three theories (cognitive modal model of learning, human information processing and information processing model) are directly related to the working memory model (See figure 7). The remaining nine theories are either directly or indirectly related to the cognitive modal model of learning or information processing model. The system usability model and mental workload model are independently related. Since, ten out of the twelve theories are directly or indirectly related to the working memory model this model will be described first.

(25)

25

Working memory model

Six studies [22, 28, 32, 37, 40, 43] acknowledge the working memory model of Baddeley and Hitch [50]. Figure 8 presents the four main components of the working memory model. The phonological loop and the visuo-spatial sketchpad respectively store temporal phonological information (verbal information in an acoustical form) and visual-spatial information (such as pictures and diagrams). The episodic buffer links verbal and spatial information in chronological order [40, 43]. The purpose of the central executive component is controlling and coordinating information processing and managing attention [28, 32, 37, 40, 43. Wastlund

et al. (2008) also described that due to the limited storage capacity of working memory, it interacts with the long-term memory by

creating schemata of information [51]. In addition, is it assumed that the episodic buffer has links to the long-term memory component [40].

Figure 8 - Working Memory Model [ (43; 40)]

The cognitive modal model of learning

Feinberg et al. (2000) describe the cognitive modal model of learning of Sweller and Cooper. The cognitive modal model of learning distinguishes between three distinct memory types (modes); sensory memory, working memory and long-term memory [22]. The sensory memory transfers incoming stimuli from our senses (sights, sounds, smells, tastes and touches) [22]. Working memory consist of four components (See section ‘Working memory model’). As mentioned in section ‘Working memory model’ the working memory is limited. Therefore, the working memory interacts with the long-term memory by creating schemata of information. In other words, learning takes place in working memory [22]. Long-term memory is responsible for storing knowledge and skills in a more-or-less permanently accessible form [22]. The working memory processes those knowledge and skills recalled from long-term memory [22]. In contrast to working memory, the long-long-term memory capacity appears to be unlimited.

Cognitive load theory

Three studies [22, 32, 39] describe the cognitive load theory. The cognitive load theory classifies cognitive load into three distinct types; intrinsic load, extraneous load and germane load [32, 39, 51-54]. Intrinsic load is imposed by the complexity of a given task or problem that must be processed or solved by the user [32, 55]. The environment and/or interface where the task is being performed imposes the extraneous load. The third type is germane load and is the intentional effort invested by an individual in learning that goes beyond simple cognitive task performance and comprehension [32, 55]. The demands placed on the working memory is the basis of the cognitive load theory [39, 56]. According to this theory the information processing capacity of humans is limited [39]. The extraneous load, such as the level of irrelevant information presented to the user is the main interest of interface designers [39]. As is it important to create an interface that reduces the extraneous load and increases the intrinsic and germane loads [39]. Which will allow users to learn and construct their knowledge more effective [39]. As described in the cognitive modal model of learning, learning takes places when information from working memory is committed to long-term memory [32].

Distributed cognition

Three studies [24, 29, 33] used the distributed cognition as an underlying theory; “Distributed cognition emphasizes the inherently social and collaborative nature of cognition and also characterizes the mediating effects of technology or other artifacts on cognition”. In other words, knowledge does not solely lie within a user but also in the social and physical environment. Distributed cognition constitutes an indivisible information-processing system. The information processing system is the process of

(26)

26

coordinating distributed internal (mental) and external (physical) representations [24, 33]. Distributed cognition can be used to understand how external representations (e.g., buttons) can reduce cognitive load [24]. For example, external representations can support recognition-based memory or perceptual judgments rather than recall [33]. The external representation can be related to the extraneous load described in section ‘Cognitive load theory’.

In line with distributed cognition, Gwizdka et al. (2010) highlight that it is crucial to understand what contributes to a user’s cognitive load on tasks. As it enhances the understanding of the interaction process and helps to identify tasks types and system features that impose increased levels of cognitive load [32, 57]. In addition, cognitive load is relative to the user, the task being completed, and the system employed to accomplish the task. In other words, the level of difficulty experienced by the user is influenced by the task, system and individual characteristics.

Distributed resources model

There is an underlying relationship between distributed cognition and the distributed resources model. As the relative difference in distributed cognition representations (internal and external) are essential for the efficacy determination of a system [24]. Horsky

et al. (2003) describes the distributed resources model proposed by Wright et al. [24, 58]. The distributed resources model

supports the analysis of the information required to perform a task and the allocation of the information (internal or external) [24]. The distributed resources model includes two major components; abstract information structures and interaction strategies. The abstract information structures can be either internalized as a mental representation or externalized in the artifact [24]. The original distributed resources model differentiates six abstract information structures; plans, goals, possibilities, history, action-effect relations and state. Table 2 provides an overview of the six abstract information structures and description. Interaction strategies describe the decisions making of actions, based on the different states. Therefore, the states can highly influence the range of available interaction strategies [24].

Table 2 - Abstract information structures of the distributed resources model

Information structure Description

Plans The resources for an action and include the sequence of actions, events, and anticipated states.

Goals The desired states the user wants to achieve, either formulated internally or developed during system interaction.

Possibilities The configurations of resources (e.g., links, buttons, and menus), that provide the user with possible next actions to take for a state of the system.

History The part of a plan that has already been accomplished.

Action-effect relations The relationship between an action and the effected change in state.

State The current configuration of resources

The Model Human Processor

Hu et al. (1999) describe the Model Human Processor (HIP) [59] paradigm. The human processor consists of essential components as memories, processors and inter-component connections [21, 60]. The memories are the working memory and the long-term memory [60]. The processors include the perceptual, the cognitive, and the motor processor [21]. The external world is detected by the sensory system, those sensations are transmitted by the conceptual processor to the perceptual processor. The perceptual processor transforms the detected sensations into symbolically coded internal representations stored in the perceptual memory [21]. The motor processor transforms thought into action by activating patterns of voluntary muscles [21, 60].

The representations of perception-based knowledge/information stored in the perceptual memory can be classified in two schemata. The image-based schema communicates an intended concept by using a graphical representation that encompasses one or several prominent visual properties. The other schema is a linear-ordering-based scheme and depends on display sequence for the communication of an intended concept [21]. Therefore, items in an interface can be either presented to the user using a graphical or list-based interface [21].

(27)

27

Information Processing model

Maior et al. (2013) and Pike et al. (2014) describe Wicken’s Information Processing model [40, 43, 61]. The model aims to illustrates the interconnection of human information processing system elements (e.g. attention, perception, memory, decision making and response selection) [40, 43]. Wickens describes three different stages of information transformation; a perception stage, a processing or cognition stage, and a response stage. The perception and processing state are processes involved in cognition; perceive information (gathered by our senses) and provide meaning and interpretation to the perceived information. In the second stage perceived information is manipulated and thought about. This stage of information processing consists of a wide variety of mental activities and takes place in working memory [40, 43]. The response stage is concerned with the response selection, response execution and the response feedback which will be registered by the sensory register [40, 43].

Multiple Resource model

Three studies [32, 40, 43] describe the Multiple Resource model of Wickens. The Multiple resource model is a four-dimensional representation of mental workload (See figure 9) [40, 43]. The stages dimensions are the three main stages of the information processing model of Wickens describe above. The modality dimension represents the different source of auditory and visual perception [40]. The codes dimension refers to the types of memory encodings; spatial or verbal. The visual processing dimension is a nested dimension within visual resources distinguishing focal (reading text) and ambient vision (orientation and movement) [40]. In other words, humans have a set of mental resources of several types [32].

Figure 9 - The 4-D multiple resource model, by Wickens

Limited resource model

The limited resource model is associated with the multiple resource model. The limited resource model describes the relationship of task demands, resource allocation and the impact on performance [40]. Gwizdka et al. (2010) describes an example of this relationship, “When demands of one task are high, the resources committed to that task become unavailable to a second task if it requires the same type of mental resources (e.g., visual vs. auditory) and at the same stage of processing (e.g., cognitive vs. response-related).” Wang et al. (2002) also assume that users have limited attentional and cognitive resources to perform a task. The limitation in the perception and cognition stages are a result of limited capacity of working memory [32]. In line with the limited resource model Bandlow et al. (2011) describe that as a result of the consumption of cognitive resources, the ability of a user to engage in higher-order reasoning minimizes.

Cognitive timer model

Block et al. (2010) describe the application of the cognitive timer model in the area of HCI. Findings in other areas suggest, that higher cognitive load causes a less accurate perception of time or even leads to a decrease in perceived time [31, 62, 63]. The Cognitive-Timer model describes the impact of cognitive load on the perception of time [31, 64]. The Cognitive-timer model is based on three assumptions; 1) in humans a cognitive timer exist which processes and generates temporal information, 2) the timer processes the temporal information by storing the number of subjective time units which have accumulated during a given interval, and 3) there is a trade-off between the cognitive timer and other cognitive modules, due to the continuous allocation of attentional resources to enable both temporal and non-temporal information processing [31]. The allocation of attention resources

(28)

28

is presented in the information processing model. Block et al. (2010) proposes that the Cognitive-Timer model could be used in the area of HCI (for the design of input techniques) to predict an actively model perceived performance of user interaction.

Hutchin’s extension to Norman’s theory of Action

Sheehan et al. (2012) describes Hutchin’s extension to Norman’s theory of Action (See figure 10). Norman’s theory proposes seven stages of action classified in the gulf of execution and gulf of evaluation [30, 65]. The gulf of execution is the degree of mental processing involved in, forming the goal and intention, and specifying and executing the action. The gulf of evaluation is the mental processing required to perceive and interpret the state of the system and evaluate the outcome. Cognitive distance can be distinguished in three types of cognitive distance; semantic distance, articulatory distance and issue distance [30, 66]. The concept of cognitive distance is used to describe the degree and type of mental transformation required to connect the gulfs of execution and evaluation [30, 66]. Sheehan et al. (2012) defined semantic distance as, “the relationship between what the user wants to communicate and the meaning of the corresponding expression in the interface language”. The relationship between the meanings of the expression and their physical form is conceptualized as articulatory distance [30]. The cognitive effort required when a shift in goal is necessary is presented by issue distance [30].

Figure 10 - Hutchins extension of Normans theory of action [30]

System usability model

Shah et al. (2016) used the system usability model of Peikari et al. [49, 67, 68] as a theoretical foundation. The model consists of six components; information quality, ease of use, error prevention, consistency, reduction of mental workload and reduction of errors (See figure 11). The model is focused on the reduction of errors. The lines present the hypotheses formation. For instance, interface consistency leads to reduction of mental workload of the user. The hypothesizes of the theoretical model are tested in practice. Seven of the nine hypotheses (lines) are significantly supported by the results. The hypotheses that information quality reduces mental workload and that error prevention reduces errors are not supported [49].

(29)

29

Model of mental workload

Lukanov et al. (2016) refer to the model of mental workload of Sharples and Megaw. The key topic is that performance can be reduced by both high and low mental workload. Performance reduces when task complexity increases. Conversely, a user can become prone to errors as a result of boredom and apathy due to a repetitive task that does not utilize a users’ mental or physical resources [48].

3.3.4 Definitions used

As highlighted in the section ‘Outline and research questions’ there is no clearly defined, universally accepted definition for cognitive load/mental workload. In this review, more than half of the articles (17/30) defined either cognitive load, mental workload, or working memory. Descriptions that are used to define cognitive load are mostly focused on the effort or amount of mental resources required to perform a task. Five articles defined cognitive load; provided definitions are presented in table 3. Hu et al. (1999) described the model human processor, which is reflected in the provided definition on cognitive load. Since, the external world is detected by the sensory system; “… to take notice of the visual stimulus (stimuli) contained in and interface…”. Furthermore, the perceptual processor transforms the detected sensations into symbolically coded internal representations stored in the perceptual memory; “… comprehend its (their) significance or intended meaning.”. Feinberg et al. (2000) and Jovanovic et al. (2012) both described the working memory model. The working memory stores and links temporal information (phonological and visual-spatial), coordinates information processing and manages attention [40, 43]. However, the storage capacity of the working memory is limited [40, 43]. The provided definition by Feinberg et al. (2000) highlights the information processing capability of the working memory. The definition of Jovanovic et al. (2012) focusses on the limited storage capacity of the working memory. The demands placed on the working memory is also the basis of the cognitive load theory. The cognitive load theory distinguishes three types of load; intrinsic load, extraneous load and germane load [39]. When the extraneous load is reduced it will allow users to learn and construct their knowledge more effective [39]. The cognitive load theory is also reflected in the definition provided by Reis et al. (2012).

Table 3 - Cognitive load definitions

Author Year Theory Definition

Hu et al. 1999 The Model Human

Processor “… the amount of information processing effort a user must expend to take notice of the visual stimulus (stimuli) contained in an interface and comprehend its (their) significance or intended meaning.”

Feinberg et al. 2000 Cognitive modal model of learning

Working memory model

“Cognitive load theory is defined as the amount of “mental energy” required to process a given amount of information.”

Chen et al. 2011 - “Cognitive load is defined as a multidimensional construct reflecting the interaction between the task and performer characteristics and occurs as a result of the limited working memory available during tasks.” [69]”

Jovanovic et al. 2012 Working memory model “Cognitive load is a global term and refers to mental resources a person has available for solving problems or completing tasks at a given time.” [70]

Reis et al. 2012 Cognitive load theory “The cognitive load refers to the demands placed on the working memory of students during the learning process.”

The definitions of mental workload are comparable to the definitions of cognitive load. As the definitions are also in the form of mental resources demanded to perform a task. A definition for mental workload is provided in six articles of the review (See table 4). Cain (2007) presented five different formal definitions of mental workload. Using the snowballing technique, the definitions provided by Cain (2007) are based on either the information processing model, or the multiple resource model and the closely related limited resource model. The theories provided by the references are reflected in the definitions as; the information processing model is concerned with the information transformation (perception, processing or cognition, and response), the multiple resource model is the set of mental resources of several types, and the limited resource model describes the relationship of task demands, resource allocation and the impact on performance. In the first definition the theory is reflected in “…information

(30)

30

processing capacity or resources…”. The other four definitions are mainly focused on the limited resource model; when resources of the same type are required at the same time the resources need to be committed/devoted over two tasks [40]. The resource theory (information processing model, multiple resource model, and limited resource model) is also reflected in the definition provided by Saleem et al. (2009). For the definitions of Roy et al. (2013) and Lukanov et al. (2016) no underlying theory could be determined using the snowballing technique. The working memory model is reflected in the definition of Wastlund et al. (2008), as the working memory has limited storage and processing capacity for temporal information. Applying the snowballing technique for the article of Ariza et al. (2015) eventually a full article could not be obtained.

Table 4 - Mental workload definitions

Author Year Theory Definition

Cain 2007 - 1. “Mental workload refers to the portion of operator information processing capacity or resources that is actually required to meet system demands.” [71]

2. “... mental workload may be viewed as the difference between the capacities of the information processing system that are required for task performance to satisfy performance expectations and the capacity available at any given time.” [72] 3. “... the mental effort that the human operator devotes to control or supervision relative to his capacity to expend mental effort ... workload is never greater than unity.” [73]

4. “... the cost of performing a task in terms of a reduction in the capacity to perform additional tasks that use the same processing resource.” [74]

5. “... the relative capacity to respond, the emphasis is on predicting what the operator will be able to accomplish in the future.” [75]

Wästlund et al. 2008 Working memory model “… the use and temporary depletion of a finite amount of information processing capacity.”

Saleem et al. 2009 Distributed cognition “Mental workload is related to the difference between the amount of finite resources (i.e., attention or mental effort) available within a person and the amount of resources demanded by the tasks being performed.” [76]

Roy et al. 2013 - “Mental workload can be defined either as the load in working memory (i.e. number of items), the number of tasks to be performed simultaneously and more generally as a measure of the amount of mental resources engaged in a task. “

Ariza et al. 2015 - “Mental workload has been defined as the mental effort involved in performing any given task” [77]

Lukanov et al. 2016 Model of mental workload “The relationship between primary task performance and the resources demanded by the primary task” [78]

Two studies [32, 38] provided a definition as a composite of mental workload and cognitive load. The three definitions all focus on the demands and resources required to perform a task (See table 5). The definition of Gwizdka (2010) focusses on the multiple resource model and limited resource model. Since, the elements of the limited resource model (relationship of task demands, resource allocation and the impact on performance) are used in the definition. Longo et al. (2012) did not describe a theory in the article. However, Longo et al. (2012) refer to the article of Cain (2007) (See section ‘General characteristics’). The definition of Longo et al. (2012) is some sort of composite definition. The references to Cain (2007) and the references of the definitions of Cain (2007) suggests that the definition of Longo et al. (2012) is based on the resource theory.