Designing a Tablet Application for Emergency Management

(1)

Designing a Tablet Application for

Emergency Management

Bob C. de Graaff, BSc.

5941261

Abstract: Emergencies are high pressure situations in which a lot of information needs to be

communicated as fast as possible. For this purpose emergency management support systems are designed and used during incidents. Based on principles found during theoretical research and the findings of the “Disaster” experiment a prototype tablet application was created to fulfill this information sharing task. The prototype was tested on eleven fire fighters. The test showed that prototyping is extremely important in these situations, as users find it difficult to articulate their demands. We found that geographic information systems should be used in such applications, as the use of a map is viewed positively. We recommend the usage of symbols on the map, enriched with infowindows. It has also become clear that it is essential to show the source of a piece of information, to achieve a higher level of trust among users. A timeline function is deemed unnecessary for a tablet application, although it might be useful for higher command structures. Overall the tablet application is considered to be very useful, scoring above average for every measure in the SUMI questionnaire.

(2)

Abbreviations

AVL Automated Vehicle Location COP Common Operational Picture

CoPI Command container on the scene of the incident CVO Commission for Deliberation

LCMS Landelijk Crisis Management Systeem (National Crisis Management System, En. ) MICK Emergency Control Room

OT Operational Team, in charge of managing the effect area and indirect impact of an incident.

OvD Commanding Officer SA Situational Awareness

VNOG Veiligheidsregio Noord- en Oost Gelderland (Safety region North and East Gelderland, EN.)

VRK VeiligheidsRegio Kennemerland (Safety Region Kennemerland, En.)

Acknowledgments: Thanks to the Veiligheids Regio Kennemerland (VRK) for organizing the “Disaster”

experiment, and allowing me to participate. Specifically Rob Peters, Ellen Oude Kotte and Roy

Schinning for assisting me during “Disaster” and this thesis research. I would also like to thank Sammy Odenhoven, Eric klein Goldewijk and Jochem Beintema for the pleasant cooperation we had during the analysis of the results gathered during “Disaster”, on which this thesis is based. Eric in particular, thanks for helping me with my own experiment, driving me around Apeldoorn to contact as much participants as possible. Finally I want to thank Tom van Engers for his supervision during this thesis.

(4)

1 Introduction

During several emergencies in the Netherlands it has become clear that improvements had to be made to the communication methods and information systems used by the emergency services. The fire at Chemie-Pack in Moerdijk raised questions about communication and cooperation between safety regions, among others (PwC, 2012). During the incident surrounding the Turkish Airlines crash near Amsterdam Airport Schiphol precious time was lost due to miscommunication with regards to the locations of emergency response personnel, information about the people on board of the airplane was insufficient, and communication was hindered by faltering communications technology (Onderzoeksraad voor Veiligheid, 2009).

Obviously an improvement to emergency management support systems is necessary. This research paper is aimed at finding out how the information system that is to support emergency personnel, particularly first responders, can be made more effective. More specifically, we will look at a mobile solution for first responders, to facilitate effective communication between first responders and the responsible emergency managers. For this purpose a graphical user interface will be designed and built based on research done previously.

To increase the ease of information sharing between emergency agencies and affiliates the emergency services in the Netherlands have adopted the net-centric doctrine (Alberts, Garstka, Hayes, & Signori, 2001). This method of working demands a central information system. Most safety regions are now connected to this information system, the LCMS (National Crisis Management System). LCMS is a secure system which serves as an active database of available information, which is updated whenever new data is found.

Using the LCMS in each stage and command level of emergency management creates a common operational picture (COP). A COP entails that every participant in the emergency response effort has access to the same information, and achieves the same level of understanding through this

information. Widespread usage of this COP throughout the emergency response achieves higher effectiveness in command and control operations (Human Factors Integration Defence Technology Centre, 2009).

Part of the adoption of the net-centric doctrine and LCMS is to introduce tablet devices to the emergency response effort, enabling the ground crew working the incident to access this COP. Allowing the emergency services to access and edit the COP as they are working around the incident reduces time between the discovery of pieces of information and the sharing of this information, potentially saving lives (Alberts et al., 2001).

To assess the effectiveness of the use of these tablet devices an experiment was organized by the Safety Region Kennemerland (VRK) in the Netherlands. The experiment, named “Disaster” (EU, 2012; VRK et al., 2014) involved multiple safety regions working together, as well as the fire department of Amsterdam Airport Schiphol. Disaster (VRK et al., 2014) simulated a medium sized incident, where an airplane failed to stop in time, and had driven into a hangar. During Disaster (VRK et al., 2014) new pieces of information became available over time, which were shared using the information system (LCMS) and tablet devices.

Disaster (VRK et al., 2014) was extensively monitored by a team of student evaluators from the University of Amsterdam. During the experiment the evaluators were asked to observe the participants closely, while making notes. Disaster (VRK et al., 2014) was set-up using a start-stop methodology, which allowed the evaluators to ask questions during the stop periods.

(5)

The evaluation effort was divided in four separate teams, each evaluating a different role and location within the emergency response effort. These were: OT/CVO (Operational Team /

Commission of Deliberation), Airside (the actual response at the emergency site), CoPI (Command on the Scene of the Incident), and the MICK (the control center). For each role a full report on the proceedings during the experiment was delivered.

In addition to these evaluation reports the experiment was extensively filmed. The video footage was taken at the four locations, as well as body mounted cameras on some of the key participants. In work done prior to this research these video images were studied and analyzed. Information exchanges were tracked throughout the experiment to create a picture of the usage of the information system.

Analysis of these reports and video images has made clear that there are still some issues with the interface and workings of the tablets used during the experiment. The first step of this research has been to build on the previous analysis of these results to create a requirements list. This list can be found in Appendix A.

From the requirements list several have been selected to implement in a working prototype to illustrate the proposed improvements to the user interface. A further elaboration can be found in the Requirements section. The prototype is tested on fire fighters who will work with the tablet devices in the field using the quantitative SUMI (Kurakowski, 2012) method. This questionnaire is

complemented with some questions about specific parts of the prototype. Additionally, some qualitative research is done to further find the participants opinion on the application. The full setup of the research can be found in the research design section. First we will take a look at some

literature on information sharing in emergency management.

(6)

2 Literature review

Emergency situations can be long lasting operations, requiring cooperation between multiple

agencies, information systems and people. Effective cooperation within such a complicated structure requires a predefined command and control method. In the Netherlands the government and safety regions have decided to adopt the network centric doctrine (“Besluit Veiligheidsregios”, 2013, p. 6), which is based on network centric warfare (Wogaman, 1998).

2.1 Working in a network centric way

As the name implies, working in a network centric way requires all the involved agencies to be connected together by a network or information system (Wogaman, 1998). This emergency management support system contains all information available about the incident, allowing everyone involved to access this information at any time. In the Netherlands the LCMS (National Crisis Management System (En)) was designed to fulfill this purpose. The system is used to combine the information infrastructure of all emergency agencies together and offer a Common Operational Picture (COP). The COP shows the available information to everybody involved in the emergency response in a standard way.

Having a COP improves the shared situational awareness during the incident (Alberts et al., 2001). Shared situational awareness implies that everyone involved in the operation has the same knowledge about the situation, including the emergency agencies’ intent within said situation and the current location of crisis response personnel and equipment (Alberts, 2007). Having a COP and thus a correct, shared image of the incident allows for more effective cooperation, which in turn will lead to more effective actions, better effects and lower business impact (Harrald & Jefferson, 2007). According to Carver & Turoff (2007) there are three main areas of research in emergency

management support systems and tools. The first and second, information prioritization and decision

support and modeling tools will not be discussed in this research. However, the third item,

representation of a common operational picture is the main focus of this paper. They define this as a

“manipulable visualization of what is happening and where resources are that is open to all members of the emergency management team” (Carver & Turoff, 2007, p. 36).

Emergency management requires a different interface than regular information systems. In emergency management, the information is often linked to a geographic location (Gunes & Kovel, 2000). Therefore the main screen of the LCMS is a map, which shows symbols and other elements that relay information to the user. Information can be added and removed from this map by the users. This screen creates the COP.

2.2 Information Overload

A big problem with having a common operational picture that has to serve everyone is information overload (Eppler & Mengis, 2004). Information overload occurs when a person receives too much information, overloading their ability to process it. Information overload can happen with computer systems, but also in other forms of communication. In their study on context awareness in

firefighting Jiang at al. (2004) found that firefighters sometimes have to listen to three radios, while simultaneously receiving mobile phone communications.

Having all the information available to everyone at any time can be confusing to users, and leave them with questions such as: “What information is crucial to me?” These problems extend to system designers. Figuring out what is ‘enough’ information, and how the users deal with the information that is presented to them is not a trivial task (Carver & Turoff, 2007).

(7)

Turoff, Chumer, de Walle, and Yao (2004) propose to overcome information overload by imposing context visibility on the metaphor of the system. In other words, they suggest that the system should contain rules that prescribe which information is shown in what context. The system should allow for the user to access further information if he so chooses. The use of context visibility reduces the necessity to sift through large databases or many sources of information (Carver & Turoff, 2007). Having rules in place in the system is one method of dealing with information overload. Taking this a step further is to offer a flexible COP. This is exactly what Stiso, Eide, Halvorsrud, Nilsson and Skjetne (2013) propose. Their research has shown that users have four top demands of the information system. These are: risk assessment, triage, resource management, and response history and progress. However, they have also found that not all agencies involved in emergency management have the same demands. The medical officers prioritize triage, while resources are key for the fire and police department. Given that their system should be usable by all agencies, they figured a flexible COP was required.

The solution: to create an overlay for each of the four key factors: risk assessment, triage, resource management, and response history and progress. Each overlay allows the current user to focus on their specific information needs, while still allowing them to access all the information available. The system also aims at reducing radio contact by showing the information in a visual way. Combining these layers creates a flexible COP which, in theory, should increase each agencies situational

awareness and thus, lead to better decision making (Stiso, Eide, Halvorsrud, Nilsson & Skjetne, 2013 ; Harrald & Jefferson, 2007).

The research of Stiso et al. (2013) shows that having an information system in place to support emergency management is viewed as positive by the participating agencies. It is believed to facilitate better multi-agency collaboration. However, there is also some skepticism. Since their system, the “Master” is aimed at large emergencies which are rare, the users will not work with the system very often. To effectively use such a system however, the users need to be well aware of its

functionalities. Concerns were raised that for the system to be effective, it should be used in

everyday operations. It is stated that “future versions of the Master must be adapted and presented as a tool that can support the everyday work of emergency personnel. As an important part of this adaptation, tablet and other mobile versions of the system should be developed” (Stiso et al., 2013, p.228).

2.3 Mobile systems

Mobile devices are a relatively new technology. Nevertheless there has been some research into the use of tablet devices in emergency response, but has not been widespread adoption yet. This section focuses firstly on academic research on mobile spatial information systems, before moving on to the usage of tablets in emergency management. For this purpose we will take a look at academic sources as well as sources from the Web.

2.3.1 Mobile spatial information systems

Mobile spatial information systems have gone through an evolutionary path which started in the seventies. The first step in their evolution was the Geographic Information System (GIS), which are still used today. Early GIS focused on assembling, storing, manipulating and displaying geographical information (Star & Estes, 1990). The next step came when the Internet started gaining popularity. Combining geographical information with Internet capabilities has led to a variety of Web-enabled GIS applications (Sugumaran & Sugumaran, 2007). These applications added functionalities for rapid data analysis and presentation, while allowing rapid data exchanges between agencies. The third stage brings us to the first mobile GIS. These systems combine the Web-enabled GIS application with mobile devices, the Global Positioning System (GPS) and wireless internet access. Finally, with the combination of spatial information technology, mobile contact technology and internet technology

(8)

we can call the system a Mobile Spatial Information System (Jun-fang, Liang, Yong, & Da-sheng, 2009). These systems focus on sharing spatial data between organizations or agencies, while offering location specific information. They are designed specifically for use on mobile devices such as tablets, offering a more convenient method of information exchange while on location (Jun-fang et al., 2009). Because these systems offer a method of effectively sharing information in a visual way, they would be perfect for first responders, as a method of accessing the COP.

2.3.2 Tablet devices in emergency management

First responders require specific information on the incident, preferably before they arrive. This information is currently received through radio or telephone communications. Allowing first responders to access the COP from a device in the field eliminates the need for constant radio communication. By showing them a visual picture of the incident, mobile devices will free up time by allowing first responders to focus on the emergency response effort instead of information gathering (Kim, Jang, Mellema, Ebert & Collins, 2007).

PDAs, tablets and smartphones have all been investigated as tools in the field, and seen as possibly useful (e.g. Luyten, Winters, Coninx, Naudts, & Moerman, 2006; Kim et al. 2007; Jiang at al. 2004). Building on these papers and doing their own research to compare smartphones, tablets, laptops and a tabletop device, Andresen and Nilsson (2014) found that tablet devices are the preferred type of mobile device among first responders. Tablets were preferred due to the size of the screen, which handled showing the map and information effectively (p.118). The touch screen input method was also viewed as positive (p.118). However, first responders often wear gloves, which could cause problems with poorly designed user interfaces (p.118).

Investigation of sources from the World Wide Web seem to confirm the research we have just discussed. According to Crum (2012) emergency services in Oklahoma see large potential for tablets. They utilize a program called Geosafe, which allows the emergency control room to relay information to the vehicles. A map is used to visualize information, including the positions of the vehicles. They are even looking in adding weather forecasts to the map, much like Stiso et al. (2013). Crum (2012) concludes: “It looks like the technology could certainly be of great benefit in the long run to medical services, depending on the software used and whether or not it’s financially feasible for smaller companies. The more success is found, the better chance tablets have of doing for emergency services what the CB Radio did for them in the 70′s.”

Tablets were also used during the emergency response effort surrounding hurricane Katrina (Huff, 2013). According to a New Jersey EMS Task Force Planner his tablet was “critical” to his work (Huff, 2013). The tablet helped him to keep track of each piece of equipment, and it was key in assigning equipment and personnel to where it was most needed. Huff (2013) interviewed emergency

personnel from several agencies, and found that “for situational awareness, there is nothing better”. As we have seen, tablet devices are considered to be useful in emergency response situations, or at least have a lot of potential. Essential in the widespread adoption for these tablets is good design (e.g. Huff, 2013; Andresen & Nilsson, 2014). During the evaluation of the Disaster (VRK et al, 2014) experiment we found some concerns with the current interface and possibilities for improvement. These will be discussed in the next section.

(9)

3 Evaluating Disaster

During the Disaster (VRK et al., 2014) project a user interface was tested on tablets for first responders in the field. This experiment involved multiple agencies simulating an incident on

Schiphol airport. The incident consisted of an airplane that ‘crashed’ into a hangar. Within the hangar there was a party going on, and of course there were people on board the plane. During the incident first responders were supplied with pieces of information, some of which were communicated through their tablet devices. Measuring the effectiveness of the interactions between agencies and the use of the tablet devices were the main research goals. Evaluation of the project showed that the users were not satisfied with the way the tablet interface worked. The most important of the issues are listed below.

3.1 Requirements

From the data gathered in the Disaster experiment (VRK et al., 2014) a list of requirements has been distilled that represent the demands on the COP that are not being met in the current system. Each role in the emergency response effort during Disaster has different demands on a COP. The focus of this research is on the demands of the user interface of the tablet devices used in the field by first responders. The full list can be found in Appendix A. In the following section we will elaborate upon those requirements that have been implemented in the prototype that was developed for this master thesis research.

The prototype has been created using JavaScript and HTML5, allowing for easy testing since it can be run in any browser. Using this method also creates the opportunity to test the same prototype on different platforms, tablets and operating systems.

3.1.1 Information Validity: FRQ_1

During the Disaster experiment it became clear that some users of the system do not trust the information in the COP. Analysis of comments made by OT during Disaster shows that the validity and reliability of the information is questioned when it is communicated through an information system, although the information is trusted when it is communicated through speech, whether radio or phone. The main reason for these trust issues seems to be the uncertainty of the source of the information. When the information is represented in the system, the source is not known.

Due to the high interoperability that is aimed for in the LCMS, combining information from multiple sources, it can be difficult to solve this problem. The solution put forward here is to require users to login to the system before they can use it. This would allow for the automatic assignment of IDs to pieces of information. This method is already used in the LCMS, although not on the tablet devices. Some concerns against this method were raised during the evaluation period. Logging of the specific identities of users of the system would create a hurdle for some to input information. If the

information turns out to be wrong, they would be held responsible. Therefore the point was made that logging the specific role of the information source would be more effective.

Logging of the role of the information source, e.g. fire commander, police commander, would give an idea of where the information originated. However, this does require an amount of trust in the capabilities of every commander that has access to the system. Additionally, it would not clearly depict the location of the information source. When the specific identity of the source is known, these issues are nonexistent. Therefore the choice was made to implement a structure using unique identities for every user, which are shown in the info window for each piece of information added by the user.

(10)

Further expanding this method of solidifying information validity is to allow only certain roles to add certain information. For example, the GHOR (Dutch: geneeskundige hulpverleningsorganisatie in de regio, English: medical aid organization in the region) should be responsible for adding information about the number of wounded. When this is the case, the validity of the information should be ensured. Unfortunately it was not possible to implement this into the prototype, so no data was gathered on this subject.

3.1.2 Timeline Functionality: FRQ_3

Having correct situational awareness is crucial for decision making in emergency management (Harrald & Jefferson, 2007). Part of this situational awareness is temporal awareness, or the knowledge when certain things have occurred in the past (Gryszkiewicz & Chen, 2012). Therefore it can be very useful in these situations to have a timeline view of occurrences and information flow. Having a timeline of events, and more importantly knowing when information was added, can be helpful in planning ahead. According to Stiso, Eide, Halvorsrud, Nilsson, & Skjetne (2013) a timeline can “help users build and maintain the temporal aspects of their situation awareness” (p.226). The timeline view can also be used to effectively brief the next command shift in a long-term crisis situation (Gryszkiewicz & Chen, 2012). Additionally, a timeline of events can be used for evaluation purposes, shedding light on the reasoning behind certain decisions at certain times.

In the prototype the timeline capability shows every piece of information in the information system It shows all information, even if it is hidden from the map. Hidden information entails that the event is considered to be dealt with, which would make the information obsolete. If for example a fire is put out, the information is hidden from the plot as it is no longer of consequence to the current incident view. However, as stated before, it can be useful to still have access to this information. This is in line with the design principle to make information about past events easily accessible

(Gryszkiewicz & Chen, 2012).

These hidden pieces of information might eventually become relevant again at a certain stage of the incident. For example, a fire that was thought to be extinguished could spring back to live. To solve this problem I have added the possibility to show information on the map after it has been hidden, essentially “un-hiding” the information.

The timeline uses a JavaScript library called vis.js1. This plugin contains the required functionalities to create a working demo. After testing the timeline as it is now some things might require slight tweaks.

3.1.3 Using symbols and text balloons: FRQ_4 & FRQ_4.1

Having a screen available which shows an accurate picture of the incident can be helpful in the field. However, showing too much information at once can clog this image, and require too much time to study and determine what is going on. Finding a balance between showing the necessary information and causing information overload is a challenge.

Part of the solution to this problem is the use of symbols on the map. Using symbols is an effective way of relaying information to the user, quickly (Horton, 1994). Using symbols eliminates the need for long stretches of text, reducing the time needed to understand what information is being shown. However, symbols are limited in the amount of information they can relay. To create a clear picture of the information a method of showing more details is necessary. In the prototype we have chosen to use infowindows for this purpose. These open when a user taps a symbol on the map. This

(11)

method allows the map to remain simple, while hiding the details to the situation in plain sight. Hopefully this limits the amount of information overload the user experiences.

3.1.3.1 Placing symbols

During Disaster (VRK et al., 2014) the interface that was used to place icons was deemed ineffective. To solve this problem in the prototype a new method of placing symbols was proposed. Placing a symbol requires the user to press and hold (long-press) a position on the map for approximately half a second, which opens a menu. This menu contains several symbol categories from which a user can choose. When the user has traversed this menu structure successfully to find the symbol he requires, he can add some additional information if he chooses. The symbol is then placed on the map at the location of the long-press.

This method requires a well-designed menu structure containing the necessary icons. When working in the field during an emergency response a user has no time to search through menus for longer periods of time. Therefore we believe it would be useful to limit the amount of symbols available on the tablet to the most used ones. In the full system this could be coupled to the type of incident and its location, e.g. if the incident is on an airport, the icons should include planes, but when there is a large fire in a building this should not be the case. This question of which icons to show at which times is beyond the scope of this research. During the experiment the participants are offered a very limited number of symbols. In fact, they will be offered only the symbols necessary for completing the scenario.

3.1.3.2 Aggregating symbols

In some situations it can be useful to be able to place a symbol on top of another one. This is mostly the case when the two are related, e.g. when there are explosives on an airplane. However, showing two or more icons at the same location renders all of them illegible. There are several methods to deal with this problem. The three best (in our opinion) are shown in Appendix C. During the testing of the prototype the participants will be asked which of these they prefer, and why. They are also asked if they know a better solution.

3.1.4 Automated Vehicle Location: FRQ_6

As we have seen in the literature section, part of the network centric doctrine is creating shared situational awareness (SSA). A crucial part in creating SSA is the positional knowledge about available equipment (Alberts, 2007). Therefore, showing the locations of emergency equipment on the map should be an integral part of any emergency management tablet application. However, it is cumbersome and frankly, unnecessary, to require manual input of this information. Automated Vehicle Location (AVL) technology is easily available and effective. According to Portillo (2008, p. 5), AVL technology assists decision support and situational awareness in emergency response situations. The prototype mimics AVL by moving a fire trucks position automatically when the user places a fire symbol on the map. AVL technology is easy to understand for most users, therefore we believe that this simulation, along with some explanation, is enough for the participants to form a clear opinion on this subject.

3.1.5 Information Requests: FRQ_7

Currently the LCMS and tablet devices are used as a database of information. During Disaster (VRK et al., 2014) the suggestion was made to encourage further cooperation between agencies by allowing users to place requests for information on the map. The prototype allows for this by including an information request symbol, in this case a question mark. The user should manually input the actual question in the additional information attached to the symbol.

(12)

When the information request is fulfilled, the information containing the answer should be added to the symbol it is related to, as well as to the information request symbol. This symbol should then be hidden from the map, to keep the map as clean as possible.

(13)

4 Research design

The research done in this paper is aimed at finding out whether the requirements found from the analysis of Disaster (VRK et al., 2014) are implemented in an effective way. Answering this question requires participation from future users, in this case firefighters.

The participants are asked to follow a scenario, and perform seven tasks in the prototype, running on an Apple iPad. The iPad in question is the same one as was used during the Disaster experiment (VRK et al., 2014). When the scenario and tasks are complete, the participants will be asked to fill in the SUMI (Kirakowski, 2012) usability test. This usability test is validated and shown to be effective in measuring the following characteristics of a system: Efficiency, Affect, Helpfulness, Controllability, and Learnability.

After the SUMI questionnaire a second questionnaire is done to achieve further clarity on the specific parts of the prototype that are under investigation: the timeline, the validity of information and the placement and use of symbols on the map. Questions are also posed about the usefulness of automatically changing vehicle location on the map and information requests. The experiment was performed to find answers to the following questions:

 Is the validity of the information on the map assured?

o Is it necessary to log the identity of the information source or is the function (role) within the emergency response sufficient?

 Is the timeline functionality considered useful in the field?

 Are symbols on a map an effective method of transferring information in the field ? o Is changing the location of emergency equipment automatically useful? o Are information requests on the map useful?

o Is changing the location of emergency equipment automatically useful?

 Which of the possibilities for showing two or more symbols that are linked (shown in appendix C) is preferred?

The experiment is done with one participant at a time, with a moderator present to take notes. After the tests described above the participants will be asked if they have any comments on the adding of symbols, information validity or timeline functionality.

(14)

5 Results

In this section we will review the results found during the testing process. The test consists of two questionnaires, which have been filled in immediately after the participants have finished a preset scenario including several tasks that could normally be expected to occur in regular usage of the application.

In the following sub-sections we will review the results from the two questionnaires. The first questionnaire contains questions regarding each problem stated in the Requirements section. These questions are Likert type questions, with answers ranging from ‘Strongly agree’ (shown in the diagrams as 1) to ‘Strongly disagree’ (shown as 5). The questions will be analyzed using frequency tables and, where necessary, the mode.

These questions were controlled within the questionnaire by a similar question, only in reverse. For example, the question “The symbols represent the underlying information effectively” is controlled by the similar question “The symbols don’t represent the information effectively”. Between these questions we should find a strong negative correlation, since the answers from each participant should be opposite. For this purpose we will calculate the correlation between the two questions. Since Likert type questions result in non-parametric data we will use Kendall’s tau (τ). This is a correlation measure often used on surveys that use Likert type questions such as these. Kendall’s tau takes a value between -1 and 1, where 1 is a perfect positive correlation, and 0 means no correlation. As we expect participants to answer the opposite, we are looking for a strong, negative correlation. The closer τ is to -1, the stronger the evidence that the participants did in fact answer the opposite to the control question.

5.1 Information Validity

In the Disaster experiment (VRK et al., 2014) the validity of the information shown in the system was questioned. Having the source of the information available would potentially solve this problem. In the prototype the users are asked to login using a fictional profile. This identity is stored, and shown with every piece of information that they add. Likewise, every piece of information that is available in the system is shown with the source.

The questionnaire asked the users whether they perceived the information in the system as valid or not. For this purpose, four questions were asked. These are:

1. “Showing the source of the information is essential.” 2. “Showing the source of the information is unnecessary.”

3. “Showing the NAME and the ROLE of the source of the information is better than showing only the ROLE.”

4. “Showing only the ROLE of the source of the information is better than showing the NAME and the ROLE.”

Questions one and two, as well as questions three and four, are identical, but reversed. This is done to control the answers for both questions, to check whether the participants were reading the questions and filling them out correctly.

Figure 1 shows the frequency diagrams for questions one and two, where question one is shown in red and question two in blue.

(15)

Figure 1 - Frequency diagram of questions related to the showing of the source of the information. Looking at the modes for these questions, 1 (strongly agree) for essential and 5 (strongly disagree) for unnecessary, we can see that the common opinion is that the source of information should be shown with every piece of information. While we can see in the diagram that it is not completely symmetrical, we can suspect that most participants answered in reverse. To make sure we calculate Kendall’s tau. Doing this we find a correlation of -0,683, which is a significant correlation, proving that our participants answer as we had predicted. We can thus conclude that an emergency support system should include the source of the information.

Additionally we asked the participants for their opinion on the level of detail that is necessary in showing the source of the information. More specifically we asked them whether they preferred to see only the role of the source of the information, e.g. fire officer, or if they want to know not only the role but also his or her name, e.g. OvD_Jansen.

Figure 2 - Frequency diagram of questions related to showing the details of the source of information. As we can see in figure 2, the consensus is quite clear. It is obviously preferred to show both the

name and the role of the source of the information. Once again looking at the correlation between

these two questions we find a τ of -0,855, which is very significant.

Many participants stated that it is more important to show the identity of the source during larger incidents. When only a single OvD (Chief Fire Officer) is present at the scene, the role would be sufficient. However, as more and more OvDs and commanders arrive at the scene, the specific identity may become useful. Keeping in mind that this system will be used in both large and small incidents, showing the name and the role would be most effective.

Some people also stated that even though the name of the source should be logged for evaluation purposes, during the incident it should not matter who exactly added a piece of information. If it is known that a fire officer has added the information, everyone else should trust that this person is

0 2 4 6 8 10 1 2 3 4 5 Fr e q u e n cy Answer Essential Unnecessary 0 1 2 3 4 5 6 7 1 2 3 4 5 Fr e q u e n cy Answer

Name and Role Role

(16)

trained properly, and has added correct information. The name of the source should not be the deciding factor when choosing whether or not to trust a piece of information. However, it can be argued that for higher command structures it may be useful to know the exact source when making a decision. It may be useful knowing the location of the source, or being able to contact the source for more information. Therefore we believe that showing both the name and source would be the best solution.

Concluding this section we can state that showing the name and the role of each information source with each piece of information is the preferred method of assuring information validity in our participant group.

5.2 Timeline

The prototype uses a timeline to show the information that is available in the system. It shows both information that is shown on the map, and information that is hidden, i.e. information that is no longer relevant. The scenario involves both adding and hiding information, filling up the timeline. When the scenario is completed the participants are asked to view the timeline, and think about whether they would use it or not. They were also asked to come up with ways they would or wouldn’t use it in the field. They gave the following answers.

First of all, the question whether they would use it at all. We can see in figure 3 that the participants agree that they don’t know whether they would use the timeline. In both cases the mode is in the middle, 3. Looking at τ for this question, which is -0.891, we notice that the controlling of the question has worked properly.

However, we can see in figure 4 that five of the participants agree that the timeline gives a clear image of the timespan of the incident. On the other hand, eight participants disagree with the statement that the image created by the timeline is unclear. Looking at τ, we surprisingly see that the value is very close to 0, actually – 0.025. A correlation that is this close to 0 suggests that there is no relation between the answers to these questions. This would imply that participants who think the timeline gives a clear picture, also think that it does not provide a clear picture. This is quite strange. Looking at the data there are hints to why this correlation is so low. It appears that there are more participants who disagree with the negative statement than that there are participants who agree with the positive statement. Looking at the raw data we see that only four out of eleven participants answered exactly opposite, which explains the low correlation.

We can state that the image generated by the timeline is neither clear nor unclear, and participants are unsure whether they would use it. The timeline is filled with the information that is put in the

0 1 2 3 4 5 6 7 1 2 3 4 5 Fr e q u e n cy Answer Clear Unclear 0 1 2 3 4 5 6 7 1 2 3 4 5 Fr e q u e n cy Answer Would use Would not use

Figure 3 - Answers to the question: "Would you use the timeline in the field?"

Figure 4 - Answers to the question: "Does the timeline give a clear picture of the timespan of the incident?"

(17)

system. Over time the view becomes quite full, leaving some participants to question the usefulness. In the comment sections with the questionnaire we find multiple participants saying that: “the timeline view needs some work, I would not use it in the present state.” Some further clarification can be found in the reasons participants give for using or not using the timeline.

The answer to this question lies in the place and time where the application is used. Mostly, the application is used to create a clear image of the incident for first responders. For this purpose, the map is used, showing only the most relevant information. The timeline however includes all the information that is available. First responders don’t have the time to sift through all the entries on the timeline. A timeline would be used mainly to look back, and as one or our participants stated: “An incident only moves forward!” Therefore, when the timeline can be used to look into the future, e.g. to visualize incoming events such as extreme weather, the timeline could be useful in the field. A standard timeline containing information from the past will be most useful for higher command structures, and during evaluation of the emergency management response.

In the current form, we believe it would be best to exclude the timeline from a tablet application. The view becomes quite full, causing information overload, which is one of the main things we are trying to avoid. Timeline views can be used in higher command structures when there is more time to analyze the information.

5.3 Using a map & symbols

A major part of the prototype and emergency management support systems in general is the map, and the information that is represented on it. In the prototype specifically we wanted to see whether symbols are a good method of showing information. These symbols are enriched with infowindows that can be opened and closed. One symbol in particular we were interested in is the question mark, or ‘information request’ that a user is able to put on the map. We wanted to see whether our participants found this useful.

The map also shows the locations of emergency response equipment such as fire engines. The location of the equipment is updated live, without manual input. The movements and automatic updates are simulated in the prototype, and some questions were asked about this subject as well. 5.3.1 Information image

The first thing we need to ensure is that the map in itself creates a clear image of the incident and the information that is available. For this purpose we asked the participants the questions: “The map creates a clear information image” and “The information on the map is unclear”. The results can be found in figure 5.

Figure 5 - Participants’ opinion on the information image created by the map. 0 1 2 3 4 5 6 7 1 2 3 4 5 Fr e q u e n cy Answer Clear image Unclear image

(18)

The participants agree that the map creates a clear image. Likewise, the statement that the

information is unclear is seen as false. This is confirmed when we look at τ, which is -0,497, a strong enough correlation to say that the participants did indeed answer in reverse. We can conclude that the findings in other research that geographic information systems (GIS) are effective in transferring emergency management related information are also applicable to this research (e.g. Jun-fang, Liang, Yong, & Da-sheng, 2009; Crum, 2012; Stiso, Eide, Halvorsrud, Nilsson & Skjetne, 2013). Therefore we can state that emergency management support systems should be based around geographic

information systems.

5.3.2 Showing information using symbols and infowindows

Now that we know that using a map as a tool for information sharing creates a clear image, we can move on to the usage of symbols and infowindows to relay the actual information. Figure 6 shows that the participants agree that the symbols are clear. They agree very strongly with each other that the symbols are not unclear, allowing us to conclude that symbols are the way to go when showing information on a map. However, looking at τ gives us -0.151. This implies that some participants find the symbols both clear and unclear. From the raw data it becomes clear that there are indeed two participants who answered negatively to both questions, clouding the correlation.

Figure 6 - Showing the responses regarding the clarity of the symbols.

This might be caused by some participants that stated that although they believe that symbols are very effective in transferring information it can’t be the only thing on the map during an emergency situations. Lines and circles are crucial as well. Allowing first responders to draw a circle around an incident to indicate the effect area can be extremely useful. This allows higher command structures to decide on evacuation plans and other necessary measures. Drawing lines can clarify which is the best approach path for arriving personnel.

Another point that was made is that symbols are great for showing information, but limited in the amount they can transfer. Additionally, the information behind a symbol is always the same, e.g. a symbol for explosives means that there are explosives. The symbol itself can’t relay the amount or type of these explosives. As a solution to this problem and to allow users to add detailed information to a symbol, we introduced infowindows. The participants responded extremely positively. According to tau, the participants answered in reverse: -0,571.

0 2 4 6 8 10 1 2 3 4 5 Fr e q u e n cy Answer Clear Unclear

(19)

Figure 7 - The answers regarding the convenience of infowindows.

Figure 7 shows that all participants replied positively to the usage of information windows. On the control question the answers are similarly homogenous, every participant disagrees or strongly disagrees that the usage of infowindows is inconvenient.

5.3.3 Aggregated Symbols

Sometimes it may be necessary to place two symbols on one location. For instance, in the scenario explosives are found aboard the airplane. When the user adds a symbol for explosives on the airplane, it is shown as in Appendix C – Screen 1. However, there is no visual cue visible on the map in this method. The participants were asked whether they preferred it in this way, or if they

preferred Appendix C – Screen 2 or Appendix C – Screen 3. Figure 8 shows their answers.

Figure 8 – Respondents’ preferred method of aggregation is Appendix C - Screen 2.

We can see that the participants prefer Screen 2. From their comments it can be derived that they find the visual cue on the map essential. However, many like that the explosives icon is also shown in the infowindow, along with extra information. Screen 3 is also considered a viable option, but it is feared that the map view would become crowded when this method is used. The best option is combining Screen 2 with the infowindow from Screen 1.

5.3.4 Hiding symbols

We have seen that using symbols on the map is an effective method of showing incident information. As time passes, the symbols might become irrelevant, such as when a fire is extinguished. When this happens the information should be hidden from the map to keep the image up to date. However, this information might be useful for making decisions higher up the command structure, and should therefore not be removed from the system. In other situations the symbol might have to be put back on the map. A fire resurging would be such a situation. To cater to these demands the prototype allows for symbols to be hidden from the map, while remaining visible on the timeline.

0 2 4 6 8 1 2 3 4 5 Fr e q u e n cy Answer Convenient Inconvenient 0 2 4 6 8 1 2 3 Fr e q u e n cy Type of Aggregation Preferred method

(20)

Figure 9 shows the participants’ opinion on hiding information from the map, by using a button in the infowindow attached to the symbol. We see that this method of hiding a symbol is considered clear and simple. This notion is enforced by the strong negative correlation, -0,463.

The method of replacing a hidden symbol from a button on the timeline view is also considered clear, as we can see in figure 10. On this subject however there is a bit more discussion. Some participants stated that instead of replacing the symbol on the map when a fire resurges, they would prefer to place a new symbol. According to them the situation is different from the one before, and they consider it a new fire. Whether operators in the field should be able to remove and replace symbols could be a point for debate, and needs further research.

5.3.5 Automated Vehicle Location

This section of map related results concerns the automated vehicle location (AVL) functionality. As we have seen before, AVL uses GPS to update the location of emergency equipment in real time. This is simulated in the prototype by changing the location of the equipment dynamically, when a user adds a certain symbol (for further information see the scenario, appendix B). Unlike a fully

functioning system this location is based on the location of the users’ input, but it illustrates the functionality.

Figure 11 - Responses concerning AVL.

The results shown in figure 11 are as expected. Having the location of equipment known at all times is crucial in emergency management. Therefore, having this information available automatically is very useful. The results reflect this sentiment. Strangely enough we find a τ of -0,177. Playing around with the data it becomes clear this low correlation is cause by two participants. These two answered

0 2 4 6 8 10 1 2 3 4 5 Fr e q u e n cy Answer Convenient Not convenient 0 1 2 3 4 5 6 1 2 3 4 5 Fr e q u e n cy Answer Clear 0 1 2 3 4 5 6 7 8 9 1 2 3 4 5 Fr e q u e n cy Answer Clear Unclear

Figure 10 - Responses regarding the hiding of symbols using a button in the corresponding infowindow.

Figure 9 - Responses regarding the showing of symbols using a button in the timeline.

(21)

“1 – Strongly agree” to both questions. When these results are removed we find τ = -0,730, which is a more representative result.

However, there were some interesting points made regarding this functionality. For instance, large incidents may include huge numbers of vehicles. It could be useful to filter these per agency, or show only those vehicles near the users’ location. These options should be investigated when this

functionality is implemented in an emergency management support system. 5.3.6 Information Requests

The emergency management support system being used by emergency personnel in the Netherlands serves almost completely as a sort of database for information, allowing the users to view this information and add to it. However, there are no possibilities for cooperation. Something that came up during Disaster (VRK et al., 2014) was the possibility to add an information request, i.e. to ask a question. In the prototype this is done be placing a question mark symbol on the map, and putting the question in the infowindow.

The participants were asked whether they would use such functionality. Their answers are fairly positive, as we can see in figure 12. Seven out of eleven participants proclaim that they would use this functionality if it was available. On the other hand, only two out of eleven participants state that they would not use it. The opinions on this subject are somewhat spread, and we see that reflected in the correlation, which is quite low at -0,193.

Figure 12 - Responses concerning information requests.

There are however some points to consider when implementing such a functionality. One can argue that information requests done using voice communications are quicker and more effective.

However, some questions can’t be answered immediately. Using the system for storing such questions can be useful.

However, placing the question mark symbols on the map might not be the most effective method of handling information requests, as it fills the view rapidly. It might be better to keep a separate screen containing the information requests. This screen should allow users to filter according to the type of question, and which agency it is related to. Knowing which agency it is related to is also crucial in finding the answer rapidly. This screen also allows users to see if there is a question they can answer.

5.4 SUMI

The SUMI (Kirakowski, 2012) questionnaire consists of 50 questions with answers either ‘Agree’, ‘Disagree’, or ‘Don’t know’. It is aimed at measuring the behavioral disposition of end users towards an information system or prototype. The questionnaire measures five properties of the system being tested. These are:

0 1 2 3 4 5 6 7 1 2 3 4 5 Fr e q u e n cy Answer Would use Would not use

(22)

 Affect – The users’ emotional reaction to using the software. In other words: how well the user likes the system and working with it.

 Efficiency – Whether the user believes the system aids him in his tasks, and does this in a clear and effective way.

 Helpfulness – Measures the degree to which the software is self-explanatory, as well as more specific things like the adequacy of help facilities and documentation.

 Control – How much the user feels in control when working with the system.

 Learnability – Measures the speed and ease with which the users can learn to use the system.

 Global usability – Measures the general usability of the system. Derived from the answers to 25 out of the 50 questions, giving a better measure of general usability than taking the average of the other measures. (Kirakowski, 1994).

SUMI is designed in such a way that most systems will receive a score within one standard deviation of the mean, which is between 40 and 60. Programs with a score higher than 60 are by definition well above average. As we can see in the ‘Mean’ column of table 1, our prototype is on the upper limit of this range.

Mean St

Dev

Median IQR Minimum Maximum

Global 58.18 4.98 59.0 10.0 51 65 Efficiency 59.55 3.72 59.0 5.0 53 67 Affect 57.73 8.95 58.0 12.0 45 72 Helpfulness 54.64 6.39 54.0 10.0 45 65 Controlability 56.64 7.38 57.0 12.0 47 69 Learnability 59.27 7.21 61.0 10.0 46 69

Table 1 - Aggregated results from the SUMI questionnaire.

We see that the prototype scores higher than average on all of the scales. The lowest score,

Helpfulness, can be explained by the fact that there was in fact no ‘help’ functionality available in the prototype. This lack has led to a large number of ‘Don’t knows’, leading to a lower score.

Starting on the left we see that Efficiency has the highest mean paired with a low standard deviation. This entails that the answers in the efficiency category are not very widespread, leaving us free to conclude that the efficiency of the prototype is considered high. Moving on to the Global Usability Scale we find that the prototype’s usability is seen as above average.

(23)

Figure 13 – Visual representation of the results from the SUMI questionnaire.

(24)

6 Conclusion

Tablet devices can be useful tools during emergency management situations. Incidents are high pressure situations, and require applications that are well designed. During this research we have tried to come up with effective solutions for some of the problems that arise when using such systems.

The goal of the information system used during emergency situations is to share information, and create an accurate, shared picture of the incident, the COP. The point of this picture is to provide an instant understanding of what is going on. For this to be effective it is necessary that the systems providing this COP are simple to use. We found that this is the most important part of any emergency management support system, and any tablet application supporting it. To keep the application simple it is necessary to mirror it to technology the users might have at home. Therefore we encourage the usage of consumer technology such as Apple iPads, Microsoft Surface or Android tablets. This will ensure that the technology isn’t frightening to users, but feels intuitive.

Moving on to the tablet application itself, we have found that an interface using a map is viewed as positive. To create an image of the incident on this map, use symbols which can be enriched with infowindows. Allow infowindows to be opened, closed and hidden when necessary. The users should be able to add and hide symbols with minimal effort. Having a menu pop up on the location where the user presses down on the map has been shown to be an effective method. However, this

requires the icon menu that appears at said location to be well structured. Users have limited time to search through such a menu. Therefore we suggest that these symbols should be chosen

beforehand, with a structured set for each type of incident. This requires some form of protocol to be created, but will save time during the actual incident. Higher command structures using

applications on more advanced machines such as laptops can have access to more detailed icon menus when necessary. In addition to these symbols the users should be able to place lines and circles on the map, allowing them to show areas and approach routes. These are an essential part of effective incident management. Users should be able to add infowindows to these items as well. Removing information from the map is another potential problem. When first responders have access to the COP and can make changes to it, the possibility exists that they make mistakes. These mistakes can be made during the placing of symbols, but also in removing them. Especially when a user has placed a symbol with extra details in the information window, it can be an issue if this is accidently removed. In the prototype we solved this by calling removing information hiding. Hiding implies that it can be found, and for this purpose we included a button in the timeline view.

However, we don’t recommend including a timeline view in a tablet application. Therefore, another method of replacing hidden information must be found. The most effective method, in our opinion, is to allow an Information Manager in a higher command structure access to all removed information. Information managers are already available in CoPI and OT during incidents in the Netherlands, so this solution will require only a slight adaption of their tasks. Hopefully, this functionality won’t be used often.

The other users of the system that are not first responders, e.g. CoPI and OT, should also have access to this COP. For them it can be essential to know who has entered the information. This allows them to check up on the information to find out more, as well as having a better idea of the validity of the information. The participants to our research made very clear that showing the source of a piece of information in the system is essential to the usefulness. Requiring first responders to login to their machine before they are allowed to add any information is a quick and simple solution to this problem.

(25)

Having information available on a map view gives a clear picture of the incident. This view also has its limitations though. There is no way of looking forward or backwards on a map. For this purpose a timeline view of the same information was proposed. We have seen that this view is not optimal for first responders. A timeline view of a situation can be useful but requires some time to fully

comprehend, unlike the map which is clear immediately. A timeline is more suited for higher command structures that have more need for a complete picture of the incident, where first responders require knowledge of what is going on NOW.

A timeline view can also be used to look forward. In certain situations there might be demand for information that is known about future events, such as weather. Some protocols also include time limits on actions. The timeline could aid in keeping an eye on these time limits. Whether or not these functionalities are included in a timeline view doesn’t change that first responders will most likely not have time to use it.

Having a clear image of the incident allows first responders to have an understanding of the situation before they arrive, creating a more aware situation. When the first responders have arrived at the incident they can use the system to find further information. For this purpose it is useful to allow for some type of filtering in the system. For example, filtering the symbols for each agency specifically, e.g. fire department. Allowing for this filtering also keeps the map view clean of less vital

information.

Keep in mind during the development of these systems that they should remain in the background. During an incident first responders want the information as quickly as possible, with as little effort as possible. These systems can be great for this purpose, but it should not be the main task of first responders to fill these systems. First responders are there to deal with the incident, not serve as information gatherers. When these two functions can be combined effectively, the implementation of these systems and tablet applications can be a great success.

During the course of this research it has been made clear that designing an effective tablet application for emergency management is not a trivial task. First responders are often put in high pressure situations with little time to interact with an information system or tablet. Emergency personnel also have a hard time commutating their desires for such a system, since they have a limited technical background. The high pressure context, combined with the fact that users often wear full gear including oxygen masks and gloves present very specific demands on these systems. To find out what these are, it is crucial to create prototypes and test these with the potential users. Finding a good balance between ease of use for first responders and the decision making capabilities required by emergency managers remains an interesting scientific question.

(26)

7 Future Work

The findings in this research are a good start for developers of emergency management tablet applications. Keeping these findings in mind when creating a tablet application will lead to more user centered design and satisfied users. However, there are some questions that remain unanswered, or that came up during this research. The following section will discuss the most prudent of these issues.

Due to time constraints during this research we focus on the fire department, ignoring other agencies that might be involved in large incidents. Whether the findings of this research can be generalized to other agencies should be further investigated. However, the research done in this thesis is a good step towards good design in tablet applications for emergency management. Keep in mind though that these tests are done in a stress-free, controlled environment, without the pressure of an actual incident. The next step is to do a simulated incident with a system that takes our findings into account, to test the solutions in a stressful situation. When such a test is completely successfully and the findings are once again positive, the next step is to create a fully working application that is connected to a back end, and test it during real incidents.

Since speed in information exchange is a main goal of these systems, it might also be interesting to do some kind of cognitive experiment to investigate the time it takes for users to understand what they see. For instance, two participants of roughly the same cognitive ability are shown the same information but in different format. One is shown the map view containing the information represented by symbols. The other is shown a timeline view of the same information. Both participants have limited time to look at the image, let’s say five seconds. We then compare their understanding of the situation by asking them a standard set of questions, and analyzing the answers. Using this method we could also test different symbols and symbol menu structures. Further research is also necessary into the usage of these systems as tools for cooperation. The functionality that enables users to ask questions within the system was viewed as interesting, but due to the local nature of the prototype it was impossible to test this effectively. Having participants solve a simulated emergency using a system that allows for such cooperation can lead to more clarity on this matter.

(27)

8 References

Alberts, D. S. (2007). Agility, focus, and convergence: The future of command and control. OFFICE OF THE ASSISTANT SECRETARY OF DEFENSE FOR NETWORKS AND INFORMATION INTEGRATION WASHINGTON DC.

Alberts, D.S., Garstka, J.J., Hayes, R.E., Signori, D.A.(2001). Understanding Information Age Warfare. CCRP, Publication Services, Washington.

Andresen, L. K., & Nilsson, E. G. (2014). Finding the best devices for emergency responders in Norway–an empirical study. In S.R. Hiltz, M.S. Pfaff, L. Plotnick, and P.C. Shih, (Eds.),

Proceedings of the 11 th International ISCRAM Conference. University Park, Pennsylvania,

USA.

Carver, L., & Turoff, M. (2007). Human-computer interaction: the human and computer as a team in emergency management information systems. Communications of the ACM, 50(3), 33-38. Crum, A.. (July 19, 2012). Will Tablets Change Emergency Services Forever?. In WebProNews.

Retrieved June 20, 2014, from

http://www.webpronews.com/will-tablets-change-emergency-services-forever-2012-07.

Eppler, M. J., & Mengis, J. (2004). The concept of information overload: A review of literature from organization science, accounting, marketing, MIS, and related disciplines. The information society, 20(5), 325-344.

European Union (EU). (2012). Disaster. Retrieved 17-6-2014, from http://disaster-fp7.eu/node/21. Gryszkiewicz, A., & Chen, F. (2012). Temporal aspects in crisis management and its implications on

interface design for situation awareness. Cognition, Technology & Work, 14(2), 169-182. Gunes, A. E., & Kovel, J. P. (2000). Using GIS in emergency management operations. Journal of Urban

Planning and Development, 126(3), 136-149.

Jiang, X., Chen, N. Y., Hong, J. I., Wang, K., Takayama, L., & Landay, J. A. (2004). Siren: Context-aware computing for firefighting (pp. 87-105). Springer Berlin Heidelberg.

Harrald, J., & Jefferson, T. (2007). Shared situational awareness in emergency management mitigation and response. In System Sciences, 2007. HICSS 2007. 40th Annual Hawaii International Conference on (pp. 23-23). IEEE.

Horton, W. K. (1994). The icon book: Visual symbols for computer systems and documentation. John Wiley & Sons, Inc..

Huff, R.. (January 24, 2013). Tablet Devices Have Many Benefits for EMS Providers. In Jems. Retrieved June 20, 2014, from

http://www.jems.com/article/technology/tablet-devices-have-many-benefits-ems-pr.

Human Factors Integration Defence Technology Centre. (2009). Common Operating Pictures and their Potential for Multi-agency Work (HFIDTC/2/WP3.1.4/4). Retrieved from

http://www.hfidtc.com/research/multi/multi-reports/phase-2/HFIDTC-2-3-1-4-4-common-op-pictures.pdf

Designing a Tablet Application for Emergency Management