Coupling Game Theory and Discrete-Event Simulation for Model Based Ambulance Dispatching

Coupling Game Theory and Discrete-Event

Simulation for Model-Based Ambulance

Dispatching

Author:

XinYu F

Daily Supervisor:

dr. Sergey V. K

Examiner:

dr. Valeria K

Assessor:

dr. Michael L

A thesis submitted in fulfillment of the requirements

for the degree of Master of Science in Computational Science

in the

University of Amsterdam

ITMO University

Patients in critical medical conditions need professional facilities, and usually, the disease occurs in an unexpected situation, therefore properly allocating medical resources becomes es-sential for them. Particularly, for acute coronary syndrome (ACS) patients in the metropolitan areas of Saint Petersburg, Russia, overcrowding is caused by the irregular inflow of patients and the limited resources of angiographies (a medical imaging technique requires profession-als and equipment). An applied solution is that overcrowded hospitprofession-als (or emergency depart-ments, ED) can divert incoming patients to other hospitals. During the redirecting process, stakeholders such as hospitals, patients, ambulance squad, emergency medical service (EMS) and city authorities are involved. Thus, decisions on the collaborative management of pa-tient flow form a non-cooperative game (a game with competition between individual players) which is incentivised by stakeholders’ self-enforcing operations. Consequently, the social opti-mum of health outcome is hardly achieved. Thus, the demand for investigating the behaviours of stakeholders is emerging.

Because of the complex interactions among stakeholders and global regulation mixed with local decision-making, the behaviour pattern of stakeholders is complicated. Moreover, the complexity is increasing because the level of patients inflow is irregular; the real-time traffic condition is affecting the delivery time; the hospitals’ strategies are dynamic. To consider the above factors, we suppose there are global mechanisms of regulation: 1. Queuing theory is guiding the process of patient inflow 2. self-enforcing operation derived from game theory are guiding the hospital’s actions. As a result, the mechanisms are influential to the social metric of health outcome (e.g. global mortality).

In general, the research sets up a GT-DES model coupling the game theory(GT) and dis-crete event simulation(DES) for analysing the behaviours of patients and hospitals.The sim-ulation assumes hospital acts for the sake of a balance between its own lower mortality and more patients being served. The model can be used to demonstrate the process of patient in-flows, reveal the predictive strategies of hospital individuals and to find Nash equilibrium. Notably, based on the real data and consultation from medical professionals in Saint Peters-burg, a real-world case about modelling ambulance dispatching of ACS patients by GT-DES model is studied. The datasets contain the information of eleven 24-7 hospitals located in Saint Petersburg, 5124 ACS records, 1866 records of serving time to stent patients during ten months and 1,310,263 node-to-node travelling paths in Saint Petersburg.

network model and game-theoretical approach to a GT-DES model for a two-dimensional sce-nario. In which, a sensitivity analysis (Sobol Indices) is performed to explore the importance of the factors. Lastly, a case about simulating the ambulance dispatching of ACS patients in Saint Petersburg is studied based on the real data. In which, a connection between the mortal-ity and door-to-balloon time is constructed for mortalmortal-ity estimation. In addition, the model is validated by the comparative analysis of the simulated mortality and observed mortality using Pearson correlation test. Also, the result among 130 possible strategy combinations shows, the strategy combination predicted by GT-DES model has the highest correlation to the observed mortality.

In conclusion, the decision-making process of hospitals is mainly influenced by the level of patient inflow and the amount of medical resource owned by itself and nearby hospitals. As a result of interactions among all hospital agents, a Nash equilibrium can be formed, and it is possible to be predicted by the game-theoretical model. Lastly, the case studied shows a practical significance in modelling ambulance dispatching for specific patients and can be extended and applied to other city environments.

CONTENTS Abstract 1 Introduction 2 1.1 Research question . . . 2 1.2 Methodology . . . 3 1.3 Research contributions . . . 4 1.4 Thesis outline . . . 4 2 Literature Review 6 2.1 Modelling simulation for emergency medical service operations . . . 6

2.2 Discrete event simulation . . . 7

2.3 Game theory for ambulance dispatching . . . 9

3 Methods and Model Development 11 3.1 Conceptual model . . . 11

3.2 One-dimensional analytical queuing-network model (QNM) . . . 13

3.3 Two-dimensional stochastic GT-DES model . . . 17

3.4 Implementation details . . . 21

3.5 Scenario design . . . 22

3.6 Sensitivity analysis by the Sobol method . . . 26

4 Experimental Study and Results 30 4.1 Simulating with artificial settings . . . 30

4.1.1 Warm-up simulation . . . 30

4.1.2 Result of GT-DES model with multiple hospitals . . . 33

4.2 Simulating ambulance dispatching of ACS patients in Saint Petersburg . . . 39

4.2.1 Background . . . 39

4.2.2 Data description . . . 40

4.2.3 Scenarios . . . 41

Modelling ACS patients in Saint Petersburg . . . 41

Local simulation with isolated hospitals . . . 49

Global simulation with all hospitals in Saint Petersburg . . . 49

4.3 Interpretation, discussion and validation . . . 52

5 Conclusion and Future Work 58 5.1 Conclusion . . . 58

5.2 Possible futurework . . . 59

A Supplementary Results 61 A.1 Probability of Nash equilibrium for all hospital pairs . . . 61

List of Figures 63

List of Tables 67

1 INTRODUCTION

Recent evidence [36] suggests that an of particular concern in the health-care system is imbalance: some hospitals are always overcrowded while the others are preferably free. For instance, according to our data about 13 hospitals in Saint Petersburg, the northern region of the city has less density of inhabitants but more hospitals compared with the southern part. It may happen, when the hospitals in the south are in full load, the hospitals in the north are working cosily. Typical reasons can be the increasing number of visits to the emergency department (ED) and decreasing facilities in ED [30]. As a result, the imbalanced allocation of medical resources may lead to economic waste, health waste and loss of public trust.

Generally, the simplest solution to overcrowding is to increase the capacity of the resource. However, R.A. McCain’s study [28] proposed a verified Nash equilibrium hypothesis for over-crowding issue, and they suggested that increasing ED capacity has a fewer impact. Thus, a practical solution is that EMS distributes patients to hospitals reasonably from a global per-spective. Another solution is to allow redirecting the ambulance fleet by the hospital receiver. For instance, overcrowded hospitals can redirect patients’ requests, so that EMS can arrange the patients to another preferably free hospital. Theoretically, all the hospitals in the system can share the loads properly [8]. However, some evidence shows that social optimum may not be achieved in this decentralised ambulance-dispatching system due to incentives of stakeholders (i.e. hospitals, patients, ambulance squad, emergency medical service (EMS) and city author-ities) [8]. Thus, the demand for investigating the behaviours of stakeholders in a ambulance-dispatching system is emerging.

1.1 Research question

The ambulance dispatching system (ADS) is a system that helps optimise the medical re-source allocation by properly scheduling urgent patients to a medical facility [3]. In which, emergency medical services (EMS) operates as the coordinator to answer an emergency call and assign an ambulance fleet to the emergency site, and transport patient to a medical facil-ity if the patient requires. Emergency department (ED) is a typical medical facilfacil-ity providing

acute care to scheduled patients from emergency medical service which is usually attached to a hospital and medical centre.

The ambulance-dispatching system can be categorised into two groups: centralised and decentralised network. A centralised ambulance-dispatching system suppose scheduling and redirecting the incoming patients are dominantly guided by emergency medical services (EMS), in which EMS functions as a centre to assign the proper ED to incoming ambulance fleets. In realistic, ambulance-dispatching systems in many cities are decentralised, in which the EMS works as a message-passing agent.

An underlying problem is whether the hospitals are willing to sacrifice their interests (be-cause redirecting incoming patients usually results in patient lost). Consequently, system per-formance may not be optimised. For instance, a recent paper [8] claimed the same situation and found a Nash equilibrium among EDs. Also, [29] mentioned that in West San Fernando Valley, hospitals lose money when profitable ambulances turned away. In other words, medi-cal facilities use defensive equilibrium to protect themselves from turning incoming ambulance to other medical facilities. Likewise, we have seen functional similarities in dispatching acute coronary syndrome (ACS) patients in Saint Petersburg. The specialist explains that when a hospital receives a request from the emergency medical service (EMS), the hospital will accept patients as much as possible if only the financial profit is considered. However, as a humanist organisation being responsible to the society, the health outcome of the patient is also vital. Thus the current capacity of medical resources and ability would be taken into account while deciding to make sure the patient will have high-quality salvation. It turns out there is a little corporation between nearby hospitals so-called as a non-cooperative game in game theory.

The research goals will be the following: (1) to observe and simulate the process of

pa-tient inflow such as arrival, travelling and serving process. (2) to predict hospitals’ actions in a non-cooperative game by Nash equilibrium, and to dig out the transition pattern of predicted actions. (2) to reason the factor importance resulting in the hospital’s action tran-sition. (3) to study the practical use of the GT-DES model by applying it to a real-world case.

1.2 Methodology

In the thesis, we will propose a game theoretical(GT)-discrete event simulation (DES) model. In this paper, an analytical and stochastic queuing-network model (QNM) princi-pally based on DES are firstly deployed for modelling arrival, travel and serving process of patients[4] [22]. What is more, a game-theoretical approach is constructed to predict hospital actions by searching for the pure Nash equilibrium.

The proposed GT-DES model is set up as a universal framework, meaning we put more effort to make the model more simple, general and extendable. At the same time, many exper-iment feasibility and possible scenarios in realistic will be discussed and reasoned. Although the prototype of our model is based on the health-care system in Saint Petersburg, Russia, we believe it could be modified to adapt to a new health-care environment with less effort.

1.3 Research contributions

Two main contributions from the thesis are

1. A generalised model coupling game theory and discrete event simulation (GT-DES) to model decentralised ambulance-dispatching system.

2. A case study about modelling the ambulance dispatching of ACS patients in Saint-Petersburg, and predicting hospital strategies with the support of real-world data and consultation from medical professionals.

1.4 Thesis outline

FIGURE1.1: A modelling and simulation life cycle applied to structure a computational model [16]

This thesis gradually explains the model following the model-simulation life cycle as shown in Figure.1.1.

1. System we studied are introduced in Chapter 1, which introduces research background and goals and Chapter 2 which is the relevant literature .

2. Conceptual model is introduced in Section 3.1

3. Computational GT-DES model is introduced in section 3.3 4. In simulation, a sensitivity analysis is performed in section 3.6

5. Chapter 4 demonstrates the simulation results and the practical case study

6. Eventually, we come back to the system we studied in Chapter ?? by concluding our findings and possible future works.

2 LITERATURE REVIEW

This chapter has included related works and literature review gathered while doing the project.

2.1 Modelling simulation for emergency medical service operations

Over the past ten years, many studies are involved in developing computational mod-els to answer ambulance dispatching questions. A critical element in ambulance dispatching system is emergency medical service (EMS); it is a pre-hospital component conducts activi-ties including screening incoming call for emergency purpose and scheduling transportation of ambulance fleets. Gradually, the trends to EMS problems were transiting from strategical, tactical decision level to operational decision level because of growing computational tech-nology[5]. Recently, considerable operational research (OR) literature illustrated models for short-term EMS decision problems such as 1) scheduling more than one ambulance fleet to a call to reduce service time. 2) assigning proper medical facilities to patients to reduce the time consumption in transportation and medical transmission. 3) switching target medical facility to another one (which usually is preferably free) as redeployment strategy [18]. This review focuses more on short-term decision problem in OR level research conducted by EMS because the project is real-time ambulance relocation which is in this area.

The emergency medical services (EMS) can be classified into two systems: anglo-American and franco-German EMS [10]. The simulation method was usually initially developed for one of the systems. However, it is possible to form typical EMS operations firstly. In which, a typical EMS operations consists of following steps: (1) arrival of an emergency call, (2) call screening, (3) vehicle dispatching, (4) vehicle travelling from its current location to the emer-gency site, (5) on-site treatment, and (6) patient transportation to a health facility if required [5].

For a computational model, the performance measure is a crucial element for the specific problems. Time consumption, survival rate and economic costs are the main aspects of perfor-mance measure for an ambulance dispatching system. A typical example of time-consumption

measures is using average response time (defined as the duration between the recital of call and the first arrival of ambulance fleet) [17][41]. The survival rate is a straightforward measure for the death-cause disease. However, it is less used in research due to the difficulty to describe it quantitatively.

[18] proposed three critical characteristics for a successful EMS operations model: the ar-rival distribution, the geographical distribution and the priority of calls. However, geograph-ical distribution varies in different cities and the priority categorisation changes as well. The arrival distribution can be seen as a result of the Poisson process or extracted from real-world data [13]. The geographical distribution has a crucial impact on travelling time. [41] uses a sim-ple mathematical method to compute the distance in time by Euclidean distance. But recent researches involves data capture technologies such as GPS (global positioning system) for de-tailed routes since traffic conjunction and road complexity are also explained by GPS distance. A defect to be considered is ambulance has high road priority.Other things to be considered while setting up an ADS is delivery efficiency. Because it is critical in reducing mortality and disability rates [25]. Coverage and response time are two vital factors for valuate delivery ef-ficiency [31]. [25] also suggested that two significant factors to ambulance relocation in the ambulance-dispatching system are the fleet size and hospital locations.

In many models, the principle of proximity is used as the main dispatch rule, such as [47] [23]; where patients were sent to the nearest available base for treatment if transportation required . The reason is that the principle of proximity is simple and practically used. Yet sometimes, "the nearest base" may not accept patients for reasons such as lacking medical resource, expected waiting time is too long [12], a redirection or relocation of ambulance fleet will be conducted by together EMS and the assigned medical facility in a decentralised system. Besides, Cheng Siong Lim’s paper [25] introduced multiple ambulance dispatch policies and dynamic ambulance relocation models from the perspective of dispatch policies. It turns out with applying dispatch policy, response time to urgent calls is reduced.

2.2 Discrete event simulation

One method for quantitatively conducting input–throughput–output analyses for patient flows in ED is through detailed computer simulation[6] In practice, techniques such as decision analysis, Markov process, mathematical modelling, system dynamics, agent-based modelling (ABM) and discrete event simulation (DES) have all been applied in modelling the heath-care system; discrete event simulation (DES) and agent-based modelling (ABM) are the most com-monly used techniques for modelling patient flow and interactions between players. Discrete event simulation (DES) is a technique for modelling a system that can change at a point in

time. The agent-based model (ABM) simulates the interactions between agents where agents are evaluated as a combination of attributes and entities. The main difference between them is the level of perspective. DES is an event-driven simulation, but ABM focuses more on in-teractions between agents. Therefore, normally DES answers questions such as estimating the waiting time and queuing length, while ABM reveals the pattern after agents interact with each other.

Discrete event simulation (DES) has been developed to allow modelling of discontinuous systems by defining activity as a network of interdependent discrete events [35] The analytical methods do not solve the discrete events, but using a computer program to run the model by generating random numbers (e.g. by Monte Carlo method) as numerical methods. The simulation of the system arises from the assumptions of the model, and the researchers collect and observe operational data to analyse and estimate the performance of the real system [4].

DES is now widely applied in various fields. It is used in production, logistics, military and other areas, and has played a decision-support role. In the area of modelling the health-care system, discrete event simulation has its unique advantages. Firstly, DES integrates the random factors in the operation of the ED and patients. The processes of the model can reflect the running state of the ED and the patient flow, and provide the critical system performance with related data, such as staying time, waiting time of patients, the capacity of resources in ED, and its utilisation. The construction of the DES model can also deepen the understand-ing of the health-care system. Runnunderstand-ing simulation experiments can virtually change the ED’s structure or inputs without changing the actual operating conditions of the ED.’ For this reason discrete event simulation is a popular and useful decision-making tool in the field of modelling a health-care system in recent years. Furthermore, the simulation results can be widely used in practice, such as reducing the cost of medical services and helping to improve patients satisfac-tion. An example for modelling emergency medical services and emergency departments that Sun Xu [43] suggested is a queuing model (principally based on discrete event simulation) sim-ulate the emergency-care delivery system. A continuous-time Markov chain is acquainted for the network model. Besides, LG Connelly and AE Bair [6] proposed a system platform named Emergency Department SIMulation (EDSIM), a newly developed DES model of ED activity in an academic Level 1 trauma centre. The research novelty is EDSIM is that DES-based simula-tion primarily concentrates on "patient path" in ED. The patient path is defined as a series of individual activity which has to be to completed before discharge. EDSIM simulates continu-ally queuing jobs with prioritisation. With the help of the understanding of staffing levels, fa-cility characteristics and patient data drawn from electronic patient tracking databases, billing records. The model predicts average patient service times within 10% of actual values. An additional scenario that should be carefully examined and simulated in DES is overcrowding.

When in overcrowding, an ED can request the emergency medical services (EMS) agency to divert incoming ambulances to neighbouring hospitals, a phenomenon known as "ambulance diversion." [8] However, A non-negligible issue to consider is when and in what condition the ED should divert patient to other EDs. Sarang Deo and colleagues propose a possible solution to this problem; they decided the redirecting threshold by the sum of the capability of ED and average patients queuing in ED if there is a vacancy in ED. So that the policy could pool the resources of many ED. Not only it optimises the benefits and easily practicable, but also it is proven to be a Pareto improvement.

2.3 Game theory for ambulance dispatching

Game theory, a concept founded in 1944, is an analytical method for the actions of play-ers in an interactive situation [32]. Game theory is a theory based on assumptions of rational choice and focusing on interactive decision making—has the potential to provide models of the consultation that can be used to generate empirically testable predictions about the factors that promote quality of care [44]. Game theory provides a supplementary means to expli-cate optimal rational strategies in situations where the actual outcome depends on the choices of the ED[9]. A game-theoretical analysis is usually performed using rigorous mathematical models to consider the optimal decision-making problem under certain conflicting conditions. Nowadays, game theory is frequently applied to in-depth analysis under various social and economic options, as well as a health-care system. And it has obtained fruitful research results in multiple disciplinary fields [1] [48].

As an analytical tool, game theory is used to help people observe and understand the interactions and relationship between players. As a method and technology to guide the for-mulation of health-care policies, the game-theoretical model mainly analyses the policy envi-ronment through game theory and predicts and explains the behaviours of policy recipients, city authorisations and related organisations.

When formulating a heath-care plan such as an ambulance-dispatching rule, the city au-thority must consider its own sake (social optimal), as well as the purpose of the relevant recipients. The sake usually consists of economic and social benefits. Game theory provides a way to demonstrate the final-interest balance among players.

The games can be categorised into two groups: cooperative game and non-cooperative games. It is commonly assumed that modelling the centralised system as a cooperative game and decentralised system as a non-cooperative game [26]. Ideally, non-cooperative games only have competition. And most of the policies in the non-cooperative game can optimise the scheme by seeking the Nash equilibrium solution of the policy game [46]. [8] founded the

ambulance diversion has defensive equilibrium as a non-cooperative-game system. Where rerouting patients could not reach social optimum because ED aims to minimise their own waiting time instead of global waiting time. In the proposed GT-DES model, we model the decentralised ambulance dispatching system as a non-cooperative game.

As a comparison, cooperative-game ADS suggests that the EMS is the only rule-maker and can arrange the diversions of the ambulance on behalf of the efficiency of the system [8]. There are two advantages compared with decentralised ambulance diversion; one is that the system will have less chance to reach defensive equilibrium. What is more, the waiting time is indeed reduced. By considering the distance between ED, we can optimise the global time. Likewise, the system reached Pareto improvement. Extensive research has shown that the game theory model is a feasible way to simulate the cooperative actions of the player. R. Hagtvedt et al. [14] provided new insights by proposing a cooperative game to reduce ambulance diversion, which is a bit more complicated. Hardy and Te [11] examined this idea in his work at DeKalb Medical Centre (DKMC) in Atlanta, GA. The two types of games can be transformed into each other under certain conditions, such as maximising the pursuit of own interests.

3 METHODS AND MODEL DEVELOPMENT

In this section, the model development and methods will be explained. Firstly, the GT-DES model will be discussed both conceptually and computationally. The GT-DES model contains two sub-models that are simulated separately 1. a queuing network model (QNM, principally modelled as discrete event simulation) to simulate patients’ arrival(including requesting and travelling process) and serving process. 2. a game-theoretical model to predict the defensive actions ("Accepting" or "Redirecting") of hospitals. Also, a situation where patients are uni-formly distributed in the city is configured for the artificial simulation. In addition, a sensitivity analysis by Sobol method is performed to understand the factor importance.

3.1 Conceptual model

This section reveals the conceptual process of the GT-DES model. There are three main roles in an ambulance-dispatching system:

1. The patient is the main consumer to emergency medical service (EMS) and needed to be served in a hospital. During the simulation, We focus on a type of patients who are obligatory to be transferred to ED, meaning patients could not be cured by First Aid such as acute coronary syndrome (ACS) patients.

2. Emergency department (ED) or hospital (because ED is usually attached to a hospital) is a medical facility to serve incoming patients. In this thesis, ED and hospital are categorized as the same type of medical facility and will be used as the same agent. Conceptually, ED is an accurate term focusing on serving the patients in emergence. However, walk-in patients to hospitals and other types or surgery-wanted patients may also be considered in the model. In the proposed GT-DES model, two main interests for ED or hospital are assumed 1. ensuring the quality of patient life. 2. considering serving more patients by ED itself. Thus, ED or hospital has the incentive to choose if accept or redirect incoming patients.

3. EMS: emergency medical service is a medical centre where it responses to the incoming emergency call and arranges ambulance squad to help and fetch patients. EMS is also

FIGURE3.1: A flowchart explicitly explains the workflow of the ambulance-dispatching system (ADS)

from the perspectives of stakeholders, including patients, emergency medical service (EMS) and ED (or hospital). The whole flowchart will be modelled by queuing-network model analytically and stochastically except the marked comment box will be modelled by the game-theoretical model (GTM)

to predict the strategies of EDs or hospitals.

responsible for sending the patient’s request to a nearby medical facility such as ED, asking for acceptance if a patient needs to be transferred for further aids. Besides, EMS usually collect medical data for the support of policymaker. For instance, our data is provided by Almazov National Medical Research Centre with the help of EMS in Saint Petersburg, Russia.

The flowchart shown in Figure.3.1 conceptually describes the process of the ambulance-dispatching system from different perspectives. The general process simulated is the following:

once patients are fetched by the ambulance, the nearest ED (target ED) is chosen as a top prior-ity. The patient will be sent to target ED if the target ED accepts ("Accepting" strategy). How-ever, suppose the target ED’s strategy is redirecting, and the current number of patients has surpassed the predefined capacity (the sum of the number of the servers and queuing room). In that case, the ambulance then brings them to the next nearest ED. We repeat this process by asking over all the EDs. If all EDs are busy and reject the patient’s requests, then the patient will be sent to the best ED with the minimum time spent on travelling and predictive serving. It shall be addressed that in an ambulance-dispatching system, the definition of "server" is seen as any medical resource in realistic which is most crucial to patient healing, could be a medical facility, professional doctors, nurses and beds.

3.2 One-dimensional analytical queuing-network model (QNM)

An analytical model is a deterministic model employing mathematical solution without randomness. Three reasons for conducting the one-dimensional analytical model are:

1. It provides us with the opportunity to quickly study the behaviours pattern of the system 2. We can verify ideas before running the stochastic model which has more randomness. 3. The analytical model has a lower cost for computational power, we can manipulate

pa-rameters freely.

This section is originally derived from my another paper [12].

Figure.3.2 shows the idea of 1-D queuing-network model . In plain words, we firstly as-sign k = 2 hospitals (denoted as H1 and H2) to each side of the one-dimensional map (Note, we use notation "hospital" instead of "ED" in the computational model). Each hospital has a strategy of “Accepting” (abbr. A, accepting all the patients’ requests) or “Redirecting” (abbr. R, redirecting/diverting patients to opposite hospital if the current number of patients in hospital exceeds the limited capacity including serving facility and waiting room or queuing buffer).

The process of 1-D model is the following: firstly, the patient is initialised uniformly be-tween H1 and H2 as pi. Next, if the hospital accepts the patient, the patient will be delivered to a hospital by the principle of proximity. Otherwise, the patient will be redirected to another nearest hospital. Whenever patients arrived at any hospital, they will be served immediately if the hospital has free servers (the term "server" is derived from queuing theory [4]). Otherwise, the patient will be queuing in a waiting room. The serving process follows the principles of "First In First Out" (FIFO) and "First Come First Served".

FIGURE 3.2: The figure shows the ambulance-dispatching process on a one-dimensional map. Hj

denotes the hospital with unique id j for j = {1, 2}, pi denotes the patient with unique id i, Tcis the

distance between H1 and H2in time. Each Hj has the number of Cj server (in realistic, the server is

a surgery facility or professional medical resource) with serving rate µi patients per hour per server.

The queuing buffer is the waiting room in realistic where patients queue here. A H1taking "Accepting

(A in short)" strategy has the infinite capacity of queuing buffer. H2 taking "Redirecting (R in short)"

strategy will redirect the incoming patient to opponent H1 when no vacancy in either its queuing

buffer or server.

The computational structure of the one-dimensional model is illustrated in Figure.3.3. The patient flow in hospitals is modelled by the multiple-servers queuing models M/M/c [4], where the first two M ’s denote Markov memory-less process, Poisson Process. As shown in Figure.3.2, cj is the number of parallel-working servers in a hospital Hj. If a hospital is tak-ing the "accepttak-ing" strategy, together the incomtak-ing patients and hospitals are modelled by the M/M/c/∞ model with infinite capacity. On the contrary, the hospital taking the "redirect-ing" strategy is modelled by the M/M/c/N model with a limited system capacity Nj. where Ni = Ci+ Wi, wi is noted as queuing buffer or waiting room in hospital i, ci is the capacity of servers.

λ is defined as the patients’ request rate also the mean of modelled arrival process as an exponential process with inter-arrival time by the equation:

f (x; 1 λ) = 1 λexp(− x λ) (3.1)

FIGURE3.3: The computational structure of a 1-D queuing-network model (QNM)

Also, M is defined as global serving rate, whilst µ is serving rate of one server with µ = 1

Tserving. Thus, we have M = cµ where 1/µ is the mean for the probability density function of Poisson distribution for serving process:

f (k; 1 µ) =

e−µ1

µk_k! (3.2)

where 1

µ = Tservingis the average serving time and also the mean of Poisson process. There-fore, the total time of patient picontains transportation time, queuing time and serving time as the following:

T_totali = T_travellingi + T_queuingi + T_servingi (3.3) Suppose each hospital can select one strategy aj with j being hospital id. Two hospitals in 1D map have {a1, a2} as strategy combination where a1 ∈ A, R and a2 ∈ {A, R} (meaning H1 is taking a1 strategy, same a2 to H2). And aj is either accepting strategy or redirecting strategy, the total norm for 1d model are AR/RR/RA/AA situation. Also, we define the load parameter of single server is a = λ/µ and load parameter with multiple servers ρ = λ/cµ.

• In the case of AA (H1and H2have Accepting strategy):

The request rate in Hj is λj = λ₂, where j ∈ {1, 2} in 1d model.

• In the case of RA (H1 has redirecting strategy and H2has accepting strategy):

probability of no patients in the system [4]: P0 = [1 + c X n=1 an/n! + ac/c! N X n=c+1 ρn− c]−1

Then we expand it to the probability of N (the capacity of system ) patients in the system, which is equivalent to the probability of redirecting patients prej in the system:

PN = ρnP0 Prej = Pmax =

aN c!cN −cP0 Next, in RA case , the effective λe1 in H1 is [21]:

λe1 = λ1(1 − Prej(λ1)) The effective λe2in H2 is:

λe2= λ2+ λ1Prej(λ1)

• AR case has same result to the case of RA with swapped hospital number i. • In RR case (H1and H2both have redirecting strategy):

An additional event may occur which is:

Both hospitals reject the request because of overcrowded. Then the nearest hospital to the patient is forced to accept the patient’s request without passing admission control. Consequently, the request rate will be updated:

λe = λ∗ = 3 X j=1 λi,j λi,1 = λi(1 − Prej(λi)) = λi− λR1i λi,2 = λR1i−1(1 − prej(λi + λR11−i)) = λR1i−1− λR2i

λi,3 = 0.5(λR2i + λ R2 i−1)

Because of rerouting, travelling times for different strategy varies: T_transpAA = T_transpRA = 0.25tc

T_transpAR = 0.25 + 0.75prej(λR) 1 + Prej(λR)

T_transpRR = 0.25λi,1+ 0.75λi,2+ 0.5λi,3 λ∗_i tc

where Tcis defined as the distance between two hospitals measured by time.

Next, we could deduce some important metrics of the system from a long-term perspec-tive.

Metric 1. The average number of patients in queue and average time of each patient spent in queue [4]: LQ = P0acρ c!(1 − ρ)2₎[1 − ρ N −c_{− (N − c)ρ}N −c_{(1 − ρ)]} WQ = LQ/λe

Metric 2. The average number of patients being serving is: Ls = c X n=1 Pnn + N X n=c+1 Pnc

Metric 3. Global time, a indicator measuring the system performance. Global time is obtained by: Tglobal= T1 totalλ1+ Ttotal2 λ2 T1 total+ T 2 total

3.3 Two-dimensional stochastic GT-DES model

A two-dimensional (2D) GT-DES model is an extended one-dimensional model with the stochastic process on the city scale. Meanwhile, multiple hospitals are invoked k = 3 as an example.

The two-dimensional stochastic model integrates a queuing-network and game-theoretical model (GTM). QNM contains the patient’s arrival process and serving process. QNM is simu-lated to obtain the payoff of each hospital. And the payoffs will be reconstructed as inputs to the game-theoretical model(GTM) to search for the pure Nash Equilibrium as a prediction for hospitals actions.

We start from introducing the arrival, serving and game process.

Arrival process

This section details the conceptual part of patient’s Markovian arrival process.The arrival process contains contains two parts mostly: requesting and travelling. The arrival process

starts from the patient being fetched at a random location, where the interval time between the patient’s fetching jobs follows an exponential distribution. The travelling time to target hospital is estimated by Euclidean distance over average vamb. As a result, we obtain the environment time that patient arrives at the target hospital and start serving process (will be explained in section 3.3) . Arrival process contains two parts of sub mathematical models:

1. Requesting process, to generate patients’ requests with attached properties: location and time being fetched by ambulance squad.

2. Responding process, to respond by accepting or rejecting/redirecting to an incoming re-quest

3. Travelling process, to estimate patients’ travelling time to target hospital.

Requesting process is seen as a non-homogeneous Poisson process. We generate a sequence of interval times Tintervalbetween patient-fetch jobs. The Tintervalfrom exponential distribution by

f (Tinterval; 1 λ) = 1 λ exp(− Tinterval λ ) (3.4)

with pre-defined λ as request rate (representing the number of requests sent per hour). Then ti+1 = ti + Tinterval where ti indicates the environment time of ith patient-fetch event (where i is again the unique id of patient). In this paper, T (upper case) is notated as a period of time while t (lower case) is notated as an instant of time (enviroment time).

Responding process depicts the reply from the target hospital when the hospital receives a patient request from EMS. The concept has been explained in section 3.2. In short, given pre-defined strategy combination for the system, each hospital has a strategy of “Accepting” (abbr. A, accepting all the patients’ requests) or “Redirecting” (abbr. R, redirecting patients to oppo-site hospital if current number of patients in hospital exceeds the limited capacity including serving facility and waiting room or queuing buffer). If a patient was rejected, another request to the nearest available hospital is sent. And the nearest hospital will be forced to accept if the request was redirected by all available hospitals.

Travelling process aims to estimate the travelling time for each patient. Geographically, we assume that the locations of patients being fetched have a uniform distribution for each coordinates px, py in a cartesian coordinate system as talked in section.3.5. Either px or py has probability density distribution of

f (x) = ( ₁

b−a, a ≤ x ≤ b 0, otherwise

where a, b are the lower and higher boundaries respectively.

For each patient pi, the travelling time Ttravellingi is estimated by: T_travellingi = d(Pp, Ph)

vamb

(3.5) Where vamb is the predefined average speed of the ambulance squad, P denotes a 2D position vector. And d(Pp, Ph)is Euclidean distance between the fetched patient pi and target hospital Hias d(Pp= {pix, p i y}, Ph= {hjx, h j y}) = q (pi x− h j x)2 + (piy− h j y)2 (3.6) where pi

x, piy are the x, y coordinate of patient pi, and hjx, hjy are the x, y coordinate of target hospital hj.

The arrival process has a direct impact on the system effect. This can be verified by sensi-tivity analysis at section 3.6,

Serving process

Serving process is a process starting from the moment the patient arrives at the target hospital until discharged. The term “server” in this paper represents a pivotal medical resource in a hospital which limits the maximum number of patients being served at the same time. Such as professional doctor, medical facility. The general serving process can be described as the following: after the patient is brought to the target hospital by ambulance squad, a vacancy check to the server will be be arranged to see if one of the servers is idle. If not, the patient will be assigned to the waiting room (aka the queuing buffer in 1D model) temporally until one of servers being released.

Serving process is modelled as a Markov’s Poisson process with parallel-working servers. The serving process can be seen as a multi-classes queuing system with probabilistic routing. Given c as the number of parallel-working servers in a hospital, we have total serving rate M = Pn

j=1cjµj where µ is the average serving rate for each server (meaning the average number of patients being served per hour per server), and average serving time is _µ1. The serving time’s probability density distribution is the Poisson distribution (same to the serving process in 1D model as described in section 3.2):

f (k; 1 µ) =

e−µ1

µk_k! (3.7)

The game process uses a game-theoretical approach to predict the defensive actions of hospitals at a specific time by searching for the pure Nash Equilibrium in a payoff matrix. The game process is independent from the arrival and serving process. This section focuses on constructing a game-theoretical model (GTM) to predict the defensive actions given λ as request rate M as a system serving rate.

To determine the definition of payoff for hospitals, consultation to experts from the medi-cal centre is considered. We introduce two significant factors impacting hospitals decision:

1. Health outcome of patients in the hospital, such as mortality rate.

2. The number of patients can be served in the hospital because of resource utilisation and social responsibility.

Below is the summarised words explaining the main interests for hospitals by doctors from Saint Petersburg: when a hospital receives a request from EMS, the hospital will accept patients as much as it can if only the resource utilisation and social responsibility are considered. Sec-ondly, the health outcome of the patient in the hospital is vital. Given both considerations, the hospital decides to accept or reject patients. For instance, accepting more patients may lead to a financial outcome and better resource utilisation, however, if patients have waited a long time, the health loss may happen. Therefore, if a hospital can redirect the patient to another nearby hospital containing a free resource, it gives a better chance for patients to be cured. However, it may result in an extra loss of incoming patients for the hospital.

Given the above considerations, a payoff metric is used as "score". The score payoff of hospital j is defined as

Scorej =

nj_served

T_totalj (3.8)

where nj_served is the number of patients being served in hospital j within the simulation time. And the average total time spent for each patient in hospital j is computed by

Ttotal= Ttrans+ Tserving+ Tqueuing (3.9) Formally, given [λ, M ], the queuing network model would produce the payoff matrix ∈ R2×2×2_{. For a non-cooperative game, N = {1, 2, 3} is a finite set of three hospital agents.} And we define A = A1 × A2 × A3 of action profiles a = (a1, a2, a3)where aj ∈ Aj = {A, R} with A as accepting, and R as rejecting/redirecting action where Aj being the set of actions available to agent j. And u = (u1, u2, u3) is a profile of utility function[24]. Thus, hospital

j receives the payoff Lj_a = uj(a) and a combination payoff for three hospitals is defined as L = (L_{1_a}, L_{2_a}, L_{3_a})_{. Eventually, a table of payoff matrix ∈ R}2×2×2_{can be obtained in Tab.3.4}

FIGURE3.4: A payoff matrix ∈ R2×2×2for non-cooperative game. N = {1, 2, 3} is a finite set of three hospital agents. And we define A = A1× A2× A3of action profiles a = (a1, a2, a3)where aj ∈ Aj =

{A, R} with A as accepting, and R as rejecting/redirecting action where Aj being the set of actions

available to agent j. And u = (u1, u2, u3)is a profile of utility function[24]. Thus, hospital j receives

the payoff Lj_a= uj(a)and a payoff vector for three hospitals is defined as L = (L1_a, L2_a, L3_a). Subsequently, we search pure Nash equilibrium (weak) from the payoff matrix in Table.3.4 to predict the potential system strategies for each hospital.

Nash Equilibrium is defined as an equilibrium no player has anything to gain by changing only their own strategy/action.[40]." Formally speaking, a action profile a = (a1, a2, a3)where aj ∈ {A, R} is a pure Nash equilibrium, if aj is a best response to a−j for every agent j ∈ N . Given notation (aj0, a−j) is short of (a1, ...aj0, ...an) where n = 3, then a∗j is the best response if uj(a∗j, a−j) ≥ uj(aj0, a−j) for all aj0 ∈ Aj And when the inequality above holds strictly (with > instead of ≥) for all players and all feasible alternative strategies, then the equilibrium is classified as a strict Nash equilibrium. If instead, for some player, there is exact equality between and some other strategy in the strategy set, then the equilibrium is classified as a weak Nash equilibrium."[40]).

3.4 Implementation details

This subsection reveals the IDE of building GT-DES model:

• Python 3.7 is used to build a "M/M/C/[N, ∞]" queuing-network model with Simpy li-brary [27].

• Game-theoretical model is manually constructed by the author.

• ArcGIS [33] is used to set a map base, the background of the map is derived from google map application.

• A library SALib [15] is used for sensitivity analysis.

3.5 Scenario design

This section depicts a specific scenario for GT-DES model’s simulation. We firstly set the basic environment of the model. Secondly, we invoked two types of agents: patient and hos-pital. Each of them is associated with self properties and event-triggered operations [45]. The idea of designing the computational model is based on Object-modelling technique (OMT) [37]. The biggest strength of using OMT is the computational model can be easily developed by object-oriented programming (OOP) such as Python.

Environment design

Space where agents live in this model, is called the environment. Figure.3.5 shows the spa-tial environment with a simulated 2-dimensional area having 3 hospitals evenly distributed in the cluster area. Each hospital has the same Euler distance to other hospitals. Shaded sectors are the "service coverage" zones. Patients from each zone are normally delivered to the near-est hospital if the hospital accepts the requnear-est. Each hospital has its service coverage: patients inside this area are delivered to this hospital firstly, in other words, the patient closing to the hospital gets priority. Patients’ locations are generated randomly in the entire two-dimensional map. The same to the one-dimensional scenario, each hospital can choose between two strate-gies: 1) to accept patients without restriction or 2) to redirect patients.

TABLE3.1: Table of General Notations in Numerical Model

i _{, The unique id(index) of a patient agent p}i

pi , A patient agent p with unique id i. pi ∈ p where p ∈ {p1, p2, ...pn}

j _{, The unique id(index) of a hospital agent h}j

hj , A hospital agent h with unique id j. hj ∈ h where h ∈ {h1, h2, h3}

Ti

event , The time slot of an event for agent i

Tenv , The time slot for the whole simulation (aka. simula-tion time), unit in hour

ti

event , The instant of time of an event triggered for agent i tenv , The instant of environment time

Environmental notations used are demonstrated in Tab.3.1.

FIGURE 3.5: Simulated two-dimensional area with three hospitals and correspond-ing capacity of servers.Shaded sectors are the "service coverage" zones. Parameters

used are Radius = 40km, origin = (40, 40), and locations of hospitals are Ph =

The patient is a core role throughout the model. The general term "patient" could be seen as a class with attributes as shown in Tab3.2. Each instance of a patient with a unique ID is named as a process in SimPy (A python library to model discrete-event simulation as explained in section 3.4). Therefore, as the inner clock env.time starts counting, we first construct a generator process to create patient process whenever "request event" is triggered. Next, each patient instance (process) starts moving immediately.

TABLE3.2: Table of Attribute Notations of Agent: Patient

pi , A patient agent p with unique id i. pi ∈ p where p ∈ {p1, p2, ...pn}

Ti

travelling , The duration of time spent in travelling from the place patient was fetched to target hospital hj where j ∈ {1, 2, 3}

T_queuei _{, The duration of time spent by a patient i in queuing at} waiting room of hospital

Ti

serving , The duration of time spent by a patient i from being served till discharged

pi.status , The statues of patient going to target hospital, pi.status ∈ {Direct, Redirect}

pi.targetHos , The current target hospital of patient pi

Agent: Hospital

Hospitals are initialised at the beginning of simulation with associated properties shown in Tab.3.3

TABLE3.3: Table of Attribute Notations of Agent: Hospital

hj , A hospital agent h with unique id j. hi ∈ h where h ∈ {h1, h2, h3}

hj.strategy , The strategy hospital j is taking, in {Accept, Redirect} hj.res , A sub property of hospital indicating resource where

patient can request.

hj.count , The number of users for resource(server) in hospital hj

hj.capacity , The number of maximum users in resource in hospital hj, equal to the number of servers.

hj.queue , The number of patients queuing for resource(server). hj.maxQueue , The number of maximum length of queue in hospital

hj. If hj.strategy == Accept, hj.maxQueue = ∞, else hj.maxQueueis predefined.

Algorithm: Searching for Nash Equilibrium Result: Search weak Nash Equilibrium

begin for a1 in {a, a−1} do for a2in {a, a−1} do for a3in {a, a−1} do if ua1,a2,a3 1 ≥ u a−1₁ ,a2,a3 1 and u a1,a2,a3 2 ≥ u a1,a−12 ,a3 2 and u a1,a2,a3 3 ≥ u a1,a2,a−13 2 then a1, a2, a3is weak Nash Equilibrium

end end end end

Algorithm 1:Search 3-d weak Nash Equilibrium

This part formally explain the computational process to obtain the Nash Equilibrium for GT-DES model. The general idea is: with running r realisations of GT-DES models by Monte Carlo simulations, we obtain a list of Nash Equilibrium, where the most frequent strategy combination is selected as Nash Equilibrium in given scenario.

Formally speaking, given generated fixed patients lists pi = [p1, p2, . . . , pfi) where each pi associated with random request time and serving time under the restrain that the average request and serving rate is λ, M . The length of piis cut off by the end of simulation time. Next, the GT-DES model is considered as a function fi(λ, M, a, pi = [p1, p2, . . . , pfi)which returns an instance of payoff L = (L1_a, L2_a, L3_a)described in section3.3, where a = [AAA, .., RRR] and a ∈ {AAA, .., RRR}. So that the payoff matrix could be obtained by

fi(λ, M, a, pi) =          fi(λ, M, a = [AAA], pi = [p1, p2, . . . , pfi) fi(λ, M, a = [AAR], pi = [p1, p2, . . . , pfi) .. . fi(λ, M, a = [RRA], pi = [p1, p2, . . . , pfi) fi(λ, M, a = [RRR], pi = [p1, p2, . . . , pfi)          (3.10)

Thus, with r random realisations of simulation, N (f (λ, M, a, p])) returns a list of Nash Equilibrium (length = r where N (f1(λ, M, a, p1)) returns a instance of Nash Equilibrium by Alg.1. So that we have:

N (f (λ, M, a, p])) =               N (f1(λ, M, a, p1)) N (f2(λ, M, a, p2)) .. . N (fi(λ, M, a, pi)) .. . N (fr−1(λ, M, a, pr−1)) N (fr(λ, M, a, pr))               (3.11)

Eventually we obtain the Nash Equilibrium anewhere ane = argmaxP (ane|N (f (λ, M, a, p]))) where P denotes the probability density function.

3.6 Sensitivity analysis by the Sobol method

Sensitivity analysis (SA) is a method conducted to quantitatively performances how much the effect of the output comes from each input to model[38]. Three reasons we apply SA to GT-DES model are:

1. To increase the understanding between the inputs and output of a model 2. Find potentially useless inputs.

3. Robustness check and validation for model

Sobol indices (or Sobol method) is utilised to SA for GT-DES model [39]. Sobol is an variance-based sensitivity analysis technique with following advantages [38]:

1. Sobol indices is a global method, meaning either independent interacted influence of inputs can be performed.

2. Passing Sobol method to Non-linear process in simulation is not a problem

To formally perform Sobol method, we want to obtain the first and total order indices. With i being the index of one of the inputs, Y being the target outcome of the model, the first-order indices (S1) is : Si = Di var(Y ) (3.12) where Di(Y ) = V ar[E(Y |Xi)] (3.13) Interaction indices is:

sij = Dij

TABLE3.4: SA indices of λ w.r.t sample size

Distinct sample size 25 ₂7 ₂9 ₂11 ₂13 ₂14

SA total indices of λ 0.25 0.27 0.28 0.28 0.28 0.28

95% confidence error ±0.19 ±0.09 ±0.04 ±0.02 ±0.01 ±0.01 where

Dij(Y ) = V ar[E(Y |Xi)] − Di(Y ) − Dj(Y ) (3.15) Therefor, total indices (ST) is:

ST i = Si+ X i<j Sij + X j6=i,k6=i,j<k Sijk+ ... (3.16)

FIGURE 3.6: Total-order sensitivity indies of λ with respect to the growing size of distinct samples N. It shows it shows It shows with the size of distinct samples increase by logarithmic, the 95% confidence can reach ±0.1 at N = 214 (2 significant digits saved). Analysed by Sobol analysis with

1000-times resembles.

To numerically perform the Sobol indices, we firstly use Sobol shuffle (also called as Sobol sequence) to sample the N (d + 2) input samples via Monte Carol approach; where the d is the number of parameters (d = 9 in the proposed model), and N is the predefined size of the distinct sample. Sobol shuffle provides low-discrepancy sequences randomly, and it is an

efficient sampling method. As N grows, the computational cost increase linearly by O(n). To determine the size of the distinct sample, N , as shown in Figure.3.6, with the size of distinct samples increase by logarithmic, the 95% confidence error can reach ±0.1 at N = 214_{as shown} in tab.3.4 with two significant digits saved. Thus, the Sobol indices with N = 214 _{will be} performed.

Next, we performed the Sobol method by setting output as "Score" and the "Score of the hospital 2" as shown in the Figure.3.7. The "Score" of hospital j is defined as

Scorej =

nj_served

T_totalj (3.17)

Where njserved is the number of patients being served in hospital j within the simulation time, it shows λ, M contributes most of the sensitivity to either system or one of hospital’s score. The velocity of ambulance contributes significantly less than other factors. This idea is used in the later case study where travelling time is not as important as door-to-balloon time (the sum of queuing and serving time). Also, for a specific hospital’s score, we found its capacity and strategy is twice important to its score than them in other hospitals.

FIGURE3.7: First and total order sensitivity analysis by setting output as "system score" (top figure)

and the "score of hospital 2" (bottom figure).It shows λ, M contributes most of the sensitivity to either system or one of hospital’s score. The velocity of ambulance contributes significantly less than other factors. This idea is used in the later case study where travelling time is not as important as door-to-balloon time (the sum of queuing and serving time). Also, for a specific hospital’s score, we found its capacity and strategy is twice important to its score than them in other hospitals. The distinct sample

4 EXPERIMENTAL STUDY AND RESULTS

This section presents the results of a one-dimensional mathematical model, two-dimensional stochastic GT-DES model. Secondly, an application of GT-DES model for simulating the ambu-lance dispatching of ACS patients in Saint Petersburg is discussed, with real-world data being used.

4.1 Simulating with artificial settings 4.1.1 Warm-up simulation

FIGURE 4.1: Global time spent at two hospitals of a one-dimensional queuing-network model. "A" in the legend stands for accepting strategy, "R" denotes redirecting strategy. The less global time, the better to the system. It shows AA strategy is the most beneficial strategy (with dominantly less global time) in most cases, except the circled interaction in the left figure; where AA switches to RA and RR, and it starts to reduce the global time when patients flow grows to a certain level, given the server

N = N1, N2= [2, 1]is not equal. Parameter used are Tc=6, patient flow is defined as_µ1+µλ ₂ The warm-up simulation contains a simple simulation of 1d and 2d queuing-network model.

Firstly, we run a batch of computationally-cheap simulations to check the truth of the 1D model quickly. Some insightful results have caught our eyes: Figure.4.1 presents the global time spent by all patients for two hospitals( the less global time, the better to the system). It shows AA strategy is the most beneficial strategy (with dominantly less global time) in most

FIGURE 4.2: A warm-up simulation showing the number of patients served by 3 hospitals using different strategy combinations. It shows that when the system is in the middle-level crowdedness, the "Redirecting" (R) strategy starts being beneficial by serving more patients. The numbers of servers in hospitals are N = [2, 3, 4]; so does the queuing lengths; with approximately 400 incoming patients. For each gradually added requests, ten simulations were run with random initialisation of the patient locations and serving times. The figures are presented by the means and error bars using one standard

deviation.

cases, except the circled interaction in the left figure; where AA switches to RA and RR, and it starts to reduce the global time when patients flow grows to a certain level, given the server N = N1, N2 = [2, 1]is not equal. It turns out given higher patient-flow, the RR/RA/AR strat-egy may be more beneficial than the AA stratstrat-egy. This result partly inspired us the potential incentive of switching strategy by agents. Furthermore, it gives us intuition to focus more on server inequality and the irregular patient-flow in simulation.

Secondly, we run the queuing-network simulation for three hospitals with unequal servers in a two-dimensional map. As it is shown in Fig4.2, when the patient flow is small (e.g. only 100 patients appeared within the simulation time), the number of patients served is approxi-mately the same for all strategies. Because only a few patients are redirected to other hospitals and the system is free (it means there is barely queues in the hospitals). It also explains that the

FIGURE 4.3: Contribution of each hospital to the number of total patients served at different levels of crowdedness. At the point when the number of patients appeared is 230. The figure shows that

hospital with four servers contributed most of the patients served.

serving ratio (the ratio between the number of patients being served and appeared in the sys-tem) reaches to 93%. The remaining 7% of patients are still being served or under routing due to the termination of simulation time. With the number of patients’ requests growing to around 150, the RRA and RRR strategies result in more patients being severed. When there is a heavy patient flow (e.g. 300 patient requests within the simulation time), the system is saturated, be-cause all the hospitals are overcrowded. Therefore, when the system is busy, "Redirecting" R strategy is not necessary since the requests will be rejected, as shown in the unchanged area in the plots demonstrated in Figure.4.3.

TABLE4.1: Definition and values of input variables in the computational GT-DES model

Input Variable Definition LB UB

server combination N = [n1, n2, n3]

The number of parallel-working servers in each hospital, meaning hj.capacity = nj, j ∈ {1, 2, 3} strategy combination

a = [a1, a2, a3]

The strategy taking by each hospital, where

hj.strategy = aj, aj ∈ {A, R},j ∈ {1, 2, 3} [A, A, A] [R, R, R] request rate λ the average number of requests per hour 3 18 serving rate M the average number of patients served per hour 3 18

4.1.2 Result of GT-DES model with multiple hospitals

In this subsection, the simulation results of the stochastic GT-DES model will be demon-strated.

With the common parameters specified in Tab.4.1. We illustrated the transient "working" Nash Equilibrium (NE) at the scenario where N as being N 0 ∈ {[2, 2, 2], [1, 2, 3], [2, 3, 2], [2, 1, 2]}. The term "working" here indicates the NE is the actual actions of hospitals instead of strategies used (e.g. it may happen that due to that the system is free (higher serving rate associated with much less request rate), there is barely actual "Redirecting" happens. In this case, the actual action is "Accepting" all patients rather than used "Redirecting" strategy. All Nash Equilibrium shown in the latter result is the "working" NE)

Figure.4.4 shows a 2-D working Nash Equilibrium (NE) matrix with outcome argument "Score" as payoff (the Nash Equilibrium is the predictive strategy combinations for hospitals with respect to the request and serving rate). Then, we define input variables λ = [λ1, ...λn]as n = 20 M = [M1, ...Mn]as n = 20, total 400 (n × n) combinations of [λ, M ]. Then, with each of [λi, Mi], N 0as inputs, the GT-DES model will give us an Nash Equilibrium (NE). And we run it multiple times to obtain the NE with the most frequent occurrence. Eventually, we obtain a transition map of NE covering all possible scenario with given input variables [N 0, λ, M ].

An overview of Figure.4.4 illustrates that the Nash Equilibrium (NE) transits from dom-inated AAA to RRR and then Inconsistent system from left-top blue area to right-bot gray area. Respectively, the blue area has AAA being the most-likely NE, because the servers are relatively free where redirecting patient won’t happen mostly (λ  M ). Next, the clay-bank area has RRR being the most-likely NE because the servers reached maximum load or slightly overload (λ ≈ M ), redirecting help balance the resources. In the grey area, where the system is saturated and paralysed any strategy is like a lead balloon (λ  M ). The limitation is that the NE exists when all hospitals take the same strategy (AAA or RRR) due to every hospital owns the same amount of resources.

FIGURE 4.4: A 2-D working Nash Equilibrium (NE) matrix with outcome argument "Score" as the payoff. given λ = [3, 18], M = [3, 18] in the scenario of N = [2, 2, 2] (left) and N = [2, 1, 2] (right). It shows with growing request rate (λ) and declining serving rate (M ), the pure Nash Equilibrium transits from AAA to RRR until the system is inconsistent. The term "inconsistent" (abbr. "Incon Sys", shown as grey square in the plot) states there is no NE due to the system being overcrowded (meaning queuing time goes to infinite in EACH hospital for incoming patients). Each block shows the most frequent Nash Equilibrium among all 106 realisations The area of a coloured inner square divided by 1 is the probability of occurrences (i.e. the probability of AAA strategy being NE at the left-top corner is 100%). And each NE is searched from payoff matrix 3.4 by algorithm 1, Simulation parameters used are: 106 simulations for each block (20 × 20 = 400 rectangles in total), simulation time ranges from 30 to 135hours, the appeared number of patients ranges from 400 to 2500, vamb =

FIGURE4.5: An arbitrary example of the dispersive distribution for the occurrence of Nash Equilib-rium in a simulation. In this case, the RRR strategy is selected as Nash EquilibEquilib-rium due to RRR

appeared 17.92% in 106 simulations.

To elaborate the story at strategy transition, a zoomed-in distribution at red-circled rectan-gle in Figure.4.4 is demonstrated in Figure.4.5. Stochastically, the NE distribution is dispersive where RRR is slightly more chance of being NE (17.92%) than AAA (16.04%)

Next, we plotted the time consumption of each patient in each hospital, shown in Fig-ure.4.6. At peak-time period, there is overcrowding in the system which contributes higher queuing time as shown in the top plot having rectangles area.

The bottom figure in Figure.4.4 reveals the NE transition in the scenario of N = [2, 1, 2]. It shows the equilibrium transits from AAA ⇒ ARA ⇒ RRR ⇒ RAR ⇒ Inconsistent. It turns out, when one hospital having less server while others have more same number of servers, the hospital with less server starts switching R strategy with growing request rate (Dark green area). However, when system is slightly overload, it firstly switched to A strategy then others. Figure.4.7 reveals the NE transition in the scenario of N = [2, 3, 2]. Likewise, hospital with fewer server are more vulnerable to environment and strategy switched firstly from Accepting to Redirecting via AAA ⇒ RAR ⇒ RRR ⇒ ARR & RRA ⇒ inconsistent. As well as the bottom figure in Figure.4.7.

To summarise, we see that 1. The system strategy transits from more Accepting to more Redirecting as the system gets busy. 2. The hospital with fewer servers tends to switch strategy

earlier than the one with more servers. These two findings can be verified in the later case study of ACS patients in Saint Petersburg.

FIGURE4.6: An example showing the time consumption when λ = 7.73, µ = 13.26. It shows the time consumption for each patient given AAA (top figure) and RRR (bot figure) strategy. It shows RRR has averagely less queuing time due to having less peak-time effect (peak-time timeslots are marked

FIGURE4.7: A 2-D working Nash Equilibrium (NE) matrix with outcome argument "Score" as payoff. given λ = [3, 18], M = [3, 18] in the scenario of N = [2, 3, 2] and N = [1, 2, 3] . It shows with growing request rate λ and declining serving rate M , the pure Nash Equilibrium transits from AAA ⇒ RAR ⇒

4.2 Simulating ambulance dispatching of ACS patients in Saint Petersburg

This subsection demonstrates the case study of GT-DES model on ACS patients in Saint Petersburg with real-world data set involved. We firstly simulate the isolated area (with only 0_hos and 1_hos). Secondly, we simulate every available pair of hospitals and predict their strategies by finding the Nash Equilibrium. The hospital pairs are selected by considering the proportion of shared patients by each of hospital in the pair. Thirdly, we calculate the weighted strategy for each hospital and predict the global strategy (the strategy combination for all hospitals in the city).

In order to understand the treatment of Acute coronary syndrome (ACS) patients by the health-care system in Saint Petersburg, a simulation supported with real-world datasets will be introduced. Firstly, the background and the datasets will be discussed. Secondly, methods of simulating request process, travel process, the serving process of ACS patients with involving the real-world dataset will be explained. Thirdly, we applied the GT-DES model to simulate the ambulance dispatching for an isolated area with two hospitals as a start point. Then we extend the simulation for every interacting hospital’s pair in the city. An interacting hospital pair such as ihos and jhos means there exist an area where patients are going to either ihos and jhos. (usually, ihosand jhosare neighbouring). As a result, the average door-to-balloon time in each hospital is measured. And the average mortality in a year is estimated by the door-to-balloon time. Subsequently, we compared the simulated mortality with observed mortality from data as validation. In future, a deep understanding of the ACS case can result in further research in policy-making for the lower mortality of ACS patients in Saint Petersburg.

4.2.1 Background

Acute coronary syndrome (ACS) is a syndrome where the patient’s heart may not be func-tioning properly due to decreased blood flow in the coronary arteries [2]. We choose ACS patients for simulation because they have two main properties: professional aids in hospital is needed, ACS occurs in an unexpected situation. Also, the angiography is set up as a server in the model because 1. an angiography is a must-have surgery resource as an imaging technique for ACS patient two due to the high price, most hospitals do not own a sufficient amount of angiography and supporting resource. Meanwhile, other types of urgent patients may also oc-cupy the server due to angiography is a universal framework for heart diseases, such as Acute myocardial infarction (AMI), stent and stroking. Thus, sometimes ACS patients have to be waiting during rush hour, extra time spent can lead to higher mortality in the hospital which is not expected in the health-care system.

4.2.2 Data description

Almazov National Medical Research Centre provides the data in Saint Petersburg. We are authorised to use them for internal research only.

Datasets contain :

1. 5124 records of ACS patients went to Thirteen hospitals by ambulance from 1th Jan to 15th Nov in 2015 in Saint Petersburg. This dataset can be used to build the request process of ACS patients for simulation.

2. Thirteen hospitals linked to the records of patients and the number of angiographies each hospital has. This dataset is used to set up the simulation environment to support pa-tients’ travel process and serving process.

3. 1,310,263 travelling time records of vehicles spread in Saint Petersburg from 143 locations to 18 other locations (including 13 hospitals) at each hour. This dataset is used to simulate the travel process.

4. Serving time for 1866 records of stent patients.This dataset is used to simulate the serving process of ACS patients.

5. Annual averaged mortality in 2015 of 10 hospitals in Saint Petersburg. This dataset is used for model validation (as an comparison to the simulated mortality)