Data Exchange and Representation in Normal Mode

where T is the bit time and _bit T is negligible on a small scale (i.e. when the network distance is _P

≤ 100m) because the typical transmission speed in a communication medium is about 2 10 m s 8 . In effect, it has been shown that at the worst, the propagation delay from one end to the other of the network cable is T = 25.6 µs for Ethernet with a length of 2500 m (Lian, 2001). _P

 The post-processing time at the receiving phase is the total time necessary for reading exchanged data from the local network and then decoding them to the native format for use by Matlab/Simulink or ESP-r. The time elapsed during this phase depends on the speed of the machine(s) used and the procedure implemented for reading data from the network and decoding them to the native format. As is pre-processing time, the post-processing time may be constant and negligible.

6.4. Translating Requirements Specification to Implementation

This section presents the different methods that have been developed and implemented to translate the functional requirements of run-time coupling between ESP-r and Matlab/Simulink discussed in Sections 2.9 and 4.2 into functions and operations in a distributed simulation mechanism:

6.4.1. Data Exchange and Representation in Normal Mode

In run-time coupling, ESP-r and Matlab/Simulink should form an integrated environment allowing cooperation between building models and their control systems distributed in a same simulation.

Therefore, both ESP-r and Matlab/Simulink should participate in the same distributed simulation using a common way of representing and interrelating data exchanged through a network. To fulfil such a requirement, two normal modes of data exchange and representation were implemented within run-time coupling between ESP-r and Matlab/Simulink: a text mode and a binary format.

Data Exchange in Text Mode

Exchanging data between ESP-r and Matlab/Simulink in a text mode, especially ASCII is well known as a relatively simple yet slow way of exchanging data between programs running on the same machine or on different machines connected by a network. It is slow because all data to be exchanges are encoded into strings of printable decimal digits (so-called ASCII strings), and then decoded into data exchanged after they have been received. Using C/C++, various ways of implementing this mode are possible, such as using the functions sscanf() and sprintf(). Whereas the sprintf() function is used to convert data to be exchanged into strings, the sscanf() function is used to retrieve that strings back to data exchanged. However, the stringstream() class was used in this work because it provides a means of manipulating the data to be exchanged and the data that have been exchanged as strings as if they were input/output streams, respectively.

The primary reason for using ASCII code (or strings) in implementing this mode is that it is the default form of character encoding used on practically all machines and by all computer languages. However, data encryption algorithms, such as the advanced encryption standard (AES), can also be used in the implementation of this mode of data exchange.

Data Exchange in Binary Format

Exchanging data between ESP-r and Matlab/Simulink in a binary format is considered the fastest way of exchanging data between programs running on the same machine or on different machines connected by a network because it exchanges data in the form of binary codes. Implementing this mode of data exchange requires using data structures (i.e. structs) or classes to package all data to exchange into one object at sending and unpackaging that object into data exchanged at receiving.

When using classes or data structures, the manner in which data are packaged varies depending on the compiler and the architecture of the machine being used (i.e. on whether big-endian or little-endian architecture is used ⁷¹), both of which affect important factors such as padding, byte ordering and

71 Examples of big-endian architectures are microprocessors such as Motorola and Sparc and examples of little-endian architectures are the Intel x86 and Dec Alpha architectures. The major difference between them is that the most significant byte comes first in big-endian byte-ordering schemes but last in little-endian byte-ordering schemes.

alignment, and endianness when exchanging data structures between programs running in heterogeneous environments. To address this issue, a method for dynamically marshalling and unmarshalling data structures over a TCP/IP network was implemented within run-time coupling between ESP-r and Matlab/Simulink that is effective regardless of their operational and network locations. Figure 6.9 illustrates how this method was implemented to exchange data in the form of a binary format across heterogeneous environments.

Figure 6.9. A simplified example of exchanging data structures between Matlab/Simulink and ESP-r running on different machines using a binary format

In this method, marshalling (or serializing) is used when sending data to flatten data structures into one-dimensional arrays before converting these arrays into a form of external network representation, and unmarshalling (or deserializing) is used when receiving data to de-convert data from a form of external network representation into a form of internal representation (i.e. one-dimensional arrays) before rebuilding the data structures. In addition to these operations, a number of primitive functions for byte swapping are also performed within this method to convert network byte order to host byte order, regardless of whether the machines being used are similar and/or run on the same OS.

While implementing this method, this work discovered that additional steps may be required, primarily due to the use of new compilers.⁷² For example, when exchanging data structures containing items (or variables) of a simple or a double-precision floating-point type between ESP-r and Matlab/Simulink running in heterogeneous environments, the received data are identical to the transmitted data when 32-bit compilers are used but not precisely identical when 64-bit compilers are used, requiring the performance of additional steps. For such reasons, this work re-implemented this mode of data exchange within run-time coupling using another method (or format) with an IDL, such as protocol buffer⁷³ libraries, to allow Matlab/Simulink and ESP-r to exchange data in a form of binary format, regardless of their operational and network locations and the compilers are used.

Of the formats with an IDL that can be used to exchange data in a binary format between Matlab/

Simulink and ESP-r, such as XDR and boost serialization, Google’s protocol buffers were chosen because they are the most rapid, simplest, and lightest. Moreover, their use has been described as a language-neutral, platform-neutral, and extensible way of serializing data structures for use in communications protocols, data storage, and other functions (Protobuf, 2008). Although obviously conceived for encoding structured data in an efficient extensible format, Google’s protocol buffers also

72 Most compilers in use today work with 64 bits, whereas they worked with only 32 bits several years ago.

73 Protocol buffers are serialization formats with an IDL developed by Google for encoding data structures in an efficient yet extensible format. For more details, see http://code.google.com/apis/protocolbuffers.

serve a purpose similar to that of XML, with the advantage of having a much denser and binary representation often wrapped by an additional compression layer for even greater density. Among the protocol buffers described with an IDL that define the content of the messages to exchange, a protobuf compiler takes this IDL (i.e. .proto file) and generates executable code to manipulate the protocol buffers, while a flag to the protobuf compiler specifies the output language, such as C++, Python, Java, and so on. For each data type (or item) declared in the IDL file, the output of the protobuf compiler includes the native class (or data structure) definition and the codes for accessing the fields, marshalling and unmarshalling the data for translation between the native representation and the external binary format, and debugging. The generated codes were compiled and linked with the socket APIs of the C/C++ client code, which were in turn compiled and linked with ESP-r and with socket APIs of the C/C++ server code, which were in turn compiled and linked with the matespexge toolbox.

In Google’s protocol buffers, the IDL file forms a clear, compact, and extensive notation describing the layout of data items and the naming of the fields in a manner roughly analogous to that of simple classes or C struct() functions. This IDL also includes two additional properties for each field – (1) a distinguishing integral tag used to identify the field in binary representation and (2) an indication of whether the field is required, optional, or repeated – that allow for the backward extension of protocol buffers by marking fields as optional and assigning them an unused tag and/or by specifying fields as repeated to dynamically size or resize the arrays. Because they offer such advantages, Google’s protocol buffers were chosen to enable ESP-r and Matlab/Simulink to exchange data in a binary format by run-time coupling.

Experimental Studies on the Effects of Communication Time Delays on the Performance and Stability of Building Control Applications

A series of experimental tests were conducted to determine the performance and stability of control systems when run-time coupling was subjected to communication- (or network-) induced time delays, with the delays being measured as the total elapsed time to transfer data from ESP-r to Matlab/

Simulink and transfer it back to ESP-r in terms of transmission load and periodicity (or simulation period). Each experiment was performed in four parts to evaluate performance using both modes of data exchange (ASCII and binary), and arranged in ten figures in accordance with the number of run-time coupled ESP-r(s) to Matlab/Simulink by the network, which varied from 1 to 10. The first two parts were conducted when Matlab/Simulink and one or more ESP-r(s) were running in the same environment (i.e. all on Unix) and the second two parts when ESP-r and Matlab/Simulink were running in distributed heterogeneous environments (i.e. one or more ESP-r(s) on Unix and Matlab/Simulink on Windows).

The total elapsed time (T_elapsed) was computed by subtracting the time-stamp of when the simulation was terminated (_Tstop) from the time-stamp of when the simulation was commenced (T_start) without counting the processing time that elapsed while ESP-r and Matlab/Simulink computations were exchanging data with each other over a network. This total elapsed time can be explicitly expressed by the following equation:

elapsed stop start

T T T (6.5)

The experiments were constructed with the maximum number of simulation time-steps allowed by ESP-r, which it is currently limited to 60 steps per hour for building zones and 100 steps per hour for building plant components (see e.g. ESRU manual, 2002). The simulation period was set for one day for 11 hours (07:00 to 18:00). By calculating the periodicity for each simulation, the results were recorded for 660 instances of data exchange⁷⁴ for building zones and 1100 instances of data exchange for building plant components. In addition to the periodicity, the transmission load (or capacity) was evaluated by varying the size of the data (or the message) from 100 B to 1000 B, then to 1500 B, 2000 B, and then to 3000 B, assuming that the 2000 B and 3000 B sizes are sufficiently large to transfer all data required for any distributed simulation between one or more ESP-r(s) and Matlab/Simulink.

74 As this exchange is bidirectional between ESP-r and Matlab/Simulink, it makes in reality twice those numbers.

Experiments in a Homogeneous Environment

The experiments in a homogenous environment were performed on a Sun Blade 1000 Workstation with 2 CPU modules of 900 MHz Ultra SPARC III and 4 GB of RAM. ESP-r and Matlab were installed over the Unix 10 (or Solaris 10) OS on the Sun Blade 1000 Workstation, and the machine was connected to the network by an Ethernet LAN cable at a speed of 10 MBps. The compilers used for Fortran and C/C++ programs were of the GNU Public License. All ESP-r(s) were run-time coupled with Matlab/Simulink by the TCP protocol and executed in increasing order of their ESP-r sub-thread numbers such that when Matlab/Simulink was run-time coupled with two or more ESP-r(s), Matlab/Simulink exchanged data with one ESP-r only until the simulation using it had terminated, then began exchanging data with the next ESP-r, and continued doing so until no run-time coupled ESP-r(s) remained. During data exchange, the time-stamp (or time-step) of the first data exchange was sent with the simulation data to be subtracted from the time-stamp of the last data exchange in order to calculate the time elapsed for distributed simulations between ESP-r and Matlab/Simulink.

Figure 6.10. Experimental results regarding elapsed times of 660 instances of data exchange in distributed simulations between one or more ESP-r(s) and Matlab/Simulink in the same environment

Figure 6.11. Experimental results regarding elapsed times of 1100 instances of data exchange in distributed simulations between one or more ESP-r(s) and Matlab/Simulink in the same environment It became clear that exchanging data by run-time coupling between one or more ESP-r(s) and Matlab/

Simulink running in the same environment using a binary format is more effective and rapid than using a text mode. Figures 6.10 and 6.11 show the elapsed times for distributed simulations of a varied number of run-time coupled ESP-r(s) with Matlab/Simulink running in a homogeneous environment.

As the curves in these figures indicate greater fluctuation when exchanging data in a text mode compared to a binary format, exchanging data in a binary format by run-time coupling between one or more ESP-r(s) and Matlab/Simulink appears to have fewer negative effects on the performance and stability of communication-based control systems (explicitly NCSs). In addition, it can be observed that when Matlab/Simulink and ESP-r exchanged 1 KB of data by run-time coupling, about 120 ms was required to perform distributed simulations of 660 exchanges, and about 200 ms for distributed simulations of 1100 exchanges. Therefore, it is important to note that run-time coupling between ESP-r and Matlab/Simulink requires only about 0.18 ms to send and receive 1 KB of data in a binary format.

Experiments in Heterogeneous Environments

The experiments in heterogeneous environments were performed on a distributed network consisting of a Sun Blade 1000 Workstation with 2 CPU modules of 900 MHz Ultra SPARC III and 4 GB of RAM and a PC with an Intel CPU of 2 GHz Pentium IV and 1 GB of RAM. ESP-r was installed over the Unix 10 (or Solaris 10) OS on the Sun Blade 1000 Workstation and Matlab was over the Windows XP OS was installed on the PC, and both machines were connected to the network by an Ethernet LAN cable at a speed of 10 MBps, and at distance of about 7 meters from each other. The compilers used for Fortran and C/C++ programs were of the GNU Public License on Unix and of Visual Studio 2008 on Windows. All ESP-r(s) were run-time coupled with Matlab/Simulink by the TCP protocol and executed in an increasing order following their ESP-r sub-thread numbers in a manner similar to that used for the experiments in a homogenous environment. During the data exchange, the time-stamp of the first data exchange was sent with the simulation data to be subtracted from the time-stamp of the last data exchange in order to calculate the time elapsed for distributed simulations. Although synchronizing the clocks of these machines was difficult – or potentially impossible due to several incompatibilities – the results of the elapsed times were obtained with a margin of at least 100 ms.

Figure 6.12. Experimental results regarding elapsed times of 660 instances of data exchange in distributed simulations between one or more ESP-r(s) and Matlab/Simulink in different environments

Figure 6.13 Experimental results regarding elapsed times of 1100 instances of data exchange in distributed simulations between one or more ESP-r(s) and Matlab/Simulink in different environments The experimental results indicated that in most cases, exchanging data in a binary format by run-time coupling between one or more ESP-r(s) and Matlab/Simulink running on separate machines connected by a network is more effective than doing so in ASCII mode. Figures 6.12 and 6.13 show that the elapsed times for distributed simulations of a varied number of run-time coupled ESP-r(s) with Matlab/

Simulink running on distributed heterogeneous environments do not vary greatly between the ASCII and binary modes for communication loads smaller than 1 KB. However, this lack of difference can likely be attributed to inaccurate synchronization between the clocks of the machines used and the time required to perform the serialization and deserialization operations that protocol buffers utilize to flatten and rebuild, respectively, data structures, depending on the machine involved and its OS. In addition, as a network speed of around 10 MBps was used, any utilization of the network on either machine could have influenced the elapsed time of data exchange in distributed simulations. Therefore, the utilization of machines with better performance characteristics can significantly reduce the elapsed times of distributed simulations running in heterogeneous environments.

Regardless of the issues mentioned above, it can be observed that when Matlab/Simulink and ESP-r running in heterogeneous environments exchange 1 KB of data by run-time coupling, completing a simulation requires about 1500 ms for distributed simulations of 660 exchanges and about 2600 ms for distributed simulations of 1100 exchanges. Therefore, when run-time coupling between ESP-r and Matlab/Simulink is performed on separate machines connected by a network, around 2.3 ms is required to send and receive 1 KB of data when using either mode of data exchange (ASCII or binary). In addition, the use of machines with better performance characteristics and a higher-speed network could significantly reduce the elapsed times of such distributed simulations between ESP-r and Matlab/

Simulink. Effectively, reducing the elapsed times of these distributed simulations would certainly decrease the impact of network time delays on the performance of building performance applications.

The objective of conducting experimental studies on distributed simulations running in a homogeneous environment and in heterogeneous environments was not only to clarify the importance of using run-time coupling between Matlab/Simulink and one or multiple ESP-r(s) but also compare the run-time required by different modes of data exchange (ASCII and binary) to send and receive data through TCP protocol using network sockets by varying the communication loads. Because using TCP protocol is safer than using UDP protocol but requires more time, UDP protocol is often used in NCS studies (see e.g. Choy et al., 2003). In addition, with the use of UDP, the response times of run-time coupling between Matlab/Simulink and one or more ESP-r(s) running in both a homogeneous environment and in heterogeneous environments can also be reduced significantly.

These experimental studies were conducted by setting the simulation periodicity to 60 and to 100 time-steps per hour. The obtained results (see Figures 6.10, 6.11, 6.12, and 6.13) show that although increasing the simulation periodicity does not necessarily increase the elapsed times of data exchange in distributed simulations, the elapsed times always increase in tandem with increases in the transmission load. In most cases, it appeared from that exchanging data in binary format results in better consistency than doing so in ASCII format. Therefore, it is clear that exchanging data in binary mode is generally more efficient in terms of response times and transmission loads when performing run-time coupling between Matlab/Simulink and one or more ESP-r(s).

Although one or more ESP-r(s) were executed simultaneously during the experiments, when two or more ESP-r(s) were run-time coupled with Matlab/Simulink, the run-time coupling was configured in such a way that Matlab/Simulink could exchange data with only one ESP-r until the simulation with it

Although one or more ESP-r(s) were executed simultaneously during the experiments, when two or more ESP-r(s) were run-time coupled with Matlab/Simulink, the run-time coupling was configured in such a way that Matlab/Simulink could exchange data with only one ESP-r until the simulation with it