Multi-touch visualization of Multi- core simulations

(1)

Bachelor Informatica

touch visualization of

Multi-core simulations

Mike Trieu

August 10, 2015

Supervisor(s): Raphael ’kena’ Poss (UvA)

Inf

orma

tica

—

Universiteit

v

an

Ams

terd

am

(2)

(3)

Abstract

Multi-core simulation is the tool that computer architects and operating system designers use to design and evaluate future multi-core processor designs. They need to compare multiple configurations in order to find the best design and to find faults in the design. Trace files produced by the simulation are often used to compare the configurations of processor designs, but it can take a lot of time. Worse configurations can be detected earlier in the simulation. That is why a visualized inspecting tool is proposed in order to make research in multi-cores easier and faster. The tool architecture always shows the newest simulation data. The tool architecture minimizes the impact the simulation speed. The tool architecture is separated in a front-end and back-end. The two parts communicate over TCP. The visualization could then run on another machine than where the simulation runs. The architecture of the tool is portable in theory, because of the generic implementation. Usability tests pointed out that the developed tool still needs many improvements. Especially in the visualization of the simulation data itself. By design, and as confirmed with users, the proposed visualization is relatively low-level.

(4)

Introduction

To challenge the increasing power consumption and heat production of processors, multi-core processor architectures have been adopted. Multi-core processors provide more performance for less power. To ensure optimal performance of multi-core processors, computer architects and operating system designers face several challenges. Challenges that becomes more complex with each technology advancement [11].

One advancement is that processors have more and more cores and accelerators. Computer architects and operating system designers need to work together to face the challenges and decide how to design and program multi-cores with accelerators best. Simulation is the tool they use to prepare future designs and predict how they will perform, before the design goes into production. The goal of simulation is to compare multiple configurations of a processor design and to detect errors in the design in order to find the best configuration. Producing every configuration would take more time and money than changing the configuration in a simulation.

Most multi-core simulators store simulation output data in a trace file, which is analyzed after the simulation is finished. Analyzing each trace file to search for the best configuration and detect errors in the design takes a lot of time. That is why analyzing simulation data during the simulation is also used, which is called online analyzis. This method saves time when hundreds of different configurations are simulated, since configurations that are no good can be detected earlier in the simulation.

Another aspect of online analyzing simulation data is handling errors in the processor configu-ration simulations can run for hours and errors could be detected halfway through the simulation. One would want to analyze and change configuration without resetting the simulation to save time. That is why a online inspecting tool for multi-core simulators is needed.

Most multi-core simulators provide two methods for online analyzing simulation data: syn-chronous and asynsyn-chronous. The synsyn-chronous method samples simulation data in between simu-lation cycles. It provides consistent simusimu-lation data where one can tie direct resimu-lations to simula-tion cycles, but it slows down the simulasimula-tion. The asynchronous method samples simulasimula-tion data periodically. In this method the simulator does not have to wait for the sampling to finish, so it is not guaranteed that the data is sampled over one simulation cycle. This causes the simulation data to be less consistent. However, the overhead to the simulator is lower.

Inspecting the simulator using only the previously mentioned methods may not be enough. Processor architectures are complex. Reporting simulation data in a linear and not very human-friendly way makes analyzing the data difficult. Most multi-core simulators provide tools to visualize the trace files, but visualizing online output is rare. To make understanding the online simulation data, a visualization needs to be added to the online inspecting tool.

(7)

Why visualize simulation data? Visualization has the benefit to interpret large amounts of data, when presented well. More information is acquired through vision than through all of our other senses combined [13]. Depending on the type of visualization, simulator users can interpret data without digging through a really long output file [9]. Users can compare data next to each other or they can observe simulation behaviour at a certain simulation point. Unanticipated observations could also become apparent using visualizations [13].

1.1 Motivation

In this project, a visualized inspecting tool for multi-core simulators is developed. We will focus on using the online asynchronous method to sample simulation data, since processing simulation data to a visualization takes time. We want to find a way to also overcome the main obstacle of asynchronous monitoring, while the overhead to the simulator stays the same.

1.2 Approach

We will be searching for answers to the following research questions in order to develop the visualized inspecting tool:

• How to design a visualization architecture where the visualization does not impact the simulation speed?

• How to design a visualization architecture where the visualization does not lag behind the simulator?

The main challenge in designing the visualized inspecting tool for multi-core simulators is that the visualization architecture needs to be able to handle interaction between the visualization and the simulator, while the visualization displays the most recent simulator data.

We also have additional goals besides the research questions and the main challenge. We will focus on designing a visualization architecture for one multi-core simulator. However, we want the architecture to be self-contained and portable. The implementation should be generic in order to achieve this. This should be possible, since most multi-core simulators use objects to represent components of processors and are connected in a hierarchical structure. The component structure is elaborated in chapter 2.1.3.

Another additional goal is that we want to support multiple users in order to make cooperation easier. A multi-touch screen is useful for supporting multiple users. That is why the tool will be designed for a multi-touch screen.

1.3 Thesis Outline

In chapter 2 we will analyze the software used in the project and elaborate on the software choices in the context of the approach. In chapter 3 we will discuss the design choices and the implementation. Chapter 4 will evaluate the implementation. Chapter 5 will contain the conclusions and future work.

(8)

CHAPTER 2

Background & Related Work

We want to visualize simulation data from a multi-core simulator. However, creating a multi-core simulator takes too much time. That is why we decided to develop the visualization architecture for an existing multi-core simulator.

There are two multi-core simulators that fit our project approach: MGSim [5] and gem5 [8]. Those two multi-core simulators are similar in purpose and software architecture [10]. We want a hierarchical object structure for the components in order to visualize the simulator data. MGSim provides an easier naming structure and access to the components than gem5, which is why we chose MGSim over gem5.

The visualization tool is also not built from scratch for the same time constraints reason. We wanted an existing visualization tool that visualizes processor components. The tool also need to be able to have touchscreen interaction with the processor. One of the tools that satisfies these requirements is the visualization and task management tool named ManyMan [12]. The ManyMan tool visualizes a less detailed chip than MGSim, so a new visualization for the components need to be designed. However, the less detailed visualization is able to serve as a template. The task management features that are present in ManyMan are not present in MGSim, so that feature will be completely omitted.

MGSim and ManyMan are further elaborated in the upcoming sections.

2.1 MGSim

MGSim is a discrete event simulator for on-chip hardware components, developed at the Univer-sity of Amsterdam since 2007. A discrete event simulator assumes nothing happens in-between events. MGSim was developed to support scientific research in design of many-core general-purpose processor architectures. It was also developed as a studying vehicle targeting students in the computer architecture field [10]. MGSim is free software under the GNU General Public License.

MGSim can function as a simulator framework or as a full-system emulator. The intentions of both functions are different. Simulators exist to debug and optimize hardware designs before the production, while emulators exist to debug and optimize software in a controlled environment before shipping them [10].

2.1.1 Simulator Architecture

Most of MGSim is written in C++. MGSim uses object classes to represent chip components. For example: processors (cores), caches, memory networks, etc. A component will define one or more processes and optionally some internal state for its processes [10]. Most components in MGSim are configurable, as is the system topology.

MGSim consist of four more main parts, other than the component part (See Figure 2.1). When MGSim starts, the the component models are instantiated by the system configuration constructor, which also connects them together to form a full architecture model.

(9)

Figure 2.1: Entity-relationship diagram of an MGSim simulator from [10].

The components themselves are driven by the simulation kernel. The execution driver of the kernel schedules ready component processes at each simulation cycle.

The simulation front-end provides a user interface with MGSim. It consist of a configuration file loader, command-line parsing, asynchronous monitoring, etc. Optionally ”interactive mode” can be enabled, which activates an interactive command parser [10].

The asynchronous monitoring is a optional part of MGSim, that periodically samples data from selected components and writes it to a trace file or FIFO for analysis or visualization by external tools [10]. This part of MGSim is one of the ways to track the simulation process, which will be elaborated in the next section.

2.1.2 Monitoring Facilities

MGSim provides three main facilities to track the simulation process: • Synchronous event traces

• Asynchronous monitoring • Performance counters

Synchronous event traces reports all events of a specific category. This facility is intended for troubleshooting or observing detailed cycle-to-cycle behaviour of components. The event traces can be printed on the terminal, written to a trace file or piped to external programs. This facility will create some overhead by formatting the events [10].

Asynchronous monitoring samples simulation data from selected components in a configurable interval, which can be written to a trace file or piped to external programs. Asynchronous monitoring runs concurrently with the simulation thread in a separate thread of execution. This facility runs faster than synchronous event traces, but is also less accurate. Monitoring is intended for capturing the evolution over time of semi-continuous variables in the simulation model [10].

Performance counters enables software running on a simulated system to read the simulation state. This is intended to capture time and resource usage of individual software components in larger benchmark programs [10].

(10)

2.1.3 Component structure

MGSim can have hundreds or thousands of components. Parsing all the components and their variables to the visualization will take too much time, which could lead to slow down in the visualization. MGSim constructs a long name based on the position of each component in the model tree to show which variable is part of which component. It separates the components using dots and colons, where the part before the dot is a parent component of everything after the dot. The part after the colon is a variable of the component before the colon. Each component can contain one or more variables as seen in the following examples:

• cpu0.perfcounters:nOtherSampleOps • cpu0.pipeline.execute:flop

• cpu0.pipeline.execute:op • cpu0.pipeline:nStagesRun

2.2 ManyMan

ManyMan is a tool (See Figure 2.2) that functions as both an interactive visualization and a dynamic task management for many-core chips, optimized for multi-touch screens [12]. It was originally created for the 48-core Intel Single-chip Cloud Computer (SCC) chip at the University of Amsterdam. The tool monitors the performance of the chip and the tasks running on the chips. Users can interactively create tasks and put tasks on specific cores. Users can also migrate tasks between cores [12]. ManyMan is available as free software under the GNU General Public License.

2.2.1 Kivy

ManyMan is developed using the Kivy framework. Kivy is a open source Python library for rapid development of applications that make use of innovative user interfaces, such as multi-touch apps [4]. Kivy was developed with multi-touch interaction in mind [3]. It provides native support for multiple input devices and a growing library of multi-touch aware widgets [2].

Kivy is designed to let people focus on building custom and interactive applications as quickly and easy as possible. It is written in the programming language Python and thus can take

(11)

advantage of the many existing libraries for Python. Most of the performance-critical parts are written on the C level language [2]. It also uses intelligent algorithms to minimize costly operations, thus increasing the performance. Kivy also supports hardware accelerated OpenGL drawing for its graphics engine, which is built over OpenGL 2.0 [4]. Kivy is optimized to use the GPU to increase the display performance [3].

Kivy applications can run on multiple platforms, while using the same source code. It sup-ports Windows, Linux, OS X, Android and iOS [4]. This makes Kivy flexible in use, but Kivy is also flexible in adapting new technologies quickly. It added new external devices and new software protocols multiple times in the past [3].

Furthermore, Kivy is 100% free to use under a MIT licence for versions starting from 1.7.2 and under LGPL3 for the versions under it. The library is a community developed project, led by professional software developers [4].

ManyMan was original developed for Kivy version 1.2.0 and higher. For the project Kivy version 1.7.2 is used. This is also the version the visualization tool is tested on.

2.2.2 Application Structure

ManyMan consists of two parts. One part is a front-end which is the visualization application itself and the other part is a back-end that collects information of the hardware it runs on. The model used to connect the two parts is a client-server model. In between them, a communication layer exists. This communication layer sends data between the front-end and back-end over TCP. The ManyMan architecture enables the front-end to be able to connect to a different many-core chip. It also enables another front-end to connect to the ManyMan back-end. Another reason for this architecture was that it was not easy to place a monitor where the Intel SCC is located. Now the monitor can be placed anywhere, because it can connect to the Intel SCC over TCP [12].

2.3 Related Work

One tool similar in function as the tool of developed in this project is a GUI-support library for an object-oriented discrete-event simulation language, named Pearl [9]. Simulations in Pearl, have objects that represents components in a computer architecture model. The GUI-support library can visualize data while the simulation is running. Even step by step analyses using the visualization is possible. Simulation statistics are displayed in various types of visualizations. However, the support library is written in Pearl for Pearl. We aim for a more generic visualization tool.

(12)

CHAPTER 3

Software Architecture Design &

Implementation

In this project, we will connect the ManyMan front-end part to MGSim. We will add a new object to MGSim in order to connect the two (See Figure 3.1). MGSim will act as the back-end for the ManyMan front-end. The new object is named “communicator”. The communicator will sample simulation data asynchronously. The ManyMan front-end will visualize the sampled data.

The monitoring object of MGSim is still retained as an option. This enables MGSim to run both at the same time. A trace file can then be written for later observation, while users can use the visualization for online observation.

3.1 Expanding MGSim

MGSim is mostly written in C++. Adding a communicator to MGSim in C++ is the most obvious choice. The communicator is instantiated in the MGSim front-end before the simulation starts. We want to keep functions of the the visualization architecture in the communicator as much as possible, because we said that the implementation should be generic in order for it to be

Figure 3.1: Entity-relationship diagram of an MGSim simulator with a communicator added (in red). Source image modified with permission from [10].

(13)

easily portable. The option to enable the communicator is added as a mode called “visualization mode”.

3.1.1 Communicator Threads

We want the communicator to not impact the simulator speed, so it uses separate threads of execution. The communicator spawns two threads of execution when visualization mode is enabled.

One thread handles the incoming ManyMan front-end and the messages it receives from it. This thread is called the “server thread”. We chose for a client-server model to connect the visualization tool and the multi-core simulator, which enables the visualization tool to run on a separate computer. The simulator can run on a fast computer, while the visualizations can run on a personal computer. The client-server model also adds the possibility to have multiple visualization clients.

The other thread continuously sends sampled simulation data and simulation status in a configurable interval. This thread is called the “sender thread”.

3.1.2 Interactive mode

MGSim has a command-line interface for direct interaction with the simulator, called “interactive mode”. This mode caused some problems for the visualization mode. The first problem is when the simulation ends, while it is not in interactive mode. MGSim will then terminate, which forces all the connected ManyMan front-ends to disconnect. For this reason the interactive mode is always enabled when the visualization mode is enabled.

The second problem is when someone interrupts the simulation on the command-line, for example is done by sending a signal like SIGINT. The interrupted simulation then falls back to the command-line interface. When this happens, the threads of the communicator have to continue to function. To do this, the signals has to be blocked. If the signals were not blocked, the visualization would stop receiving data. Any interaction from the visualization would also be stopped and new visualization clients would not be able to connect anymore. Figure 3.2 shows that the communicator is uninterrupted by SIGINT.

3.1.3 Component And Variable Selection

Sending all the simulation data would delay the visualization, because of the time it takes to send ten to a hundred megabytes of data. Processing the simulation data also takes time. That is why a selection of the components and their variables needs to be made. MGSim already provides functions to make selections of variables that users want to sample from, since component and variable selection are also necessary with the existing monitoring object.

(14)

The name of selected components and their variables are mapped to a data-structure that contains a pointer to the data produced by the simulator and meta-data about the data. The monitoring object uses pattern matching to select specific variables. The pattern will be used to make a list of matching variables and the mapped data-structure. The same method is used to select the variables for the communicator. The kernel.cycle variable is always selected. This is because it is important to know the kernel cycle of when the data is sampled to interpret the data.

3.1.4 Client Data Structure

The data structure stores information about the connected ManyMan visualizations in order to correctly handle communication. The data structure contains the following fields:

• socket: The file-descriptor of the connected ManyMan visualization. This is used to be able to handle multiple connection at the same time.

• initialized: A Boolean for a state of a client. If the client is not initialized yet, then data should not be send to the connected client.

• changed: A Boolean for another state of a client. This state is necessary for when the selection is changing. The communicator should stop sending data to the client, until selecting new variables is finished.

• buffer: The communication thread should collect incoming data from connected ManyMan visualizations and process them when a complete message is arrived. To do this every client needs to store their own buffer.

• default vars: A pattern for selecting variables when a ManyMan visualization connects. This enables the visualization side to at least build a visualization, instead of showing nothing on start up.

• selected vars: The list of selected variables. This includes a pointer to where the simulation data of the selected variables are stored and the necessary meta-info about the data to decode it.

3.1.5 Sending Simulation Data

The sender thread does not send data when the client has not been initialized. The visualization has to build the components first, before the visualization can parse the data. The change state of a client is in the same situation. If a client wants to change the component selection, the sender has to stop sending data to the client that requested the change. Sending simulation data is resumed, when the client finishes building the new visualization of selected components.

Sending simulation data in an interval does not mean that it ensures the sent data is processed before the next interval. The write buffer will clog up with simulation data if we keep sending before the visualization parsed the simulation data it received first. The visualization will then parse the next simulation data in the buffer. However, this message will be old data. We do not want the visualization to lag behind the simulator. We use a Linux-specific function to check if the buffer is empty and only write when the buffer is empty (See Figure 3.3).

When the visualization can parse the data, the thread keeps sending data even when noth-ing happens in the simulator. This is because when the user changes configurations usnoth-ing the

(15)

Figure 3.4: Command handling diagram.

command-line of MGSim, the visualization should reflect those changes. Other than the sampled data sent to the visualization, multiple status variables are sent to the visualization. These status variables let the visualization know the current configuration and state of the simulator, which are mentioned in section 3.1.4.

3.1.6 Interaction from Visualization

The communicator sends the state of the simulator to the connected ManyMan visualizations. However, the running state of MGSim was not accessible from the communicator. That is why a new variable and function is added to MGSim, where the state changes when the simulator runs and when it is stopped.

To actually pause the simulator while it is running, a function in the simulation kernel can be called. The function sets a variable to let the kernel know to stop running the simulator. Then the kernel thread returns to the command-line if interactive mode is activated. If not, the simulator stops.

Calling a function to resume the kernel from the communicator, will resume the simulator in the communication thread. That is not what we want, because then the communicator can not process new visualization clients or their commands anymore.

That is why the command-line interface is used to resume the simulator. As mentioned in section 3.1.2, the visualization mode forced interactive mode on MGSim. Instead of letting the command-line wait for user input, a timeout is added (See Figure 3.4). Then it can periodically check for a variable, which can be changed by the communicator. The command-line handler can resume the simulator, when that variable is set. The command-line interface will then process a hard-coded run command.

Using this method all the command-line commands of MGSim can be called from the Many-Man visualization. At the time of writing, run and step commands are hard-coded, since those two commands are implemented in the visualization as buttons (See section 3.3.4). The step command runs the simulator for a certain number of kernel cycles, then it pauses the simulator automatically.

3.2 Communication

The communicator added to MGSim communicates with the ManyMan visualizations through TCP connections. ManyMan visualization clients can connect to the communicator through a configurable port.

3.2.1 Message Format

Messages themselves are sent in JSON format. JSON is a data-interchange format that is easy for humans to read and write and easy for machines to parse en generate. It is also completely programming language independent [1]. Other than the previously mentioned features of JSON, ManyMan already uses JSON as a message format.

While JSON is supported in Python, which ManyMan uses, it is not in C++. To simplify the implementation an external library name RapidJSON is used. RapidJSON is a JSON parser and generator inspired by RapidXML [7]. RapidJSON being self-contained was the deciding factor

(16)

for choosing RapidJSON, since we want the the visualization architecture to be easily portable to other multi-core simulators. RapidJSON is free to use under the MIT Licence.

All JSON messages of the communicator and the ManyMan visualization follow the same base format used in ManyMan:

{ " t y p e " : < type > , " c o n t e n t " : { < content > } }

The type of the message simplify what the communicator or the ManyMan visualization has to do with the message, while the content contains useful data like the sampled simulation data. Tables B.1 and B.2 in Appendix B show all types of messages currently implemented.

3.2.2 Protocol

The communication between MGSim and ManyMan front-end follows a protocol. Before the communicator begins sending data to the visualization or process interaction messages from the visualization, initialization messages have to be sent. When the ManyMan visualization starts, a message with the client init type is sent to the communicator. Then the communicator sends a message with the type server init to the visualization. This message contains info about the simulator and also the default selected variables. The visualization needs a selection of variables to generate a component layout to show. Although the visualization can start without a variable selection, it looks nicer to show something at the start. After the initialization, simulation data and simulation status is sent in a configurable interval. The type of this message is sim data. The exchange of messages is shown in Figure 3.5.

The communication follows a protocol for making another selection of variables and for inter-action with the simulator, as shown in Figure 3.6. For selecting variables, the communicator of the visualization sends a message of the type selection new, which contains patterns used to select the variables. Then the communicator of MGSim sends selection set as a response, which con-tains the names of the selected data. MGSim stops sending sim data when selection new arrives (See Figure 3.7). The ManyMan front-end will then build the new visualization. Any sim data messages the ManyMan front-end receives while building the visualization are discarded. The communicator of ManyMan then sends selection send to MGSim. This causes MGSim to resume sending sim data.

For simulator interaction the ManyMan front-end sends messages to MGSim without expect-ing a response. This is because the status of the interaction is send periodically with the sim data message. The reason for this choice is that if for example the simulator pauses, all the other

(17)

Figure 3.6: New variable selection protocol (Situation 1).

Figure 3.7: New variable selection protocol (Situation 2).

connected ManyMan front-ends need to know the simulator is paused. The message types in this category are: pause sim, resume sim, set step and change delay. See tables B.1 and B.2 in Appendix B for detailed descriptions of all the messages implemented.

3.3 Front-end

The front-end visualization uses code from the ManyMan front-end as template, which uses the Kivy framework. The whole front-end is written in Python. Because Kivy was written in Python, the rest of ManyMan was written in Python [12].

3.3.1 Main Window

The visualization main window is shown in Figure 3.8. We mentioned in Chapter 2 that the original visualization in ManyMan is less detailed than in MGSim. ManyMan only visualized the processor component. We want to be able to visualize every component MGSim has, depending on the selection made by the user. We use the naming structure of the components to display the components in hierarchical order. The child components are placed inside the box of the parent components. The visualization can be cluttered with components when a large selection is made, unless a high resolution screen is used. Some components will be drawn really small or components will be drawn over each other. However, the amount components selected in practice is not that large. We left it to the user to make a modest selection.

The overlay that is used to visualize payloads of CPUs in the original ManyMan, is now used to visualize how many variables are changed in a component. The overlay changes in size, which depends on the amount of changed variables in relation to the other selected variables in the component. This condition also changes the color of the overlay. The colors ranges from green to red where green have the least change in variables and red the most.

3.3.2 Variable Selection

On the left side of the center window a button called New Selection opens a pop-up (See Figure 3.9). Here users can enter patterns to make a selection of variables to sample data from. Each new pattern has to be entered on a newline. Non-matching patterns (including typos) will not be

(18)

Figure 3.8: ManyMan front-end main window

processed for the visualization. Finally the Send Selection button in this pop-up is used to send it to MGSim. Entering patterns means that the user needs some knowledge of what variables are available to sample data from. The list of component and variable names can be retrieved using the command-line interface of MGSim.

3.3.3 Saving Selections

Entering patterns can take a lot of time, especially if the user want to change it often to observe different components. That is why a feature to save the selected variables is added. On the left side of the center window a list of saved selection is seen (See Figure 3.8). Tapping one of the saved selection automatically sends a previous selection to MGSim. In the current implementation users

(19)

can only recognize the selection by name. The selections can be further edited using an external standard text editor.

3.3.4 Interaction with the Simulator

Buttons to interact with the simulator are placed on the right side. The Resume button, runs the simulator when it is paused. The Pause button pauses the simulator when it is running. Nothing happens if the Resume button is pushed when the simulator is already running. The communicator of MGSim will ignore resume message. The pause message is also ignored when the simulation is already paused.

The Change Send Delay button opens a pop-up, where you can enter a decimal number to change the sampling interval of the communicator of MGSim. The Set Steps button opens a pop-up, where a natural number is entered to set the number of kernel cycles MGSim needs to run when the simulator is resumed.

3.3.5 General Simulator Information

General information about the simulator is shown on the bottom right side of the main window. The first one is the simulator status. This shows whether the simulator is currently running or paused. The second one is the current kernel cycle of the simulator. The third one is the current send delay. The last one is the current steps, which is zero if the simulator needs to run until the end of the loaded program.

3.3.6 Component View

Each component widget in the visualization is a button widget, that opens a pop-up (See Figure 3.10). The name of the component and its parent components are displayed in the title bar of the pop-up. On the left side all the variables of the component are displayed as button widgets. Pressing these buttons will change the right side of the pop-up. The right side will display information about the variable selected using the left buttons. The buttons will light up green if the variable sampled is different from the previous variable sampled. One does not have to tap every variable button to see which one is active. Figure 3.10 shows that the number of active variables in a component is reflected in the number of active buttons in the pop-up.

On the top right side of the pop up, the sampled data is displayed. It lists the data from oldest to newest. Only displaying raw data is not too helpful, because the data is not sampled on every kernel cycle. Therefore a graph is added to the bottom right side of the pop-up. This graph shows the change in the variable over time. For the graph the derivative of the sampled data is calculated using the following equation: f0(x) = f (xn)−f (xn−1)

tn−tn−1 . Where f is the sampled

data and t is the kernel cycle of when the data is sampled.

To observe multiple components at once, multiple pop-ups can be opened. This is one of the advantages over plain text. Users of the visualization can have multiple variables next to each other to observe at the same time.

(20)

(21)

CHAPTER 4

Evaluation

In this chapter, we present the evaluation results of the visualization tool introduced earlier in this thesis. We performed several experiments in order to confirm the performance of the tool. We tested if the tool is functional in section 4.1. The impact of the added communicator was evaluated in section 4.3. The modified ManyMan front-end was evaluated in section 4.3 and 4.4.

4.1 Functional Validation Test

We connected the computers over an Ethernet network. This way we can test if the remote visualization works. One computer is a laptop, which runs MGSim. The other computer is an Apple Mini, which runs the modified ManyMan front-end. The Apple Mini is connected to a multi-touch screen, which we used to test if the multi-screen capabilities of the modified ManyMan front-end work. See Appendix C for detailed hardware descriptions.

We started MGSim with the visualization mode enabled. We had to change address and port configurations of the modified ManyMan front-end in order to let it connect to MGSim. This configuration is stored in a external text file. The address should be the laptop address and the port number should be the default port number of the visualization mode of MGSim, which is 2300.

We can now test if the visualization is working after the modified ManyMan front-end and MGSim are connected. We tested the various interact-able widgets of the modified ManyMan front-end by pressing them on the touchscreen. The visualization is working as expected (See Figure 3.8).

4.2 Simulator Performance

We had additional requirements for the tool to a working tool. We wanted to see if the visual-ization architecture could not impact the simulator performance. We performed an experiment in order to see if it is really the case.

MGSim ran on the same laptop as the previous experiment. The Apple Mini is replaced by a personal computer running Windows 7, because the it was not available during this experiment. See Appendix C for detailed hardware descriptions.

We measured the overhead of the added communicator of MGSim, while the communicator samples and sends simulation data every second. We ran MGSim for 26 million simulation steps and measured the execution time of those steps. We calculated the relative increase in overhead. The reference execution time is measured by running the simulator without the visualization mode enabled. This measured reference time is an average of 58.7 (+/- 0.05) seconds. We performed the experiments for multiple variable selections to see if it added even more overhead. The results of the experiment (See Figure 4.1) show that the relative added overhead on average is around 2-4%, which is beyond expectations. Increasing the number of selected variables does not seem to increase the overhead more.

(22)

Figure 4.1: The added overhead of the communicator. Relative to the execution time of 26 million simulation steps without the visualization mode enabled. Multiple variable selections are measured. Simulation data is sampled once every second.

Figure 4.2: The added overhead of the communicator. Relative to the execution time of 26 million simulation steps without the visualization mode enabled. Multiple variable selections are measured. Simulation data is sampled ten times per second.

We also wanted to know if sampling speed affects the simulator, so we increased the sample and send rate to ten times per second. The results (See Figure 4.2) show a similar result as sampling and sending simulation data once every second, which means that it does not add more overhead by sampling and sending faster.

We also measured the simulation data message size per variable for multiple selections. We expected that the results are consistent. Figure 4.3 shows that the message size per variable is consistently 37-40 bytes. However, the variables in MGSim are 4-8 bytes. The significant increase in size is probably caused by the inclusion of the variable name and the associated component

(23)

Figure 4.3: Simulation data message size for multiple selections of variables. structure. The JSON formatting also adds to the size of the message.

4.3 Modified ManyMan Front-end Performance

We measured the average frames per second of the modified ManyMan front-end in order to estimate the performance. We selected a processor component and all their child components. We observed the frame-rate in relation to the number of open component view pop-ups. We used the same setup as the previous experiments. We measured the frame-rate in two situations. The first situation is when the visualization is updated once every second. The second situation is when the visualization is updated ten times per second.

The experiment setup is the same as in section .

The results of the first situation (See Figure 4.4) show that the frame-rate slowly decreases with more pop-ups open. The results of the second situation (See Figure 4.5) show that the frame-rate decreases really fast. However, the frame-rate stays around 10-13 frames per second after seven pop-ups. Pop-ups start to overlap after six pop-ups, which could explain this result. The decision to make the sampling and sending interval configurable seems to be good, since users can make interval longer if the visualization lags to much. We also did the experiment for a larger selection, but there were no significant difference.

4.4 Usability test

A usability test is performed to test the modified ManyMan front-end. We wanted to know what potential users think about the current user interface and visualization, so that improvements could be made in the future.

The users needed to satisfy the following requirement: they needed to have knowledge in computer architecture research. The people of the Computer Systems Architecture (CSA) re-search group at the University of Amsterdam satisfy this requirement. A total of four males participated in the test. The test setup is the same as in the functional test: A laptop running MGSim and a Apple Mini connected to a multi-touch screen.

The usability test consist of a demo. During the demo, several questions were asked. First, the main window of the visualization is shown. The participant is asked to explain what they think the visualization is visualizing and the functions of several elements on the main window. Then the participant is asked to interact with the visualization tool and are asked to explain

(24)

Figure 4.4: Frame-rate in relation to the number of open component view pop-ups. Visualization update rate in this situation is once per second.

Figure 4.5: Frame-rate in relation to the number of open component view pop-ups. Visualization update rate in this situation is ten times per second.

what he thinks of the visualization of data. Finally, the participant is asked to give a general opinion on the features and the usability. The participant is also asked what he thinks could be improved or added.

The opinion on the visualization of the components was good. The nested box structure of the components was clear for them. The colour block overlays that represent the change in the components was not clear to them. 75% thought it represents the load of the components. The confusion was caused by the change in colour, when more change happened. It was suggested to change the block overlay to something like separate blocks or circles in the component visu-alization to represent each variable in the component or add an explanation to the tool. 50% preferred that the change representation stayed coloured, when the simulator pauses. The same

(25)

applies to the graph in the detailed component view.

The opinion on the rest of the detailed component view was less positive. 75% failed to see the use of it. 25% implied that the display of raw data was too low level for a visualization and desired a higher level statistical visual representations. That brings us to another missed feature, namely more interactive multi-touch features. Features like zooming in on components to display detailed information or task management.

The patterns for selection components and variables was also a problem. 50% prefer to have a list to select from, as they are not that familiar with the patterns to select variables. One of them also would like to have explanations of the variables themselves. The ability to save a previous selection was perceived as a great addition.

(26)

CHAPTER 5

Conclusion

A visualized online inspecting tool is developed in order to make research in multi-core easier and faster. We developed the tool for a multi-core simulator named MGSim and used a existing visualization and management tool named ManyMan as template for the visualization. The tool architecture is developed with portability to other simulator in mind. We kept the use of libraries to a minimum and made the implementation as generic a possible. The tool was also developed with multiple users in mind.

We defined a communication protocol in order to make MGSim and the ManyMan front-end communicate. The protocol handles interaction between both, while simulation data is constantly sent over. The developed tool is working as seen in the functional validation test.

The simulator and the visualization tool do not have to run on the same machine, thanks to a client-server model. The impact of the modified ManyMan front-end is minimized when running it on another machine. However, the communicator added 2-4% relative overhead. A larger selection of simulator variables does not increase the overhead further and a shorter simulation data sampling and sending interval also does not further increase the overhead. We can not say that the visualization architecture does not impact the simulation speed, but the added overhead is small enough.

The modified ManyMan front-end only displays the newest simulation data. This was achieved by checking the buffer between the modified ManyMan front-end and the MGSim com-municator. The sampled simulation data is discarded when the previous data is not processed.

The modified ManyMan front-end itself features the ability to interact with the simulator. A list of selections can be saved for future use. Tapping the components will display a detailed component view pop-up. It also features some simple multi-touch features, provided by Kivy.

The test regarding to the usability of the visualization were not positive. The visualization of the components of the simulation was clear, but the visualization of the data themselves were not so clear for users. Although comparing raw data and the evolution over time next to each other is useful, additional information has to be displayed with it. Some GUI naming choices also needed to be clarified, so it needs a lot of improvement in intuitiveness. Also the visualization lack of real multi-touch features fitting the visualization. Kivy provides multi-touch features of widgets for the tool, but more interaction with the components themselves was desired.

5.1 Future work

The visualization of simulator data can use some improvements. The various types of simulation data are currently not distinguished. All the data is displayed in detail are currently raw data and the visualization of the change in data. Displaying the different types of data differently could make a higher level of visualization than the current visualization. A highly requested feature was to have more multi-touch interaction with the simulator data. For example to have the ability to zoom in on components.

(27)

also be useful. The usability test pointed out some confusion regarding the current naming conventions. A explanation in the visualization tool would also help, to understand the current visualization.

The original ManyMan tool had task management features. The current MGSim loads a program to run at the start of MGSim. If simulators are modified to provide task management features or if simulators already have them, then task management could be added again in the visualization of multi-core simulations. Task could be dragged to specific components.

In section 3.1.6, it was mentioned that all command-line commands could be accessed from the visualization tool. Two commands are currently hard-coded. However, a command-line interface could be implemented to show up optionally in the visualization. Implementing specific visualizations for certain commands is also possible. Output from the command-line does have to be limited or buffered, otherwise the communication buffer would clog.

The developed visualization tool architecture could be ported to other simulators in theory. A future project could be to try to port it to other simulators. By adding a component that sends and receive data to it. The visualization front-end would have to be modified to conform with the simulation data of the simulator.

(28)

Bibliography

[1] Introducing JSON. http://json.org/. Accessed: 2015-06-11.

[2] Kivy - Open source Python library for rapid development of applications that make use of innovative user interfaces, such as multi-touch apps. http://kivy.org/#home. Accessed: 2015-06-11.

[3] Kivy - User Guide - Philosophy. http://kivy.org/docs/philosophy.html. Accessed: 2015-06-11.

[4] Kivy framework. http://kivy.org/docs/api-kivy.html. Accessed: 2015-06-11.

[5] MGSim - Many-Core Simulation for Research Education. https://github.com/svp-dev/ mgsim. Accessed: 2015-06-11.

[6] PQ Labs - Multi-Touch G3 Basic. http://multitouch.com/download/datasheet/ DataSheet_g3_basic.pdf. Accessed: 2015-06-11.

[7] RapidJSON. https://github.com/miloyip/rapidjson/. Accessed: 2015-06-11. [8] The gem5 simulator. www.gem5.org/Main_Page. Accessed: 2015-06-11.

[9] H. Kok, A. Pimentel, and L. Hertzberger. Runtime visualization of computer architecture simulations. In Proc. of the Workshop on Performance Analysis and its Impact on Design (in conjunction with the 24th Int. Symposium on Computer Architecture), pages 15–24. Citeseer, 1997.

[10] M. Lankamp, R. Poss, Q. Yang, J. Fu, I. Uddin, and C. R. Jesshope. MGSim-Simulation tools for multi-core processor architectures. arXiv preprint arXiv:1302.1390, 2013.

[11] A. Roy, J. Xu, and M. H. Chowdhury. Multi-core processors: A new way forward and challenges. In Microelectronics, 2008. ICM 2008. International Conference on, pages 454– 457. IEEE, 2008.

[12] J. M. van der Woning. Interactive visualization and dynamic task management of many-core systems. Informatica Thesis, 2012.

(29)

APPENDIX A

Implementation and Libraries Link

A.1 Implementation

• Expanded MGSim implementation: https://github.com/6366295/mgsim

• Modified ManyMan front-end: https://github.com/6366295/ManyMan-for-mgsim

A.2 Libraries

• RapidJSON: https://github.com/miloyip/rapidjson

• GNU Readline library: http://tiswww.case.edu/php/chet/readline/readline.html • Kivy: http://kivy.org/

(30)

APPENDIX B

Descriptions of Communication messages

type content Description Response type client init - Initialization message. Requests

infor-mation about the simulator.

server init selection new sample vars Tell the simulator that a new selection

of variables has been made. The sam-ple vars field contains a list of patterns used to select the variables.

selection set

selection send - Tell the simulator that the ManyMan front-end application finished building a new visualization using the variables names returned by selection set.

-resume sim - Tell the simulator to resume the simu-lator for a number of steps set. If the number is zero, the simulator runs un-til the end of the loaded program.

-pause sim - Tell the simulator to pause the simula-tor.

-set step step Set the number of steps the simulator

will run.

-change delay delay Set the frequency of sending the

sim data message.

-Table B.1: Messages from the ManyMan front-end application

(31)

type content Description Response type server init name,cores,

frequency, sample vars, default vars

Sends information about the simula-tor itself. Also the variable names, which matches the patterns in de-fault vars. default vars is send with the current vars because otherwise the pattern used is unknown to the Many-Man front-end application.

-selection set sample vars The names of all variables the pattern sent from selection new matches.

selection send sim data data, status The data that matches the selected

variable names is send in the data field of the content. The status field con-tains the running status, current se-lected send delay and current sese-lected steps the simulator runs or will run. invalid message message For debugging purposes. This lets the

ManyMan front-end know when a mes-sage is send, which MGSim does not recognize.

(32)

APPENDIX C

Experiment Hardware Description

C.1 42” Philips Screen connected to an Apple Mini

• Multi-touch solution: G3 Basic[6] from PQ Labs. • Model: PT-MUT-42-6TP

C.2 Apple Mini running visualization

• Operating System: Windows 7 Professional 64 bit, SP1. • Processor: IntelrCoreTM_{i5-2520M, 2.5GHz.}

• Memory: 4 GB DDR3

• Graphics-card: AMD RadeonTM_HD6630M

C.3 Personal computer running visualization

• Operating System: Windows 7 Professional 64 bit, SP1. • Processor: IntelrCoreTM_{2 Quad Q8300, 2.5GHz.}

• Graphics-card: AMD RadeonTM_HD4630

C.4 Laptop running MGSim

• Operating System: Linux Mint 17.1 Rebecca • Processor: IntelrCoreTM_{i5-3210M, 2.5GHz.}

Multi-touch visualization of Multi- core simulations

Bachelor Informatica

touch visualization of

Multi-core simulations

Mike Trieu

August 10, 2015

Inf

orma

tica

—

Universiteit

v

an

Ams

terd

am

Contents

CHAPTER 1

Introduction

1.1

Motivation

1.2

Approach

1.3

Thesis Outline

CHAPTER 2

Background & Related Work

2.1

MGSim

2.1.1

Simulator Architecture

2.1.2

Monitoring Facilities

2.1.3

Component structure

2.2

ManyMan

2.2.1

Kivy

2.2.2

Application Structure

2.3

Related Work

CHAPTER 3

Software Architecture Design &

Implementation

3.1

Expanding MGSim

3.1.1

Communicator Threads

3.1.2

Interactive mode

3.1.3

Component And Variable Selection

3.1.4

Client Data Structure

3.1.5

Sending Simulation Data

3.1.6

Interaction from Visualization

3.2

Communication

3.2.1

Message Format

3.2.2

Protocol

3.3

Front-end

3.3.1

Main Window

3.3.2

Variable Selection

3.3.3

Saving Selections

3.3.4

Interaction with the Simulator

3.3.5

General Simulator Information

3.3.6

Component View