Electric: a tool for capturing, replaying and verifying vulnerabilities in web applications.

MSc Software Engineering

Master Thesis

Electric: a tool for capturing, replaying and

verifying vulnerabilities in web applications.

by

Hasan Bilen

July 31, 2018

University supervisor:

Mr J.A. (Jaap) van Ginkel

Assessor:

dr. A. M. (Ana) Oprescu

Host company:

Northwave

Company supervisor:

Mr E. J. (Edwin) Smulders

Abstract

A common problem with vulnerability reports is that the vulnerabilities described in such reports are not easily repeatable and verifiable. Repeatability is an issue because reports often contain hard to reproduce request/response data. For example, due to expiring cookies, special headers and tokens. Verifiability is an issue because verifying the presence of a vulnerability requires expert knowledge. After a fix is deployed, the penetration tester often retests the vulnerability.

Electric, the tool developed during this thesis, solves these issues by recording steps pene-tration testers take during their test. Electric is built on top of Electron [1] which is a GUI framework. At a later moment, Electric can reliably replay the steps against the live website of the System Under Test (SUT) in an automated fashion. At the end of a replay, verification mech-anisms can tell whether a vulnerability is present. Vulnerability scanners often lack completeness because they are only able to check a limited set of aspects. Electric does not lack completeness because it does not find vulnerabilities on its own. Instead, it lets the penetration tester find vulnerabilities while increasing repeatability and verifiability for those found vulnerabilities.

This paper covers the design decisions made to answer the following question: how do you design a tool that makes vulnerabilities in web applications repeatable and verifiable? The presented solution is a tool with an integrated browser that records user input of a penetration tester. Requests and responses that flow between the integrated browser and the web application are captured and modified by user-written scripts. User-written scripts are able to inspect the content and set environment values such that a verification mechanism can be implemented. A log tab displays results of these verifications.

1

Introduction

The US National Institute of Standards and Technology (NIST) estimates that the annual cost of inadequate infrastructure for software testing is between $22.2 and $59.5 billion [2]. Security testing is an activity to evaluate whether a software product can adequately prevent improper ac-cess to information [2]. Inadequate security testing would allow a data breach to occur. Ponemon states that the average total cost of a data breach is 3.62 million USD [3]. Of the data breaches they have studied, 47% were caused by malicious or criminal attacks.

Cyber attacks are increasingly large, sophisticated and disastrous [4, 5] and companies are actively working with security experts to prevent this type of attacks [6]. One aspect of security testing is penetration testing. Penetration testing is performed to evaluate the security of a particular system and find loopholes in that system [7]. Occasionally such evaluation leads to the discovery of vulnerabilities in the system (ex. unauthorised user data retrieval [5]). Penetration testing helps to find security issues early on and reduces the risk of vulnerability abuse by unauthorised parties.

Penetration testing can be performed with automatic tools, so-called ’vulnerability scanners’ (ex. Acunetix [8]). Antunes and Vieira [9] show that vulnerability scanners face two problems. The first problem is that the tools lack precision (i.e. a large portion of detected vulnerabilities are false positives). The second problem is that they do not detect enough vulnerabilities that are present in the application.

Huang et al. [10] state that traditional crawlers tend to skip web pages that require session management or user input. For this reason, vulnerability scanners with this kind of crawlers lack completeness. According to the paper, in security assessment completeness is an important issue because all data entry points must be correctly identified. Without completeness, certain parts of web applications may remain untested and vulnerable.

Manual penetration testing usually yields what vulnerability scanners could not find. There-fore, a manual penetration testing on top of vulnerability scanning leads to a test of the System Under Test (SUT) that exhausts the input space more thoroughly. Sometimes exploiting a vulner-ability requires many steps and repeating those steps manually can be time-consuming. Reports written by the penetration tester explain the necessary steps to reproduce vulnerabilities. Often they provide request/response data but the requests are not easily repeatable. Repeatability is made difficult due to expiring cookies, special headers, CSRF tokens or dynamic/temporary URLs. Therefore manual penetration testing lacks repeatability.

Moreover, when the application owner deploys a fix for the vulnerability, the penetration tester verifies whether the vulnerability is actually fixed. This is because the application owner might not possess the required expert knowledge to verify this on its own quickly. Therefore, manual penetration testing also lacks verifiability.

This research will cover the steps required to make the process of finding vulnerabilities manually better with respect to repeatability and verifiability without losing completeness. A tool will be developed that will capture manually performed actions of penetration testers so it can easily be replayed, verified and reported. The effectiveness of Electric will be measured by the ability to increase repeatability and verifiability.

In Chapter 1.1 the research questions of this thesis are discussed. In Chapter 2 papers are presented that are relevant and can provide insights to how a tool can be designed and which properties it should have. In Chapter 3 I address the design and implementation choices I made for the tool that I built and why they are important and well chosen to come to an acceptable solution. In Chapter 4 the design choices and final result are evaluated against the requirements and research questions. In Chapter 5 I conclude this thesis and give a summary. In Chapter 6, the future work chapter, I talk about the aspects that can be further worked on and how

someone else could improve on my work.

1.1

Research Questions

As mentioned in the introduction, traditional crawlers lack two aspects: precision and complete-ness. Furthermore, vulnerability scanners lack functionality to deal with pages that require user input or session management. This means that vulnerability scanners are not always useful to use for finding vulnerabilities and that it is often required to find vulnerabilities manually. However, manually finding vulnerabilities lack two aspects as well: repeatability and verifiability. In this thesis a way to improve on these two aspects is researched and for that the following research question is formulated:

Main research question: How do you design a tool that makes vulnerabilities in web ap-plications repeatable and verifiable?

Because it is hard to give an answer to this research question directly it is split into three parts. To find a way to improve repeatability you would have to find a way to reliably capture and reliably replay vulnerabilities. Both sub-questions should take into account user input and network communication. The sub-questions are formulated as follows:

• Sub-question 1: How can we reliably capture user input and network communication such that a replay mechanism could use the data to attain a similar outcome?

• Sub-question 2: How can we reliably replay captured user input and network communi-cation such that a similar outcome is attained when the captured data is replayed? The third part is verifiability. Verifying a vulnerability is checking whether it is possible to successfully trigger a vulnerability. This is useful, for example, when a web application is patched and the application owner desires to know if the vulnerability is still present. To make vulnerabilities verifiable you would have to find a way to reliably tell if the vulnerability is present after steps to trigger the vulnerability are executed. For this part the following sub-question is given:

• Sub-question 3: How can we verify that an earlier found vulnerability is still present?

The research questions will be answered through constructive research. Commonly found vul-nerabilities in penetration testing and manual vulnerability verification methods will empirically researched.

2

Related work

Related papers can be grouped into three different groups: papers that discuss a way for captur-ing/replaying user input, papers that discuss a way to capture/replay network communication and papers that give additional insights in how a tool should be designed and which stumbling blocks there are. In the table below a comparison of approaches by the authors above is given. Some authors that only provide general insights such as how to design a vulnerability scanning tool are excluded. You can see that Electric is able to cover all functionality and conducted tests later in this thesis also show that these functionalities work reliably. For example, Electric listens to window events in the capturing phase making it able to capture all user input reliably (see section 3.6.1). This is much different from Neasbitt et al. [11] that tackles the problem with a less reliable approach: reconstruction of user input from network traces. Electric is also different from Mickens et al. [12] as it is not a library that must be used by the application owner. Electric works with no setup or configuration required by the application owner. It is also different from Andrica and Candea [13] because it covers functionalities required for penetration tester and is not simply a tool that captures and replays user input.

Table 1: Comparison in approaches by different authors, u.i = user input, n.c. = network communication, vuln. = vulnerabilities

Approach Capture u.i. Replay u.i. Capture n.c. Modify n.c. Replay n.c. Verify vuln.

Electric _{X X X X X X}

Andrica and Candea [13] _{X X} Mickens et al. [12] _{X X}

Neasbitt et al. [11] _{X X}

2.1

User input

Andrica and Candea [13] show WaRR which is a tool that records and replays user input and consists of two main components: WaRR Recorder and WaRR Replayer. The recorder uses an embedded browser which means it is able to reliably capture user input. Electric also uses an embedded browser but captures user input by injecting JavaScript rather than interacting with the browser. Although it works differently, like Electric’s Replayer the WaRR Replayer is able to reliably replay user input and it makes use of a developer-specific web browser [13]. WaRR does not offer capture/replay functionality for network communication.

Mickens et al. [12] show a different approach to capture user input called Mugshot. Mugshot is a system for capturing events in a JavaScript program such that it can be replayed later. They describe why it is superior to existing systems. It works by making use of a JavaScript library. Electric also uses JavaScript for intercepting user input but does not require the developer to set it up. The Electric browser injects the code into every loaded page. The paper describes the various types of non-determinism and how they can be captured. Similar to WaRR it does not offer capabilities to capture network communication. However, Mugshot does support a wide variety of non-determinism making it more replayable for applications that heavily use JavaScript (ex. games). Mugshot is something that the application owner must add and therefore is unsuitable for penetration testing.

2.2

Network communication

Neasbitt et al. [11] show ClickMiner which reconstructs user-browser interactions from network traces. The result is similar to the two papers by Andrica and Candea [13], Mickens et al. [12] but rather than directly intercepting the user input it is reconstructed. In a user study, the authors concluded that ClickMiner was able to correctly reconstruct between 82-90% of user-browser interactions. Their measured false positives were 0.74-1.16%. This approach does not require a special browser, browser plugin or JavaScript library. For this reason, it is less reliable in determining what the actual user input was. ClickMiner is more suited to figure out how users use a web application with minimal data rather than how a vulnerability is triggered and how it can be replayed. For these reasons it is very different from Electric. The overhead is much lower in return for a less reliable solution, something that is not desirable for Electric.

2.3

Insights

Antunes and Vieira [14] propose a standardised and consistent procedure to design vulnerability testing tools. In another paper by the same authors [9] they show that penetration testing tools that perform vulnerability scans face two problems. The first problem is that the tools lack precision (i.e. a large portion of detected vulnerabilities are false positives). The second problem is that the tools do not detect enough vulnerabilities that are present in the application. These problems give insight in which things this thesis addresses when vulnerabilities are found manually with Electric.

Garn et al. [15] did a case study by using penetration testing tools ZAP and Burp suite to test various applications and compare the percentage of attack vectors that were detected out of the total present attack vectors. Something that can be noted here is that an automated way of finding vulnerabilities is undesired, Electric should only try to improve repeatability and verifiability.

Huang et al. [10] explain that traditional crawlers tend to skip web pages that require session management or user input. For this reason, tools that use such crawlers for automated pene-tration testing lack completeness. Without completeness, certain parts of web applications may remain untested and therefore vulnerable. Their paper describes the design of a crawler that mimics web browser functionality in order to test web applications. In doing so they are able to identify more data entry points (leading to more possible points of attack). In that way it is similar to the approach of replaying user input of Electric.

DURIC [16] starts by explaining three common vulnerabilities in web applications. Then he shows his approach of building a penetration testing tool called WAPTT to efficiently detect vulnerabilities. He explains how he implemented the tool and analyses the performance of the tool by comparing it against existing penetration tools by checking how many vulnerabilities each tool detects.

Finally there are a bunch of tools capable of capturing, modifying and replaying network traffic such as Wireshark [17], Fiddler [18], Burp Suite [19], OWASP ZAP [20] and mitmproxy [21]. Selenium is a popular tool to capture and replay user input.

3

Design and implementation

Electric is a tool for repeating and verifying vulnerabilities in web applications. It contains an embedded browser and is able to capture and replay actions needed to trigger a vulnerability. It also has mechanisms to verify vulnerabilities.

Electric consists of two important parts that are highly dependent on each other: capturing and replaying. Steps of a penetration tester need to be captured so they can be replayed. Replaying needs to happen reliably to be an effective tool as discussed in section 1. Capturing and replaying also have subparts: network communication and user input.

3.1

Requirements

A tool is only useful and effective if it meets requirements such as functionality and reliability. Northwave has set a list of requirements that Electric must meet and requirements that should be in the product but without it the product would still be usable. The requirements are set using the RFC 2119 requirements levels [22]. This RFC is a standardised way to prioritise requirements and therefore very useful.

Must haves

• It must be able to repeat actions that trigger vulnerabilities

– It must be able to capture request/response data – It must be able to modify request/response data – It must be able to execute custom requests – It must be able to capture user input – It must be able to replay user input

• It must be able to verify the presence of a vulnerability

Should haves

• It should make use of Electron

• It should display the captured request/response data • It should be easy to use

May have

• It may use a UI library like React [23]

• It may export reports in a format that Dradis [24] supports

3.2

Framework

Electron [1] is a framework for Node.js [25] that you can use to write a GUI in HTML and JavaScript. It works by launching a Chromium [26] based browser and pointing that browser to your web server or local HTML files. Electron has many features to configure the browser including but not limited to ignoring certificate errors. Certificate errors need to be ignored

otherwise the browser will refuse to serve the web page (more on this in section 3.3). Webviews1

[27] in Electron can be used to run a different web application within a web page. By doing this you can add browser elements outside of the webview such as tabs and URL bar to mimic a traditional browser. These elements can be used to control the behaviour of a webview.

Moreover, Electron is a multi-platform framework and is well documented. You can generate self-signed certificates and launch a proxy by making use of the APIs and libraries of Node.JS. In section 3.1 it was stated that making use of Electron is a requirement but an analysis also shows that Electron is a suitable candidate. Electric will use the Electron framework and React to create the UI of the control panel. Other UI libraries could have been chosen over React but React is easy to learn and has many libraries built on top of it.

3.3

Capturing network communication

In this section, the network communication aspect of the first research question will be answered. Capturing network communication is needed for vulnerabilities that require modification of re-quest data (ex. SQL injections [28]). This is tricky because over 70% of web applications communicate over SSL [29]. This means that if a proxy intercepts the SSL traffic with a naive approach, the intercepted data would be encrypted and therefore unreadable. To be a useful tool you need to be able to inspect the network communication in a readable format of this large portion of traffic. Below two approaches that tackle this problem are presented.

Self-signing SSL certificates When you are self-signing an SSL certificate you are able to inspect the network traffic while your browser still communicates over SSL. You launch a proxy and re-encrypt SSL traffic with your self-signed certificate. HTTP Strict Transport Security (HSTS) [30] would cause no issue because communication still happens over SSL. However, since the certificate is self-signed your browser would not trust the certificate as it does not know and trust the Certificate Authority (CA) that signed the certificate. The certificate can be trusted by adding the certificate of the CA that you have created earlier to your trusted set of CAs. Moreover, Certificate Transparency [31] detects SSL certificates that are unjustly issued and Chromium-based browsers (Electron falls into this category) block these certificates from being trusted. To circumvent this Electron can be configured to ignore certificate errors and treat self-signed certificates as valid.

Stripping SSL When you are stripping SSL you are essentially launching a proxy and serve any web application as if it did not have SSL. This requires some understanding of when and how the web application makes use of SSL. Replaying actions would become unreliable if the web application suddenly disables SSL (ex. a request to HTTPS URL that previously worked but now does not return a response). Here HSTS would cause a problem because HSTS forces an application to use SSL.

Design decision Stripping SSL has significant disadvantages. Self-signed certificates would normally cause problems but in section 3.2, the decision to use Electron was made. Electron is capable of treating the self-signed certificates as valid and therefore this approach works better than stripping SSL. Therefore the self-signed certificates approach is chosen.

3.3.1 Architecture capturing User Electric browser Proxy Log Web application Response Request’ Response’ Request/Response Request User interaction

SSL using self-signed certificate if Proxy-App connection uses SSL, otherwise plain-text

SSL using original certificate of web application if available, otherwise plaintext

Figure 1: Architecture of how network communication is intercepted. θ is unmodified, θ’ is modified.

In section 3.3 it was decided to use self-signed SSL certificates in combination with a proxy. In figure 1 the architecture of this approach is shown. Communication between the browser and proxy uses a self-signed certificate for requests with HTTPS URLs and is in plaintext for requests with HTTP URLs. The communication between the proxy and the web application is done using the default configuration of the web application. Because the proxy sits in the middle it is able to inspect requests and responses. The Electric browser is configured to use this proxy for all web applications it communicates with. The Electric browser resembles the basic features of a real web browser with the added benefit that controls such as opening a tab can be replayed using code. Request and response data is saved to a log for later inspection for example when a penetration tester is collecting evidence.

3.4

Modifying network communication

Some vulnerabilities require modification of request or response data. Electric is intended to work autonomously and therefore the modification needs to happen in real time and automatically. Programmatically changing the request/response objects is something that works really well in commonly used tools. Two tools mentioned in section 2 that have such functionality are Fiddler with FiddlerScript and Mitmproxy with add-ons. FiddlerScript provides two main APIs: onBeforeRequest and onBeforeResponse. Add-ons for Mitmproxy provide a similar API with request and response methods. The FiddlerScript functions, as their names already suggest, are executed respectively before the request is sent to the server and before the response is sent to the client (usually your browser). During those phases, data from the request or response can be modified. A similar API is implemented for Electric so users can quickly get familiar with

it through previous experience with FiddlerScript/mitmproxy add-ons or by quickly learning it. For logging and storing/accessing data a very basic and straightforward log and environment API is implemented. The log API is a wrapper of console.log and the environment API works as a JSON object. This would make the learning curve for new users small.

NPM2 _{offers a wide variety of proxy libraries but a library that Electric will use must meet}

the requirements listed below. • Support for SSL

• Support for modification of request/response objects • Fast

• Actively maintained

• Good usability for developers and penetration tester

Many libraries failed to meet these requirements. The two libraries that met the requirements were Hoxy [32] and AnyProxy [33]. Both could have been used in Electric. They were equally easy to use and well documented. Since performance is also a requirement they were both tested on performance and Hoxy was chosen because it performed better. Hoxy is a proxy library that automatically generates certificates and allows you to edit the request/response object using its API.

3.4.1 APIs

An overview of the implemented APIs and a short description of what they do is given below. • Log API: can be used to log values for debugging or to display vulnerability verification

messages

• Environment API: can be used to store or retrieve values to use between requests • Modifier API: can be used to modify requests/responses

Log API definition

log(param1, param2, ..., paramn) = ∅, n ∈ N (1)

The log method (definition 1) is a wrapper around the native JavasScript method console.log [34] and additionally saves logged values to display them in the user interface. The interface can be used to display relevant information such as messages that tell if a vulnerability was triggered, more about that in section 3.8.

Environment API definition

env.x = y (2)

env.x (3)

The environment can be used to store variables using definition 2 and retrieved again using definition 3. This API is useful when certain data (i.e. CSRF tokens) or configuration is needed in a later phase.

Modifier API definition

onBef oreRequest(log, env, request, response) = {env0, request0, response0} (4)

onBef oreResponse(log, env, request, response) = {env0, request0, response0} (5) Method of definition 4 is executed before the request is sent to the web application and can be used to change the request object or to directly send a response back to the client. Method of definition 5 is executed before the response is sent back to the client and can only be used to retrieve values of the request and retrieve/edit values of the response. Request and response objects are created by Hoxy and examples using their API can be found in their documentation [35].

Example API usage Suppose you are penetration testing a web application that has a form that is vulnerable. You would like to edit the field name. You also want to keep track of the number of requests made by creating a counter environment variable. Finally, you want to log the current counter after every request. Those things can be achieved with the JavaScript code in figure 2.

1 f u n c t i o n o n B e f o r e R e q u e s t ( log , env , request , r e s p o n s e ) { 2 // E d i t r e q u e s t b o d y f o r m p a r a m e t e r 3 if( r e q u e s t . b o d y T y p e === ’ p a r a m s ’) 4 r e q u e s t . b o d y . n a m e = ’ H a s a n B i l e n ’; 5 6 // U p d a t e r e q u e s t c o u n t in e n v i r o n m e n t 7 if( env . r e q u e s t C o u n t ) 8 env . r e q u e s t C o u n t = env . r e q u e s t C o u n t + 1; 9 e l s e 10 env . r e q u e s t C o u n t = 1; 11 12 // Log c u r r e n t c o u n t e r 13 log (’ R e q u e s t c o u n t = ’, env . r e q u e s t C o u n t ) ; 14 }

Figure 2: Example onBeforeRequest usage

And similarly, you could log a message when certain strings occur in the response, see figure 3.

1 f u n c t i o n o n B e f o r e R e s p o n s e ( log , env , request , r e s p o n s e ) {

2 // V e r i f y t h a t a c e r t a i n s t r i n g is p r e s e n t in the r e s p o n s e b o d y

3 if( r e s p o n s e . b o d y T y p e === ’ s t r i n g ’ &&

4 r e s p o n s e . b o d y . i n d e x O f (’ M y S e c r e t P a s s w o r d 1 2 3 ’) ) 5 log (’ S e c r e t p a s s w o r d f o u n d ’) ;

6 }

Browser _applicationWeb Proxy Request onBefore Request Request’ Response’ onBefore Response Response

Figure 4: Architecture of how requests and responses are modified in the proxy. Logging omitted from this figure.

In figure 3.4.1 the architecture of the proxy is shown. The proxy makes use of the onBe-foreRequest and onBeforeResponse APIs that are executed by Hoxy. Users of Electric would implement scripts that are called from this API and modify requests and responses that go through the proxy.

3.5

Replaying network communication

Penetration testers often prove the presence of a vulnerability on the basis of requests with malicious input. Electric needs to send that malicious input in some way during a replay to successfully re-trigger the vulnerability. Before an explanation is given why some requests should not be replayed, consider the following distinction between two types of requests:

• A request that is made by the browser (ex. as a result of clicking on a menu item on the page or resources such as stylesheets that are needed to render the page)

• A request that is made from a tool (ex. Burp Suite or Electric)

In general, if we replay requests exactly as how we captured them you would face several problems: • Pages can be removed (i.e. http://example.com/profiles?userId=1 can be non-existent

after user account with id 1 is removed) • Cookies expire

• Login credentials change

• Tokens can expire (ex. API keys or Cross-Site Request Forgery (CSRF) tokens [36]) The requests made by the browser during a capture will be made again if we can reliably replay user input. For example, suppose you click on a menu item and your browser opens a new page. Your browser would request that page automatically. That request does not need to be replayed as it would be made again if we would replay the click that clicks on the menu item. The problems listed above would not occur when requests are made naturally by the browser.

3.5.1 Manually triggered network communication

Earlier it was stated why requests triggered by the browser should not be replayed and why they should not be replayed. The other type of requests are requests that are triggered manually by a tool such as Electric. In section 3.5 it was made clear why it is undesirable to replay the request literally. So the question is: how could one capture and replay a manually triggered request? Electric needs to execute manually triggered requests during capture automatically during replay. For consistency and simplicity, this is also done with code. Below are two options that could be used to tackle this problem.

Injected JavaScript Inject JavaScript written by the user of Electric into the page that is currently viewed. The code could make requests using the context of the web application and therefore certain things like headers or cookies are automatically set.

• Upside: Cookies and headers are automatically set.

• Downside: requires the user to already have an open page and would be slow because it runs on the UI thread. Furthermore, you still need to set API and CSRF tokens.

Request library Provide a request library and execute requests from the core of Electric. • Upside: better performance because requests are not done on the UI thread. Also keeps

the possibility to easily edit configuration/environment variables of Electric open. • Downside: cookies, headers and tokens must be manually set.

Design choice The second approach is better because it would perform better and is more future proof. Therefore Electric will use the second approach.

In some cases, the code would need to set cookies, tokens and possibly take into account chang-ing pages. When the code needs data this data can be accessed from the environment uschang-ing the environment API. Data can be saved to the environment (see definitions 2) and later accessed (see definition 3) to extract data such as cookies. Code for use cases like this can be implemented using the executor API (definition 6) and an example using that API is given in figure 5.

Executor API definition

executeScript(log, env, requestLib) = {env0} (6)

1 f u n c t i o n e x e c u t e S c r i p t ( log , env , r e q u e s t L i b ) { 2 let C S R F t o k e n = env . C S R F ; 3 r e q u e s t L i b (’ h t t p :// uva . nl /? c s r f = ’ + C S R F t o k e n , f u n c t i o n ( error , r e s p o n s e , b o d y ) { 4 log (’ S t a t u s c o d e of r e s p o n s e : ’, r e s p o n s e . s t a t u s C o d e ) ; 5 }) ; 6 }

Figure 5: Example of how the executor can be used.

In figure 5 a CSRF token is retrieved from the environment and used to craft a URL. A request is made to this URL and the status code of the response is logged. In figure 6 a screenshot of the log window after execution of the script is given.

Figure 6: A screenshot of the log window after execution of the user script

3.6

Capturing user input

The sophistication of modern web applications vastly increases the possible interactions that users can perform. HTML code could change in reaction to user events [13]. An important question that needs to be answered is: what kind of user input needs to be captured to effectively replay vulnerabilities? There are several types of user input that Mickens et al. [12] have listed. Below you can find a list of different types of user input that needs to be supported to be able to replay vulnerabilities later on. A different table is given of events that will not be supported together with arguments why that is.

Supported events

1. Mouse clicks: page loads can be attained when URLs are clicked

2. Keyboard input: specific pages can be accessed if input meets certain conditions

3. Timing (Load): wait until the DOM is ready. Certain DOM elements could not be found if JavaScript is executed too early.

4. Timing (Delay): wait until animations are finished. Certain DOM elements could not be found or DOM elements of a previous state are retrieved if JavaScript is executed too early. 5. Scrolling: a replay is less useful if you cannot see the clicks happening. A user could click on a button somewhere on the bottom of the page that is not visible without scrolling. If page scrolls are replayed button clicks would become visible.

Not supported events

• Text selection (highlighting text with a mouse)

Vulnerabilities in web applications do not depend on text selection. If any data on the page is required you can use API definition 5 to extract that data with a possibility of showing the data by logging it using API definition 1.

• Interaction with Flash/Java elements

Outdated plugins. Major web browsers Chrome, Microsoft Edge and Safari have taken steps to block these plugins [37, 38, 39].

• Randomness

Mickens et al. [12] say that games often make use of randomness but in web applications tested by Northwave randomness plays no role.

3.6.1 How to capture

Capturing is interrelated with replaying: without enough information, a replay of user input would fail to attain a similar result as when it was captured. Clicking a button and typing into a field should also work when the browser window has a different resolution. Below two approaches are presented that one could use to capture user input of clicks and text input reliably.

Interacting with the Electron API Interacting with the Electron API is simple and allows you to click on given coordinates or input text. However, this approach is unreliable when the page changes even slightly. Buttons or input fields can have different positions when the browser window is resized. If this approach is chosen we can only apply it to web applications where the content does not update as often and resolution stays the same between runs across all machines running Electric. Suppose that a button is positioned at coordinates x, y : 100, 200 during a capture. Another user wants to replay the capture but uses a different screen resolution. For that particular user the button might be positioned at x0, y0 : 150, 250 during the replay. The replay would fail to click on the button. The Electron API clicks on the coordinates that were determined during capture and the button is not there.

Injecting JavaScript During the capturing phase JavaScript code is injected into loaded pages. The code listens for DOM events and retrieves the DOM target of these events. Previous research [40] states that you can determine a unique CSS selector [41] or XPath for the target DOM element. Both are able to define the position of the element in the HTML tree. With a solution like that, you could find the DOM element on the page on a next run even if the content is slightly changed or when the resolution is different. The position of the DOM element within the tree stays the same between runs. That is because the selectors describing a DOM element do not change when the resolution is changed or if the content is slightly different. An example of a CSS selector is given in figure 7 and an example of an XPath selector is given in figure 8.

1 # s e a r c h > div . search - box > s p a n > i n p u t . tt - i n p u t

Figure 7: Example CSS selector of the search input field on the home page of uva.nl

1 / / * [ @id =" s e a r c h "]/ div [ 1 ] / s p a n / i n p u t [2]

Figure 8: Example XPath selector of the search input field on the home page of uva.nl

Design choice Pages are dynamic and users have machines with different resolutions. Clicking on certain coordinates breaks down quickly. Injecting JavaScript to determine the selector during the capturing phase and using it to find the element during the replaying phase is a reliable way to capture and replay user input. CSS selectors are slightly easier to read than XPath selectors. For these reasons, Electric uses the injecting JavaScript approach together with CSS selectors for capturing user input.

Another hindrance is how DOM events can be captured. Previous research [12] explains how the W3C of DOM events work. Browsers process DOM events in two stages: capturing and bubbling. Event listeners are able to cancel DOM events meaning that ’bubbling’-type

event listeners higher in the DOM tree are unable to receive them. Ideally, you would attach one listener to the window to receive all events but you would miss events that are cancelled. Mickens et al. [12] state that this can be fixed by using the useCapture flag to capture events in the capturing phase. This means you could listen to events before any other listener gets the chance to cancel it. The flag is also how this problem is solved in Electric.

3.7

Replaying user input

Mouse clicks, keyboard input and scroll events are logged with a CSS selector. The target DOM element that corresponds with this selector can be retrieved with the querySelector API [42]. In the examples below the target variable is the variable that stores the target DOM element retrieved with this API. JavaScript is injected during replaying similar to how it is done during capturing. In figure 3.7 a table of user input is given together with how they are replayed:

Event type Can be replayed with Mouse click target.click

Keyboard input target.value

Timing (load) document.onreadystatechange Timing (delay) setTimeout

Scroll target.scroll

Table 2: Different types of user input and replay approach

To reliably replay user input the replay script needs to meet these requirements:

• Execution synchronously and in chronological order: actions could depend on each other and must be done in the same order as when it was captured (i.e. a submenu could be non-existent until the menu button is clicked and dynamically added to the DOM). Action execution should also have no overlapping as two actions could counteract each other. • Execution after the page is done loading: interaction with DOM elements could fail if it is

done before the DOM is ready.

To meet these requirements, actions are replayed in the same order that they were captured. The event replay script notifies the main script when an action is done executing. When the main script receives this notification it sends the next action back to replay. The event replay script only executes an action if the DOM is ready.

A feature not necessary to meet the requirements of Electric but certainly useful is the functionality to simulate the mouse cursor. A fake mouse cursor image is injected into the page and moves in the direction of the target DOM element. The cursor disappears when a click action is executed. This is useful for demonstration purposes as it is easier to understand what is happening in the replay.

3.8

Verify

The third sub research question is:

How can we verify that an earlier found vulnerability is still present?

Similar to unit testing of software, verification should tell the user if the execution was successful. Here successful means that the tested vulnerability was triggered successfully. You can use visual

feedback to say a vulnerability is present and that the web application is vulnerable. However, it must be made clear that you cannot conclude that a system is not vulnerable when a verification fails to prove the presence of a vulnerability. For example, suppose an application owner fixes an XSS exploit by applying a case-sensitive string check. An exact repetition of the steps that were made initially would fail, the vulnerability would not be triggered. If you would rewrite the script with uppercase characters you could successfully trigger the vulnerability again. Saying that the application is vulnerable when a repetition of actions is able to trigger the vulnerability is a valid statement (except in some cases, see section 3.8.5). Saying that an application is not vulnerable when you could not verify the presence of a vulnerability is a false statement.

Below a list of common types of vulnerabilities is given. For every vulnerability, a way of verifying a successful execution of that vulnerability is also given. The list of vulnerabilities is not complete but selected based on what the most commonly found vulnerabilities are such as those listed in the OWASP Top 10. Less common and not listed vulnerabilities can be verified in a similar way.

3.8.1 SQL injection [43]

With SQL injections you can execute arbitrary SQL queries against the database of the web application. If a static SQL query is executed you could check responses for a particular string to verify if the SQL query is executed. An example SQL3 _{query that would allow this is given}

below.

SELECT concat(’Universiteit’, ’van’, ’Amsterdam’, 2000+18); The SQL query above returns the following string:

”UniversiteitvanAmsterdam2018”

Verification of the SQL injection is possible by checking the response body in the browser against this expected query result. To give the user visual feedback an onBeforeResponse script (figure 3) is written. The script would inspect the response body and use the log API (definition 1) to write ”SQL injection successful, expected query result found” to the log when the string is found. Setting a variable in the environment is also possible. The user would view the log window at the end of a replay to see if the vulnerability is present.

3.8.2 Cross-site scripting (XSS) [44]

With XSS exploits you can execute arbitrary JavaScript code in a victim’s browser4. Electric is built in such a way that you would separate vulnerabilities by adding captured data into different evidence sets (see section 3.9). Therefore the last step of an evidence set for an XSS vulnerability would be the execution of the JavaScript code. Electric captures alerts into the list of actions but also shows the alert itself within the browser. Both of these types of visual feedback could be used to verify the presence of the vulnerability.

3.8.3 Path Traversal [45]

With path traversal files from different directories can be read than it was intended. Verification of such exploit would be done by checking for certain strings or patterns that occur in the response of accessed files. For example, if you access a password file or a file that you have uploaded yourself you can search for strings in the response when you access those files. The occurrence of those strings can be logged using the log API.

3_{MySQL 5.6}

3.8.4 Cross-site request forgery (CSRF) [46]

A CSRF exploit can be used to execute arbitrary URLs by computers of victims. onBefor-eRequest or onBeforeResponse APIs can be used to intercept requests/responses and URLs. The intercepted data can be logged to verify if a certain request occurs and whether a web applica-tion is vulnerable to this type of attack. For example, if a bank website is vulnerable to CSRF exploits you could verify if a request to bank website is executed by making use of the APIs.

3.8.5 Limitations

It is important to question if the above methods are a reliable way of verifying a vulnerability. The static query for verification of a SQL injection mentioned earlier is specifically crafted so that the query result should only be obtained if the SQL syntax is understood by some SQL compiler. Normally the database is the only actor that could understand this syntax and produce the special string. However, just because the special string is returned does not mean the application is vulnerable. A honeypot [47] could respond with disinformation making attackers believe that there is a vulnerability. In a case like this, Electric would be unable to verify the SQL injection vulnerability. The nuance is that penetration testers usually explicitly ask customers to shut off things like intrusion protection systems and web application firewalls [48, section 4.1.7]. Moreover, it is the task of the penetration tester to make sure the verification process is correct. Electric simply executes the verification steps and checks whether the result is equal to some expected value.

3.9

Evidence sets

A common occurrence during penetration testing is finding several vulnerabilities. These vul-nerabilities are reported as different sections in documents so readers are not blown away by the vast amount of information. Electric’s approach for this is evidence sets. Evidence sets can be exported, imported and replayed. An evidence set is specific to one vulnerability and consists of the following elements:

• Evidence set name • Actions – User input – Request/Response • Scripts – onBeforeRequest – onBeforeResponse – executeScript

3.10

Displaying request/response data

Burp Suite, a tool that was mentioned in section 2, is a penetration tool used by many pene-tration testers. Penepene-tration testers at Northwave all use this tool during a penepene-tration test. A requirement for Electric is to display request/response data. A logical implementation would be to display them using the HTTP 1.1 specification (RFC 2616) [49]. Burp Suite and other

Figure 9: Import/Export tab

commonly used tools also display requests and responses using this specification. Electric uses this specification to display requests. In figure 10 and 11 a screenshot of the request and response inspect view in Electric is given.

Figure 10: Request inspect view of Electric for the first request to http://uva.nl

3.11

Reporting

Vulnerabilities in web applications are often reported using one of the two methods below. Elec-tric needs a way of reporting found vulnerabilities. By looking at what is commonly used a design decision can be made for how Electric will do reporting.

Export of vulnerability scan In the introduction, it was already mentioned that vulnerabil-ity scanners can be used to find vulnerabilities. Vulnerabilvulnerabil-ity scanners are automated tools that look for known vulnerabilities and do not require a lot of effort to run. Whey they are finished they are capable of exporting the found vulnerabilities to a report.

Manually written document When a penetration tester performs a penetration test he/she has to manually write a report containing the vulnerabilities that are found. Sometimes reporting and collaboration tools such as Dradis [24] are used to make it easier to write reports.

Design decision The desire of Northwave is to generate automated reports preferably using tools that are currently used. For this reason, Electric will also provide functionality to export evidence sets to a format that Dradis supports. Dradis currently supports the markup language Textile [50]. An example of an exported and rendered Textile report generated by Electric is given in figure 12.

Figure 11: Response inspect view of Electric for the first request to http://uva.nl

4

Results

In this chapter tests will be conducted to verify the presence and reliability of functionalities discussed in earlier chapters. First the setup details are given so the test results can be verified. Each test will cover a subset of functionalities and explain how the test is executed and what the result of the test is. Finally, every requirement is discussed to see if the tool meets the requirements.

Figure 13: Screenshot of Electric, action tab

In figure 13 a screenshot of Electric is given. On the top, there is a navigation menu bar that you would use to switch between tabs. On the action tab actions can be viewed in the left half of the page. On the top of the page, there are toggles to filter request actions and user interaction actions. There is also a drag and drop for adding delays to an evidence set. On the right half of the action tab, you can create new evidence sets, add/remove actions to/from an evidence set or replay an evidence set. When you click on an action the action is inspected.

4.1

Tests

4.1.1 Setup

In table 3 details of the environment that is used to execute the tests is given.

System MacBook Air (13-inch Mid 2012) OS OSX High Sierra 10.13.4

Node.js 10.3.0 x64 Electron 2.0.2

Electric Electric v0 2 0.zip (MD5 = 724ded24392b46b9ec1687695f3fd963)

Table 3: Setup conditions

4.1.2 Test: scrolling

A test website is created that contains a large list of items. The test is executed as follows: 1. Launch Electric.

2. Navigate to the test website using the Electric browser.

3. Scroll until the first item that is visible in the window is list item 30. 4. Create a new evidence set in the control window of Electric.

5. Drag and drop every action in the current recording to the new evidence set. 6. Press REPLAY SET.

Expected result A replay browser is launched that scrolls until the first item that is visible is list item 30.

Actual result Identical to the expected result.

4.1.3 Test: button clicks

A test website is created that contains one button. A row of text is added to the page when the button is clicked to give visual feedback when a button is clicked. The test is executed as follows:

1. Launch Electric.

2. Navigate to the test website using the Electric browser. 3. Click on the button three times.

4. Create a new evidence set in the control window of Electric.

5. Drag and drop every action in the current recording to the new evidence set. 6. Press REPLAY SET.

Expected result A replay browser is launched that clicks the button three times. This can be verified by counting the number of rows that are added to the page.

Actual result Identical to the expected result.

4.1.4 Test: JSON and logging

A test website is created that returns a JSON object with field ’a’ set to 42. The test is executed as follows:

1. Launch Electric.

2. Create a new evidence set in the control window of Electric. 3. Go to the Request Modifier tab in the control window.

4. Select the newly created evidence set and click on onBeforeResponse. 5. Write a script that logs the JSON response.

6. Navigate to the test website using the Electric browser.

7. Drag and drop every action in the current recording to the new evidence set. 8. Press REPLAY SET.

9. Go to the log tab.

Expected result The JSON object that we return in the test website is logged to the log tab.

Actual result Identical to the expected result.

4.1.5 Test: form

A test website is created that contains an HTML form. When the form is submitted the same page is shown but with all the posted form parameters shown below the form. The test is executed as follows:

1. Launch Electric

2. Navigate to the test website using the Electric browser.

3. Verify that the website works as intended by displaying the input values of the form when it is submitted.

4. Create a new evidence set in the control window of Electric. 5. Go to the Request Modifier tab in the control window.

6. Select the newly created evidence set and click on onBeforeRequest. 7. Create a script that modifies the request params.

8. Drag and drop every action in the current recording to the new evidence set. 9. Press REPLAY SET.

10. Verify whether the original params you had put in have changed by checking the request params that are displayed on the test page.

Expected result The request params that are shown are those that are set in the onBefor-eRequest script.

Actual result Identical to the expected result.

4.1.6 Test: alert

A test website is created that executes alert(1)5_{. The test is executed as follows:}

1. Launch Electric

2. Navigate to the test website using the Electric browser

Expected result The user is prompted with an alert box with the message ”1”. The alert and the arguments of the method call can be inspected in the action tab of Electric.

Actual result Identical to the expected result.

4.1.7 Test: CSRF tokens

A test website is created that consists of two pages. The first page contains a non-visible CSRF token and the second page contains a message that can only be accessed if the CSRF token is given. The test is executed as follows:

1. Launch Electric

2. Navigate to the test website using the Electric browser.

3. Inspect the response in the action tab to figure out how to retrieve the CSRF token. 4. Create a new evidence set.

5. Navigate to the Request Modifier tab and click on onBeforeResponse.

6. Write a script for the current recording and the new evidence set that saves the CSRF token to the environment. The CSRF token is retrieved from the response body.

7. Click on onBeforeRequest

8. Write a script for the current recording and the new evidence set that changes the CSRF token field of the request to the one saved in the environment.

9. Visit the second page with a dummy CSRF token.

10. Drag and drop every action in the current recording to the new evidence set. 11. Press REPLAY SET.

Expected result The replay browser launches. The first page is loaded. The CSRF token on the first page is saved to the environment. The second page is loaded but is edited with the correct CSRF before it reaches the server. The hidden message can be viewed because the given CSRF token is valid.

Actual result Identical to the expected result.

4.1.8 Test: export/import evidence sets

For all of the above tests in which evidence sets were created a new test is executed as follows: 1. Navigate to the import/export tab while Electric is still running after the previous test. 2. Select the evidence set that was created.

3. Click export evidence and save the evidence file to your disk. 4. Relaunch Electric.

5. Navigate to the import/export tab. 6. Click on browse electric evidence file. 7. Select the file that you saved earlier. 8. Navigate to the action tab.

9. Press REPLAY SET on the import evidence set.

10. Verify that the expected result of the replay browser is equal to the result of the previous replay.

11. Verify that the request modifier scripts, executor script and contents of the log tab are identical to the first run.

Expected result The replay browser launches and performs exactly the same actions as the first run. The result is the same as the first run. The request modifier scripts, executor scripts and log tab contents are identical to the first run.

Actual result Identical to the expected result.

4.1.9 Test: capturing request/responses

In this test, Electric will be used to visit websites with different security configurations. The request and response data will be inspected to see if the network communication of the websites is captured and correctly displayed.

Domain SSL HSTS

http://uva.nl/ no no https://sidn.nl/ yes yes https://northwave.nl/ yes yes https://nos.nl/ yes no https://utwente.nl/ yes yes https://tudelft.nl/ yes no

Table 4: Websites with their configuration used in this test

Network communications of all the tested websites are captured successfully. In figure 14 an example is given of how captured network communication is displayed in Electric.

Figure 14: Action inspect for a request action of sidn.nl

4.1.10 Test: executing custom request using executeScript

In this test, two scripts will be executed. In the first script, a variable x will be set into the environment with value software engineering. In the second script, that variable will be used to create a custom request to http://www.uva.nl/en/search?q=ENVIRONMENT VARIABLE X&type=subject. Finally, the request/response data is viewed in the action inspect.

Figure 15: Using execute script to execute a custom request

4.1.11 Test: test lab certifiedsecure.com

The website certifiedsecure.com is a lab environment where penetration testers can train for ex-ecuting vulnerabilities such as SQL injection and cross-site scripting. One particular training is called the iDB challenge. In that challenge, the penetration tester needs to find hidden creden-tials, login into the hidden admin panel using those credentials and execute a SQL injection in that admin panel. Electric will be used to see if it can successfully replicate the SQL injection with the preconditions such as logging into certifiedsecure and logging into the admin panel of the challenge. The test is executed as follows (spoiler alert):

1. Navigate to the certiedsecure.com website 2. Login using your certifiedsecure.com account 3. Navigate to the iDB challenge

4. Navigate to the hidden readme on the admin panel

5. Inspect the response in the action tab to see how the username and password can be extracted from the response body

6. Navigate to the Request Modifier tab and click on onBeforeResponse 7. Write a script that extracts and saves the credentials in the environment 8. Navigate to the hidden readme on the admin panel again

9. Go to the login page of the admin panel 10. Fill in dummy credentials

11. Inspect the request in the action tab to understand what needs to be modified 12. Navigate to the Request Modifier tab and click on onBeforeRequest

13. Write a script that modifies the credentials that are sent using the credentials in the environment

14. Go to admin page that is vulnerable of the SQL injection and edit the query param to trigger the SQL injection

15. Navigate to the Request Modifier tab and click on onBeforeRequest

16. Add code that inspects the response body and verifies the presence of the SQL injection and logging the result to the log tab (see section 3.8.1 how it can be verified)

17. Navigate to the action tab.

18. Create a new evidence set with the created scripts 19. Press REPLAY SET on the import evidence set.

20. Visually verify that a SQL injection has occurred by looking at the page and the log tab

Expected result The replay browser launches and navigates through the website. The cre-dentials are saved and used while logging into certifiedsecure.com and the admin panel of the challenge. The page that is vulnerable to SQL injections is opened and a SQL injection is suc-cessfully triggered. The page contains the SQL injection query result and the log tab contains text verifying the presence of the vulnerability.

Actual result Identical to the expected result.

4.1.12 Test: exporting Dradis supported Textile reports

For every test above, a Textile report is generated of the evidence set containing the actions. Then the report is manually checked to see if the report accurately reflects what happens.

Expected result Every report contains a complete list of actions that are present in the evidence set. For every action, a small description is given and the report accurately reflects what happens when the evidence set would have been replayed. Every action type is manually checked to see if correct syntax is produced.

Actual result Identical to the expected result.

4.2

Requirements

In section 3.1 a list of requirements was given. In section 3 an elaborate explanation is given how Electric is designed and implemented to meet these requirements. In this section, Electric is evaluated with regards to these requirements.

Usage of Electron Electric makes use of Electron. Additionally React is used to make the UI codebase maintainable and was chosen over others because it was the preferred UI library of Northwave.

Ability to capture request/response data In section 3.3 it was already stated that over 70% of web applications communicate over SSL. Therefore to effectively capture requests/re-sponses Electric needs to be able to support this large set of web applications. Websites using no SSL, SSL or SSL with HSTS all work as expected with Electric. Request and response data is captured such that implementations for logging and modification work.

Ability to modify request/response data Electric is able to modify request/response data successfully. It is able to distinguish between different content types and allows modification of anything related to requests/responses such as URLs, response codes, headers and content. The modification is done by writing JavaScript code using the API defined in section 3.4. The API is simple and concise but a limitation is that it is the only way to modify request/response data. This might be an issue for those that have no programming experience.

Ability to execute custom requests Requests can be triggered from code with a simple and concise API (definition 6). This functionality is to serve those use cases where only modifying requests triggered naturally by the web application is insufficient. Similar to the ability to modify request/response data this works great but requires a little programming knowledge.

Ability to capture user input Electric is able to capture user input that is most relevant for penetration testing use cases. A selection of user input events is made and therefore web applications that require text selection or outdated plugins might be untestable. However, as it was already stated in section 3.6 the percentage of web applications where this is the case is so low that it is not worth it to spend time on supporting those events. Instead, more time is spent on reliability and other features. The user input events that are supported by Electric work as intended.

Ability to replay user input Electric is able to replay user input reliably because a CSS selector is used to find DOM elements. Standard browser APIs are used to interact with these DOM elements. No overlapping of actions occurs because actions are executed after any previous action is done executing. Moreover, actions are only executed after the page is done loading to allow the DOM to be ready. Delay actions can be added to wait when animations block DOM loads for happening or simply for demonstration purposes.

Ability to verify the presence of a vulnerability Verifying the presence of a vulnerability is tricky because web applications could make use of honeypots as explained in section 3.8.5. Nevertheless, Electric would work in common use cases. Vulnerabilities are verified in a similar way a human would and the checks are easy to implement.

Ability to export to a report document To export a report in Electric all you need to do is select an evidence set and click the export report button on the import/export tab. The report uses the Textile format which is used by the Dradis platform.

Ability to export/import captured data Electric serializes the list of actions (generated from network communication and user input) and scripts of a particular evidence set to a JSON file. This JSON file can be imported again by someone else. JSON is chosen because it is a format easily editable by Electric and escaping of special characters is done automatically.

4.3

Research questions

4.3.1 Research question 1: How can we reliably capture user input and network communication such that a replay mechanism could use the data to attain a similar outcome?

User input is captured by injecting JavaScript into pages loaded by the embedded browser. The JavaScript code listens for window events and saves the target elements with a CSS selector. CSS selectors reliably describe the target element. Network communication is captured by using a proxy and to support SSL inspection the proxy self signs certificates for requested domains.

4.3.2 Research question 2: How can we reliably replay captured user input and network communication such that a similar outcome is attained when the captured data is replayed?

Replaying user input is done by injecting JavaScript into pages loaded by the embedded browser. For every user interaction that needs to be executed the CSS selector is used to find the target element on the page and native JavaScript functions are executed to trigger actions. Network communication does not need to be replayed for type 1 when user input is reliably replayed. Type 2 requests are replayed using a request library.

4.3.3 Research question 3: How can we verify that an earlier found vulnerability is still present?

Verification of vulnerabilities is done in two ways. For vulnerabilities that can be verified using data patterns a modifier API is given to inspect request and response data. Data can be logged or temporally saved using environment variables. For vulnerabilities that rely on alert pop-ups these pop-ups are shown in the embedded browser and in the action list.

5

Conclusion

The first challenge this research tackled is capturing network communication. To capture network communication Electric launches a proxy and an integrated browser that communicates through that proxy. It was noticed that almost three-quarters of web applications currently use SSL. To support this large portion of web traffic, Electric generates SSL certificates on the fly and disables SSL verification for the integrated browser. In the process of implementing this, it became more clear what the relation between Certificate Authorities and SSL certificates is. Modification of network traffic is done by transforming request/response objects with the onBeforeRequest and onBeforeResponse modifiers that are accessible from an API that Electric provides. These APIs expose request or response objects generated by Hoxy and can be modified with simple JavaScript code. Network communication is automatically replayed for requests that are executed by the browser as a result of user input but are procedurally replayed for custom requests. These custom requests are written with JavaScript, executed during capture and are executed again during a replay.

Capturing of user input happens by injecting JavaScript into the page before any other script is executed. This script listens for events and logs them. Event targets are saved using a CSS selector. CSS selectors reliably describe the position of target DOM elements in the HTML tree. Replaying of user input happens similarly by injecting JavaScript into the page. The script listens for actions that it needs to replay. The target DOM element is calculated with the CSS selector that was determined during capture. JavaScript is used to interact with the target and to replay user input.

Verification of vulnerabilities is done using user scripts. User scripts are able to inspect request/response objects and look for patterns. Based on patterns environment values can be set and messages can be logged. Visual feedback like logged messages can tell the user if a vulnerability is present. XSS exploit alerts are logged to the action list. All captured data is saved to an evidence set which also contains the user scripts.

Evidence sets can be exported as JSON so it can be imported later on or it can be exported to a report that Dradis supports. Request/Response data can also be inspected in the application using the HTTP 1.1 specification.

Finally, the conducted tests show that the application is able to reliably capture, replay and verify vulnerabilities. Therefore the design decisions made are adequate to provide a solution to the research questions.

6

Future work

In this thesis, many aspects of Electric are discussed and although Electric meets the requirements that Northwave has set there is still room for improvement.

Firstly, Electric could be improved by researching ways to reliably capture and replay different types of non-determinism such as randomness. It was mentioned that tested web applications by Northwave do not depend on randomness but this might be different for other organisations or individuals that for example do penetration testing of web-based games. Capturing and replaying randomness is purposely not implemented (see section 3.6) making Electric unable to be used on a specific set of web applications (i.e. games).

Secondly, Electric could be improved by adding the ability to provide more ways to export an evidence set. The presented solution currently offers two ways to export an evidence set: as JSON (so it can be imported later on) or as a report (in a format that Dradis supports). Exporting as a video is a functionality that could extend these two and provide penetration testers with an easy and quick way to add interactive evidence to their presentation. Opening the tool to show a replay might not always be possible, for example when no internet is available. Thirdly, automatic request/response editing could be further researched. Currently, re-quest/response data is automatically modified with user-written scripts. Something that is easy to use but for simple changes (ex. setting one param to a predefined value) a different approach might be more useful. Simple changes could be implemented with a graphical user interface.

Finally, with more time improvements can be made to usability, support for other types of vulnerabilities and support upcoming technologies such as HTTP 2.0 and WebSocket communi-cation.

References

[1] Electron. Electron — build cross platform desktop apps with javascript, html, and css, . URL https://electronjs.org/. accessed 2018-05-28.

[2] Gregory Tassey. The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology, RTI Project, 7007(011), 2002.

[3] Institute Ponemon. 2017 cost of data breach study. URL https://www.ibm.com/security/data-breach. accessed 2018-06-05.

[4] O Sami Saydjari. Cyber defense: art to science. Communications of the ACM, 47(3):52–57, 2004.

[5] Sam Thielman. Yahoo hack: 1bn accounts compromised by biggest data breach in history. The Guardian, December, 2016. Accessed 2018-06-19.

[6] Rijksoverheid (Dutch government). Grapperhaus sluit cybersecurity alliantie met bedrijven om nederland digitaal veilig te maken (english: Grapperhaus signs cyber-security alliance with companies to make the netherlands digitally safe ). URL

https://www.rijksoverheid.nl/ministeries/ministerie-van-justitie-en- veiligheid/nieuws/2018/05/24/grapperhaus-sluit-cybersecurity-alliantie-met-bedrijven-om-nederland-digitaal-veilig-te-maken. accessed 2018-06-18.

[7] Mohd Ehmer Khan. Different forms of software testing techniques for finding errors. Inter-national journal of computer science issues, 7(3):11–16, 2010.

[8] Website security - keep in check with acunetix. URL https://www.acunetix.com. accessed 2018-05-28.

[9] N. Antunes and M. Vieira. Penetration testing for web services. Computer, 47(2):30–36, Feb. 2014. ISSN 0018-9162. doi: 10.1109/MC.2013.409. URL doi.ieeecomputersociety.org/10.1109/MC.2013.409.

[10] Yao-Wen Huang, Chung-Hung Tsai, Tsung-Po Lin, Shih-Kun Huang, D.T. Lee, and Sy-Yen Kuo. A testing framework for web application security assessment. Computer Networks, 48 (5):739 – 761, 2005. ISSN 1389-1286. doi: https://doi.org/10.1016/j.comnet.2005.01.003. URL http://www.sciencedirect.com/science/article/pii/S1389128605000101. Web Security.

[11] Christopher Neasbitt, Roberto Perdisci, Kang Li, and Terry Nelms. Clickminer: Towards forensic reconstruction of user-browser interactions from network traces. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, CCS ’14, pages 1244–1255, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2957-6. doi: 10.1145/2660267.2660268. URL http://doi.acm.org/10.1145/2660267.2660268. [12] James W Mickens, Jeremy Elson, and Jon Howell. Mugshot: Deterministic capture and

replay for javascript applications. In NSDI, volume 10, pages 159–174, 2010.

[13] Silviu Andrica and George Candea. Warr: A tool for high-fidelity web application record and replay. In Dependable Systems & Networks (DSN), 2011 IEEE/IFIP 41st International Conference on, pages 403–410. IEEE, 2011.