Extending Actionability in Better Code Hub

(1)

Extending Actionability in

Better Code Hub

Suggesting move module refactorings

Teodor Kurtev

teodor.ivanov92@gmail.com

July 14, 2017, 48 pages

Supervisor: dr. Ana-Maria Oprescu,a.m.oprescu@uva.nl

Host supervisor: dr. Magiel Bruntink,m.bruntink@sig.eu

Host organisation: Software Improvement Group, Inc.,https://www.sig.eu

Universiteit van Amsterdam

Faculteit der Natuurwetenschappen, Wiskunde en Informatica Master Software Engineering

(2)

Abstract

Undoubtedly, refactoring can have a positive effect on overall system quality, and it is a critical part of the work cycle of every developer. However, finding the right refactoring opportunities can be a difficult task. This is particularly the case for refactorings related to overall system structure. One of the best ways to address such issues is to apply the move module refactoring.

In this thesis, we propose a way of automatically detecting refactoring opportunities for move module refactorings in the context of the C# language using a Compiler as a Service (CaaS) - Roslyn. We evaluate our approach using seven open source projects and an expert panel. The results from these validation experiments showed our approach as promising - the group of experts found more than half of the proposed refactorings useful.

(3)

Acknowledgments

I would like to express my sincere gratitude to both my supervisors - Ana-Maria Oprescu and Magiel Bruntink, as their constant feedback and advice was a critical factor in the success of this project. Furthermore, I would like to thank everybody else at Software Improvement Group (SIG) for providing me with the opportunity to write my thesis there and the help they offered me in every way possible.

Finally, I would like to thank all my friends who helped me in the hardest of moments.

Teodor Kurtev

Amsterdam, The Netherlands July 2017

(6)

Acronyms

BCH Better Code Hub. 2,6–8,11,12,18–20,22,25,28,41

CaaS Compiler as a Service. 1,3,7,13,40, 42

TAR Technical Action Research. 18

uFDN Undirected feature dependency network. 2,8,12,24–26

(7)

Chapter 1

Introduction

Refactoring is an essential part of every software project life cycle and as stated by Robert Glass - ”Maintenance typically consumes 40 to 80 percent (average, 60 percent) of software costs.” [1]. Therefore, it is safe to assume that any help in automating refactoring would be a great asset for software development teams.

However, before we can delve into refactoring, we first need to know that there is a problem in our code. Code analysis techniques have certainly progressed far in recent years, and we could quickly build a tool that analyzes code and produces a list of problematic areas. However, a more interesting problem is how to propose fixes for these problems.

In this thesis, we introduce a general approach for detecting possible move module refactorings in the context of the C# language and evaluate its practical usefulness. We define a module as a group of declaration - class, struct, enum or an interface. Additional ideas on how to group modules(assign them to components) and how to present and order our produced refactorings are discussed.

To illustrate the problem in a practical environment, we chose a web tool developed by our host company - Better Code Hub (BCH)1. The core idea of the application is to grade a project maintain-ability on metrics described in the SIG Maintainmaintain-ability model [2] and providing a list of problematic areas in the code. However, BCH we want to extend the actionability of the tool by providing refac-torings for these problematic areas. To do that, we apply our proposed refactoring recommendation algorithm in the context of BCH and assess its effectiveness. We hope that by providing a list of actionable items, we will increase the value of BCH system for its users.

Component Independence is defined as having loose coupling between top-level components. Our aim is to improve the Component Independence in BCH by proposing move module refactorings. Our rationale for choosing this metric is that code smells related to system structure are, by their nature, harder to detect and refactor. We hope that our approach will help developers by offering them a list of possible refactorings they can apply.

1.1 Problem analysis

In a general context - automatic proposal of refactorings is not a new area. Significant research [3] has been done already, and at first glance, this particular problem should have excellent solutions and tooling. However, this is not the case, as we found almost no tools for the Java and .NET ecosystems(or for any of the other popular OOP languages). When we dug deeper, we found that most of the proposed approaches for automatic refactoring proposals are experimental and untested. This gives a lot of room for choosing an approach for our recommendation tool and conducting a validation study.

Before describing the method we chose, it is important to discuss the scope to which we will limit our research. The BCH tool assesses a piece of code on ten metrics - Volume, Duplication, Unit Size, Unit complexity, Unit interfacing, Module Coupling, Component Balance, Component Independence, Automatic Tests and Clean Code. Code smells related to the first five metrics(Volume, Duplication,

1

(8)

INTRODUCTION

Unit Size, Unit complexity, Unit interfacing), most of the time, have a clear refactoring solution and are in general easier to refactor. In contrast, code smells, related to the system structure metrics(Module Coupling, Component Balance, Component Independence), are harder to refactor and the refactoring solutions and opportunities are less obvious. For this reason, we will dedicate this thesis to trying to propose refactorings for an architecturally significant metric, and more specifically - Component Independence.

Albeit, all reviewed approaches lacked extensive testing, our choice was based on different criteria and namely - reproducibility of the proposed approach. Of course, we modify it to better suit our context. During our literature survey we found the following problematic points, which we will need to address:

• The approach in question is targeted at Java code, and even though it proposes a general algorithm, the semantic difference with other programming languages are disregarded. We try to explore those syntax differences in the context of the C# language (and .NET projects in general) and see if we can extract an even more general algorithm.

• The approach we choose is not extensively tested and would benefit from more validation against projects from different domains and sizes.

• We will need to modify the proposed approaches to suit the context of BCH and target it specifically at improving Component Independence.

• Our approach lack proper preconditions and postconditions to guarantee the correctness of the program, after the refactoring. We will add the need conditions.

The aim of this paper is to solve the problems mentioned above and can be summarized in the following points:

• Propose an approach for detecting reasonable refactorings.

• Propose an approach to ordering the identified refactoring opportunities. • Validate the approach using several open source projects.

• See if the suggested refactorings are practically valuable for developers.

• Observe the effect of these refactorings on the Component Independence and Component Bal-ance metrics in the SIG Maintainability model [4].

1.2 Research questions

This thesis aims to answer the following research questions:

• Research question 1 - Can we adapt existing approaches for move module refactorings to C# projects?

– What are the most notable syntax differences between the Java and C# languages, that can affect the approach mentioned above?

– Can we extract a more general algorithm that can be used for other languages?

• Research question 2 - How can we design a prototype tool that implements existing refactoring approaches for move module refactorings?

– What is the best way to modify this approach, so that it extends the actionability of a code quality feedback tool?

– How does the application of refactorings, influence the architecturally important code met-rics(Component Balance, Component Independence), described in the SIG Maintainability model [2]?

– What is the best way to map modules to components in the context of different program-ming languages?

(9)

INTRODUCTION

1.3 Solution outline

The approach we use for our refactoring recommendations tool looks at systems as a undirected dependency graph(uWCDN) [5][6] of modules and each module belongs to a community. The core idea of the refactoring recommendation is trying the move a module from one community to another and access the effectiveness of that particular move. We extended the mentioned approach, so it supports the syntactical specifics of the C# 7 programming language. Additional precondition/postconditions for the application of the algorithm will be added.

Moreover, modifications on the refactoring recommendations criteria will be made, so it will propose refactorings that have a positive effect on the Component Independence BCH metric. This will be done by studying the current scoring criteria in Better Code Hub[4] and translating them into a quality metric to be used by the algorithms. The produced refactoring recommendations will be ordered by using the ideas behind Pareto efficiency, to balance the trade-off of increasing Component Independence and decreasing Component Balance. This allows us the present the most appropriate refactorings to the user in an optimal order.

To validate our research, we apply our proposed approach on a selection of open source projects by executing the following steps:

• Gather code metrics using the Better Code Hub software. • Apply the move class/module refactorings.

• Gather code metrics for the project after the refactorings.

• Study the changes in SIG Maintainability Model metrics and extrapolate any relevant informa-tion.

Additionally, we will consult experts in .NET development to assess whether the proposed refac-torings have any practical value.

A detailed description of our tool design and implementation process can be found in Chapter 5. The setup for our validations experiments and the results from those tests can be seen in Chapter6.

1.4 Definitions

We use the following definitions throughout this thesis. A system is a collection of components.

A component can be viewed as a sub-part of the system, and it contains modules, grouped by a common trait - functional, technological or some other. In the context of C#, these will be namespaces. It is up to us to decide what namespace depth to use to define components.

A module is defined as a group of declarations. Sometimes this refers to a single file. In the context of C#, we define a module as a class, value class (struct), enum or an interface.

A unit is the smallest named code block of executable logic. In the context of C#, this can be a method, an extension method, a partial method, operator overload or a local function.

A Undirected feature dependency network (uFDN) is a network representing all the features of a system(methods, fields, properties, constructors). Each node of the network accounts for a single feature, and the connection between the nodes represent a usage dependency. For example, the method X uses the property Y - there is an edge between them.

A Undirected weighted class dependency network (uWCDN) is the core representation of our systems. A network of nodes(modules), separated into components. The edges between the nodes represent the connection between modules. The weight of the edges depends on how many connections exist between the two ends of the edge in the systems uFDN. All the features are grouped in communities which regarding OOP languages are modules.

1.5 Outline

In this section, we outline the structure and general idea of this thesis. In Chapter2 we discuss the background of our problems - refactorings, architecture significant code metrics, Better Code Hub,

(10)

INTRODUCTION

the Roslyn compiler and Pareto efficiency. In Chapter3 we demonstrate example scenarios for our refactoring approach. The next chapter(Chapter 4) is dedicated to explaining our research method. In Chapter5 we will provide implementation details of move module refactorings tool. In Chapter

6 we describe the setup we used to prove the correctness of the modified algorithms and the results from our validation experiments. After we have all the data from our experiments we reflect on it in Chapter7. We conclude the thesis in chapter9.

(11)

Chapter 2

Background

In this chapter, we present all the background needed for our research - refactoring, code smells, move module refactorings, C# and Java language differences, the Roslyn compiler, and Pareto efficiency.

2.1 Refactoring

In his infamous book [7], Martin Fowler defines refactoring as:

Refactoring is a controlled technique for improving the design of an existing code base.

To be able to execute the refactoring process correctly we need to follow a particular set of actions. As described in numerous papers, related to refactoring [8], The following generic steps can be defined:

1. Identify a code smell or in other words where the refactoring can be applied. 2. Choose an appropriate refactoring for the identified code smell.

3. Check if all the preconditions for the application of that refactoring are satisfied(The refactoring preserves the behavior of the program).

4. Apply the refactoring.

5. Check if all the behavior is preserved and all the postconditions are satisfied.

6. Assess if the refactoring has a significant effect on the quality characteristics of our software(It is pointless to apply refactorings that have no effects or degrade software quality).

By applying these prescribed steps, we can assure that our refactoring is done reasonably and preserves the behavior of the system.

We also did a literature study into current refactoring approaches [3] for different code smells to select the one that is the best for our context.

2.2 Code smells

The process of refactoring is a powerful tool. But the interesting problem is when to use that powerful tool. Martin Fowler and Kent Beck define the idea of a code smell in their book ”Refactoring: Improving the Design of Existing Code” [7]. By a code smell, we understand a particular set of symptoms about our code that indicated that a possible refactoring could be applied.

As our research is focused on improving Component Independence, it is interesting to see which code smells can affect it. To identify these code smells, we did a literature survey of several popular research papers [7][9][10]. As a result, the following code smells can have a potentially negative effect on the metric:

(12)

BACKGROUND

• Component coupling - when modules are situated in the wrong components, they have a lot of dependencies on external components. To reduce this, we can apply the Move module refactor-ing.

• The ”Common” pattern - a lot of developers tend to create components named Common, Shared, etc., which serve no design purpose besides a dumping place for modules that have no other place. This is again easily fixable with the Move module refactoring.

2.3 Move module refactoring

We define a move module refactoring, as the action of moving of module from one component to another. In the context of .NET a component can be a namespace or a project and in the context of Java is a package. Of course, custom components that do not use the namespace/project/package idea are also possible. In further chapters, we discuss the ideas between component mapping and the problem they present.

And again, for this refactoring to be executed successfully, a certain number of precondition and postconditions must be satisfied:

• The class to be moved, should not be moved to a component, where a class with the same name exists.

• Classes that are part of class hierarchies should not be moved.

• Partial classes should be ignored in the analysis, as they span multiple files and a single move module refactoring may not be enough to refactor them.

• Views, Forms and resource classes should be ignored, as most of the system containing them, relies on convention about their position, to function properly.

• When a class is moved all its references should be updated, to reflect the move operation. • Files from third party dependencies should be ignored as they are not the target of our

refac-toring.

The described preconditions and postconditions above will be used in the implementation of our refactoring tool.

2.4 Better Code Hub (BCH) and the SIG Maintainability

Model

Better Code Hub is a web tool that assesses a software systems maintainability based on the SIG Maintainability Model[2][4]. The following metrics are for the evaluation process:

• Volume - the size of the system measured in man-years. • Duplication - the duplicate lines of code in the system.

• Unit Size - the size of a unit(function, method, etc.) measured in LOC. • Unit complexity - the McCabe complexity [11] of a single unit.

• Unit interfacing - the number of parameter for a single unit. • Module Coupling - the number of incoming calls for a module. • Component Balance - the number and relative size of components.

• Component Independence - the amount of interface code. We define interface code as code in modules that are both called from and call modules of other components (throughput), and code in modules that are called from modules of other components (incoming).

(13)

BACKGROUND

• Automatic Tests - the test code percentage and assertion density.

• Clean Code - various code smells like unused code, commented out code, unnecessary comments, etc.

For each metric BCH provides a list of problematic areas that can be refactored.

2.5 Construction of the uWCDN and uFDN

A uFDN is dependency network build on a feature level. We can use it to detect connections between attributes of a system. For example, if the Method X uses a field Y, then there is a link between these two attributes. The nodes in the network are grouped in communities(modules). The construction of the uFDN is rather, simple, so we will not go into detail. A graphical representation of the uFDN is provided in Figure5.2.

Figure 2.1: Sample uFDN

As mentioned in our base paper [5], in a uWCDN each node represents a module, and each module is present in the network only once. The connections between the nodes represent certain dependencies between classes. These dependencies come from the attributes of the class - properties, methods, fields, constructors(we can use the systems uFDN to see the connections in the whole system). Additionally, each connection has a weight, which depicts the dependency strength between classes. So in short a uWCDN can be defined as:

uW CDN = (Vc, Ec, Wc) (2.1)

Vcis the set of all the nodes/modules in the network/system, Ecis a set of all edges between the nodes and Wc is a symmetric adjacency matrix where a non-zero entity represents a connection between corresponding nodes. For example if Wc[i][j] > 0 then there is a connection between node i and node j. We calculate the weights in the matrix using the following formula.

Wc(i, j) = | [ ni∈Fi Ri1 \ Fj| (2.2) Wc(j, i) = | [ nj∈Fj Rj1 \ Fi| (2.3)

The following abbreviations are used in equations2.2and2.3.

• Rikis a set of all reachable nodes from node i with a distance of k. • Fi is a set of all features module i contains.

Several example of a uWCDN are provided in Chapter3.

2.6 The Java and C# languages

The Java and C# languages are both C based, higher level, object oriented languages. As most research into refactorings is done using Java we use the specifications of both languages [12][13] to discover any differences that can affect our research.

(14)

BACKGROUND

2.7 The Roslyn compiler

The Roslyn compiler is an open source project1 from Microsoft, which introduces the idea of CaaS. The traditional design of a compiler is a black box - code goes in, compiled assemblies come out. The Roslyn project takes a new approach and exposes different APIs for the various parts of the compilation process(see Figure2.2).

Figure 2.2: The Roslyn pipeline

In this way, the compilation process is more ”transparent”, and we can easily access information like:

• Method, Field, Property, Constructor invocations. • Method, Field, Property, Constructor declarations.

• Attribute, Exception, Input parameters, Return Values usages. • Data and control flow.

• Complex semantic information(overload resolution).

Additionally, the compiler is heavily integrated into the new Visual Studio IDE (versions 2015+), which allows the easy development and integration of refactorings tools and services.

The introduction of the CaaS concept [14] allows for an easier creation of analysis and refactoring tools. Additionally, it opens new possibilities for meta-programming, code generation and transforma-tion, interactive use of the C# and VB languages, and embedding of C# and VB in domain specific languages.

2.8 Pareto efficiency

The idea of Pareto Efficiency stems for the economics domain, and it is defined as follows: An economy is in a Pareto Optimal state when no further changes in the economy can make one person better off without at the same time making another worse off. [15] A visual example of a Pareto optimal choices plot can be seen on Figure2.3.

(15)

BACKGROUND

Figure 2.3: Pareto Efficient Frontier

But the applications of this approach are not limited to its original domain. Pareto efficiency has been used when selecting optimal refactorings based on two or more criteria [16].

For example, we can easily redefine the original definition to fit the software engineering domain: A refactoring is in a Pareto Optimal state when no further changes in the system can make Component Independence better off without at the same time making Component Balance worse.

(16)

Chapter 3

Motivating Examples

In this section, we provide some classical examples of refactoring opportunities that we want to detect(or ignore) and correct with our tool. These examples are not exhaustive but are given only with the purpose of illustrating the general idea of our tool.

To detect move module refactoring opportunities, a large number of classes must be analyzed to determine if the module in question is indeed a refactoring candidate. Therefore, providing a code sample would be impractical. To ensure simplicity, we use a uWCDN representation to showcase our idea. In this way, the reader can get a more high-level overview of the sample system.

3.1 Improving refactorings

Figure 3.1: uWCDN original Figure 3.2: uWCDN refactored

The first case we look at is the straight forward, move and improve scenario. We can see on Figure

3.1that module E is being accessed only from ComponentA. It is logical to think about moving the module from ComponentB to ComponentA. Of course, we are not considering any context behind what ComponentA and ComponentB represent, but for the purpose of our research, we are only going to focus on reducing the external dependencies between components.

We can see, on Figure3.2, that after the move module refactoring the external dependencies between ComponentA and ComponentB have been reduced.

3.2 No change refactorings

Of course, there are cases where the move module refactoring will offer no benefit. For example, we have a simple system, illustrated by its uWCDN(Figure3.3).

We can try to move Module C from ComponentA to ComponentB or move Module E from Compo-nentB to ComponentA. However, as seen on the refactored uWCDN(Figures3.4and3.5), this leads to no benefits - the connect between ComponentA and CompontnB still exists and it’s of the same weight. Ergo, we do not consider such refactorings.

(17)

MOTIVATING EXAMPLES

Figure 3.3: uWCDN original

Figure 3.4: uWCDN refactored ver. 1 Figure 3.5: uWCDN refactored ver. 2

3.3 Non-beneficial refactorings

The worst case scenario for our refactorings is when a possible refactoring does not provide any beneficial effect on our system. As seen on Figure3.6we can try to reduce the dependencies between ComponentA and ComponentB by moving module C to ComponentB or module E to ComponentA.

Figure 3.6: uWCDN original

However, as seen on Figures3.7and3.8, this will not create in our system. It will have the opposite effect - the dependencies between ComponentA and ComponentB increase. It is logical that in such cases we ignore these refactorings.

3.4 Balance vs Independence - reflection

Another interesting aspect, from these examples, is which metrics from the SIG Maintainability Model will be affected after the move module refactoring. To begin with, Component Independence for

(18)

MOTIVATING EXAMPLES

Figure 3.7: uWCDN refactored ver. 1 Figure 3.8: uWCDN refactored ver. 2

CommunityA, CommunityB, and the overall system will change. However, there is a high possibility that at the same time the Component Balance metric may worsen, as we are putting more classes in one community and reducing the number of classes in the other. We have to find a solution to choosing the optimal refactoring that provides the best Component Independence and Component Balance. The answer to this problem will be discussed in further chapters.

(19)

Chapter 4

Prototype design research method

This chapter is dedicated to illustrating the research method that we use. After looking into the existing research methods, we decided to base our approach, partially, on Technical Action Research (TAR). As proposed in [17], the following definition for TAR is given:

Technical action research (TAR) is the use of an experimental artifact to help a client and to learn about its effects in practice. The artifact is experimental, which means that it is still under development and has not yet been transferred to the original problem context. A TAR study is a way to validate the artifact in the field.

In short, the idea behind our method is to first apply the concept on a smaller scale, in a controlled laboratory setting, and then gradually expand the experiment to a more realist context. The end goal is to produce an artifact that is usable in a real world setting.

Before an adequate technique can be produced, we need to define and solve any improvement problems that exist and again identify and answer any knowledge questions. A good example of an improvement problem is ”How can we detect move module refactorings, automatically?”. And knowledge problems could be ”What information do we need to identify a move class refactoring?” or ”How do we define a component?”.

To solve an improvement problem, we can develop treatments. A treatment can be defined as a solution that solves an improvement problem or reduces its severity. After the plausible treatment is developed it must be validated and, if it solves the improvement problem, implemented. In our particular context, the treatment is a tool that implements our idea to extend the actionability in BCH. A treatment can be validated by applying it in a particular context and comparing the actual results, expected results, benefits, and trade-offs, etc.

It is important for developers to use correct refactorings. As it currently stands BCH only provides the problematic areas of a system. Our goal is to develop a way to build upon that and provide solutions for those problems. This will give more power to the user and transform BCH from an observation system to a system that provides refactoring actions.

The generalization of the solutions is done by distinguishing between a particular problem and problem classes. For example, in our case, the particular issue is improving the actionability of BCH, but the general problem class is automatically detecting and proposing move module refactorings. This is important because extracting general knowledge and not just case-specific information is vital if we want to apply the ideas in a broader context.

4.1 Assumptions

At this point of our research, we have assumed that all of the Component Independence issues come from incorrectly placed classes in the wrong components. We must note here that in practice that is not the case and for a ”perfect” system refactoring the move module refactoring will not be enough - move method, extract class, and other refactorings should also be considered. Our tool can also implement detection of other types of refactorings, but due to our time limitations, we have focused our research only on the move module refactoring.

(20)

PROTOTYPE DESIGN RESEARCH METHOD

4.2 Problem analysis

We analyze our problem from two viewpoints. First, in the very specific SIG context and then in a more general, software engineering context.

4.2.1 SIG context

SIG has developed a web-based tool(BCH) that access the quality of a given software system based on their SIG Maintainability Model [2]. This is useful for the developer, but the current version of the system only shows problematic code areas and does not recommend any actions to be taken.

The goal of the tool that we develop is to recommend specific measures to be taken(refactorings) for a specific metrics - Component Independence. We chose that particular metric due to the fact finding refactoring opportunities for it is usually harder than, for example, Unit length or Unit Complexity.

We have identified an approach that can potentially, be adapted to fulfill the above requirement. We construct a proof of concept tool that incorporates that approach and access what value it adds to the BCH assessment tool. Table 4.1shows the stakeholder in the context of SIG and what their goals and criteria are.

Table 4.1: Stakeholders in the context of SIG

Stakeholder Goals Criteria

SIG Allow clients to take action when code smells for Component Independence are detected.

The tool should provide refactorings that only increase the maintainability rating. Moreover, developers should consider the provider refactorings - use-ful and sensible.

SIG’s clients The allows the clients to detect candi-dates for refactoring and suggest possi-ble refactorings quickly.

Easier to apply refactorings and improv-ing maintainability of the system.

We expect that our tool will add value to the users of BCH by allowing them to take action on the detected problems in their code quickly. This, in turn, should increase their performance and the quality of the systems they are working on. It is worth mentioning that the tool is usable even outside of BCH because the approach is universally applicable.

4.2.2 General context

The more general context of research is automatic detection of move module refactorings. We develop a tool that applies the ideas in existing research in the context of the .NET ecosystem and extract a more general algorithm from the research results.

By observing the phenomenon, we can conclude that there is indeed a need for a tool that recom-mends refactorings. The reason behind that

• Maintenance consumes on average 60 percent of software development, and with that is one of the most important phases of the software development life cycle [1].

• Refactoring is essential to the longevity of any system. However, applying the correct refactor-ings is hard and sometimes can lead to a negative effect on system quality [18].

We are optimistic that our tool is going to be useful in a variety of different context and scenarios. You can see a summary of all the tools stakeholders, in a general context, in Table4.2.

Of course, there is a particular risk to our research:

• The analyzed system depends on file position conventions. • The proposed refactorings disregard valuable context. • Usefulness is a subject of opinion and varies per system.

(21)

Table 4.2: Stakeholders in a general context

Stakeholder Goals Criteria

System maintainers Increase of system maintain-ability by making it easy to de-tect and refactor code smells.

The tool proposes sensible refactorings with value to the user. Not too much false positives are reported.

Researchers Utilizes refactoring data to

draw conclusions.

Accuracy of detected refactor-ing opportunities.

Third party service providers Provide a refactoring services to clients.

Scalability and reliability of the approach.

• The refactorings provide no usefulness. Not likely but, still possible.

• The analyzed system has extremely coupled components, and adequate refactorings can not be proposed.

4.3 Solution design

Using the suggested prototype tool, we can recommend the following treatments:

• Tool proposes move module refactorings to increase Component Independence → Tool ranks the refactorings based on their increase of Component Independence and decrease of Component Balance → The tool provides only the most highly scored refactorings to the user → the user of BCH applies the proposed refactorings → Component Independence is increased and the system maintainability increases.

This approach is reasonable because it allows the user to apply only the best refactorings and does not overload him with a hundred refactoring options.

• Tool proposes move module refactorings to increase Component Independence → The tool provides all of the refactorings to the user → the user of BCH applies the proposed refactorings → Component Independence is increased and the system maintainability increases.

This approach is reasonable but can quickly provide too much information to the user and reduces his willingness to apply the refactorings.

After the application of each of the proposed refactorings, the maintainability score is expected to increase or in the works case, remain the same.

The improvement goal is to extend the BCH web tool to increase its actionability, by providing users with a list of possible actions they can take(refactorings). We have chosen treatment one as described above as it provides more value to the customers.

The research questions will be answered in the context of SIG. Firstly, we collect all the refactoring opportunities information from our tool. After that, the refactorings are validated using the SIG Maintainability Model and external experts. Based on the data we hope to see if our approach is feasible.

4.4 Validation

The proof of concept tool will be used to analyze several open source systems and the results from it will be manually verified by several developers with real world experience. Additionally, effects on the SIG Maintainability Model will be studied. These directions will help us discern if the tool adds value to the SIG and particular - to the users of BCH.

The results are expected to be useful for a maintainability viewpoint, and the approach should apply to all OOP languages. Therefore, we expected these results to be valuable in other contexts and perspectives.

(22)

We analyze our results, evaluate our observation and give explanations for any unexpected results. Generalization, limitations, and opportunists are noted and discussed in Chapter7.

(23)

Chapter 5

A proof-of-concept refactoring tool

In this chapter, we explain the structure of our prototype tool. We start by defining the research context and giving a high-level component overview of our tool design, followed by a more in-depth explanation of each component and idea.

5.1 Research context

SIG has developed an easy to use, online analysis tool - Better Code Hub. BCH scores a given system using the SIG Maintainability model [2] and provides a list of problematic areas the user can address. The problem here is that BCH does not provide any possible actions to be taken after the problematic areas are pointed out.

For the purpose of this thesis, we develop a prototype tool that detects possible move module refactorings that are aimed at improving the Component Independence, of a given system. We are hoping that our proof-of-concept will allow BCH to adopt a similar approach for recommending refactorings.

5.2 Refactoring approach

This section will be dedicated to describing all the design and implementation decisions taken in the course of our prototype tool development.

Our main reason for developing this tool is to find refactorings that improve the Component Inde-pendence metric, in Better Code Hub. Before we showcase our approach lets first define Component Independence. As described in the ”Guidance for producers” [4] SIG document, Component Inde-pendence, in the context of SIG and BCH is measured as:

Based on the dependencies between modules residing in other components the module is classified as either hidden or interface. ... Therefore, Component Independence is calculated as the percentage of code that is contained in modules that are classified as hidden.

Given that, the move module refactoring seems a reasonable, treatment for the problem of improving Component Independence.

We used an existing literature survey [3] of various refactorings in object oriented code. After a thorough investigation of all the possible move module approaches [5][6][19][20], described in the mentioned paper, we selected the approach that is most easily modified to fit our research context [5]. Our rationale for choosing that particular method is reproducibility. Albeit all of the considered approaches had a solid idea, the one that was easiest to reproduce was the one we selected.

As a basis for our approach, we use an already existing idea [5] that threats the dependency network of a system as community network(uWCDN). By using this, we calculate the improvement of moving one class from one community to another(in the context of Better Code Hub, one community is a

(24)

A PROOF-OF-CONCEPT REFACTORING TOOL

component and in the context of .NET a community is a namespace/project/etc.). We give more details about the idea in the following sections.

5.2.1 High-level tool design

Conceptually, our prototype tool works in the following way:

• Extract all the code data from our code using the Roslyn compiler. By code data, we mean all the dependency sources (attributes declarations/invocations).

• We construct a uWCDN to represent a dependency network of the analyzed system. Each node is assigned to a different community(component).

• Execute a move module refactoring. If it improves the external dependencies of that module -consider it as a possible refactoring.

• Filter all possible refactorings by apply preconditions and postconditions. • Rank the final refactorings and present them to the user.

Figure 5.1 provides a description of all the steps involved. We discuss each step in the following sections.

Figure 5.1: High level tool design

5.2.2 Java syntax vs C# syntax

All of the proposed refactoring opportunities algorithms are designed to run on Java code. However, these approaches will not work directly for C#. There are several reasons for this. While Java was influenced more by Smalltalk, C# tried to stay closer to the C/C++ syntax. And although C# 1 was very similar to Java, that was 15 years ago. In its latest versions, C# started to add syntactical features that have no equivalent in the Java language. After comparing the specifications of Java 8 [12] and C# 7 [13] we found the following obvious syntax differences between the two languages, which can have an effect on our approach.

• Properties - possible sources of dependencies between modules and components.

• Partial classes - need to be considered when trying to propose move class refactorings as partial classes can span multiple files.

• Partial methods - a possible source of dependencies that can span multiple files.

• Tuples - the inner types of the tuple can pose as potential dependencies if they are part of the input parameters or the return value of a function/method/constructor.

• Value type classes (Structs) - they are a type of module that needs to be considered when proposing move module refactorings.

(25)

• Extension methods - a potential dependency source for modules. • Local functions - possible input and return value dependency sources.

The existence of these features, affect the approach we are using in various ways. For example: • We need to collect additional data for the invocation of properties.

• We need to consider partial classes when detection move class refactoring opportunities. • We should treat structs as modules in the context of our system.

We discuss in a further chapter, all of the modification that we made in regards to using the C# language.

5.2.3 Gathering code data

For the collection of the system related information, we are using the Roslyn compiler. It allows us to easily access to almost any information on a syntactical or semantical level. To gather all the dependencies between our classes we need information about the following items:

• Method declarations/invocations • Property declarations/invocations • Filed declarations/invocations • Constructor declarations/invocations

This information is easily obtainable using the syntax and semantic models provided by Roslyn. For the purpose of our research we ignore the following code constructs:

• Partial classes - they are tough to refactor due to them being in several places at once.

• Test classes - our approach is targeted only at source code systems and not at test suits. Although the approach could theoretically work, we are not exploring this possibility.

It is worthwhile to mention that we are currently missing a few sources of additional code depen-dencies. For example:

• Custom exception declaration and usages. • Custom attribute declaration and usages. • Input parameter dependencies.

• Return value dependencies.

This can result in false positives and is something we should address in further iterations of this tool/approach.

5.2.4 Construction of the uFDN and uWCDN

For the practical construction of the uWCDN, we made the following modification to the original approach to accommodate the differences in syntax and functionally, provided by the C# language. This allowed us to add some additional nodes to our uWCDN.

• Structs are treated as nodes in the community network, meaning they have the same role as classes.

(26)

After we have collected all the nodes, we need to calculate the weights of the edges between them. After a few experiments, we found out that the formula (5.1) proposed in the original paper is incorrect, and the symmetric relation does not hold in all cases (5.2). We propose a new formula(5.3) that fixes the problems in the old one.

Wc(i, j) = Wc(i, j) (5.1)

Wc(i, j) 6= Wc(i, j) (5.2)

ConWc(i, j) = ConWc(j, i) = max (Wc(i, j), Wc(i, j)) (5.3) Illustrating example We construct a simple system as an illustrating example. Figure5.2shows its uFDN and uWCDN. Consider the set of modules, Vc = {X, Y }. Since Y.e() dependents directly on X.c() and X.b(), there is an edge between module X and module Y. Intuitively, the weight of this edge should reflect how much module X and module Y depend on each other. To quantify this dependency, we first compute the dependency set for each method in each module:

RX.a()1= {X.c()} RX.b()1= {X.c(), Y.e()} RX.c()1= {X.a(), X.b(), Y.e())} RY.d()1= {Y.e(), Y.f ()} RY.e()1= {X.b(), X.c(), Y.d()} RY.f ()1= {Y.d()}

Next, we compute the weights based on the dependency sets and the equations2.2,2.3:

Wc(X, Y ) = |RY.d()1∪ RY.e()1∪ RY.f ()1∩ {X.a(), X.b(), X.c()} | = | {X.c(), X.b()} | = 2

Wc(Y, X) = |RX.a()1∪ RX.b()1∪ RX.c()1∩ {Y.d(), Y.e(), Y.f ()} | = | {Y.e()} | = 1 And finally, we apply our proposed fix(seen in equation5.3) to the original weight calculation.

ConWc(X, Y ) = ConWc(Y, X) = max (Wc(X, Y ), wc(Y, X)) = max (2, 1) = 2 As seen on our example figure the connection between Module X and Module Y is indeed 2.

5.2.5 Detecting improvements in external dependencies of modules

The original paper [5] used the Newman and Girvan modularity index [21] to determine if a potential refactoring has a positive effect on the overall system quality. To accommodate for the Component Independence metric used in BCH, we need to develop a metric to detect if a move module change is positive, negative or has no effect. After analyzing how BCH measures Component Independence, we developed a way to see if a refactoring increases or decreases the external dependencies of a component. Before we explain the metric we chose, we introduce the expressions that will be used in our formulas:

• P WExD - the sum of the weights of all external dependencies for a single component. • P WIntD - the sum of the weights of all external dependencies for a single component. • Ki - external dependency ratio before the refactoring for a single component in percentages. • Ki+1 - external dependency ratio after the refactoring for a single component in percentages. • ∆ExtDep - the change in the external dependencies for a single component in percentages.

(27)

Figure 5.2: Sample uFDN and uWCDN

The external dependencies change calculation is done in the following way. First, we calculate the external dependencies ratio before refactoring:

Ki=

P WExD P WExD+P WIntD

× 100 (5.4)

And after that the external dependencies ratio after the refactoring. Ki+1 =

P WExD P WExD+P WIntD

× 100 (5.5)

And in the end, we calculate our external dependency improvement metric. ∆ExtDep = Ki− Ki+1

Ki

(5.6) The interpretation of the ∆ExtDep is done as follows:

• If ∆ExtDep > 0 - the refactoring has improved the number of external dependencies for the component and it is possibly a sensible move module refactoring.

• If ∆ExtDep = 0 - the refactoring has neither improved nor worsen the number of external dependencies for the component. We do not consider it as a possible refactoring.

• If ∆ExtDep < 0 - the refactoring has worsen the number of external dependencies for the component. We do not consider it as a possible refactoring.

We use this metric when we choose possible refactorings in the next chapter.

5.2.6 Algorithm for detecting possible move module refactorings

We propose a Modified Refactoring Recommendation Algorithm(MRRA), that we apply to our uWCDN in order the detect possible move module refactoring candidates. In our algorithm the matrix Wc[][] stores the weights for each connection between each module in our uWCDN(Adjacency Weighted Matrix). If Wc[i][j] ≥ 1 that means that there is a connection between module i and module j with weight Wc[i][j]. If Wc[i][j] < 1 there is no connection. The nodeCommunites[] array stores all community(component) identifiers for all modules. For example, module i belongs to the commu-nity(component) with identifier nodeCommunites[i]. The pseudo code for this algorithm can be see in Algorithm1.

(28)

Data: The uWCDN

Result: A list of possible move module refactorings

Initialize Wc[][], nodeCommunites[] (modules in the same component will be assigned the same community), and i = 0, j = 0;

for i = 1 to |Nc| do for j = 1 to |Nc| do

if Wc[i][j] ≥ 1 && nodeCommunites[i] 6= nodeCommunites[j] then

Suppose move module i to community nodeCommunites[j] and calculate ∆ExtDep according to (5.6) and store it into an array ∆ExtDep[];

end end

Select the maximum ∆ExtDep, ∆ExtDepmax; if ∆ExtDepmax> 0 then

Recommend moving module i to community nodeCommunites[j] that produces the largest ∆ExtDep;

end end

return List of move module recommendations

Algorithm 1: Modified refactoring recommendation algorithm(MRRA)

5.2.7 Filtering recommendations

After we have a raw list of recommendations, we need to filter them by applying preconditions and postconditions, discussed in section2.3. For the filtering process we use the following rules:

• The class to be moved, should not be moved to a component, where a class with the same name exists.

• Classes that are part of class hierarchies should not be moved.

• View, Form and resource classes should be ignored as they are most commonly placed in specific folders due to a convention.

• Recommendations for files from third party dependencies should be ignored. After we apply our conditions, we get the final set of our recommendations.

5.2.8 Ranking recommendations

As mention in previous sections, recommending a move module refactoring can come at a cost - the reduction of Component Balance due to moving classes around. To counter this, we use the ideas of Pareto efficiency. It is worthwhile to mention that Component Balance is not the only possible cost of our refactorings, but for the purpose of our research, it will be our only focus.

The first part of our ranking process is simple - order the refactorings, in descending order, by the percent of improvement they offer. That means that the refactorings with the highest improvement will be provided first.

For the second phase, we need to decide who to order the suggested refactorings in order of less likely to reduce the Component Balance metric. Ideally, we would calculate the actual increase or decrease in Component Balance, for each move module refactoring. This is impossible due to some constraints of our host company. In order compensate for this refactoring, we use an alternative metric - a normalized LOC. The rationale for this is, that moving small classes, has less probability of affecting Component Balance, in contrast to large classes. Figure5.3shows the final ordering process - the initial and final refactorings state and the steps they go through.

Additionally, we construct the Pareto optimum of our refactorings based on two metrics, External dependency improvement and normalized LOC:

(29)

Figure 5.3: Ordering final refactorings

• Relative module size(RMS) - we base this metric on the Lines of code metric. We assume that smaller module have a smaller changes to affect the system Component Balance:

RM S = 1 − M odule LOC f or suggested ref actoring

max(M odule LOC f or all ref actorings f or the system) (5.7) This approach is experimental, and we discuss it in Chapter7of our thesis.

5.3 Components mapping and their role in the context of

Bet-ter Code Hub

One of the first challenges we faced when we started to develop our refactoring approach was to define what is a component. As mentioned before in Section1.4a component is officially defined as a sub-part of the system, and it contains modules, grouped by a common trait - functional, technological or some other. However is important do define, which modules should be included in which component and how do we map a module to a component. By manually exploring the class grouping mechanisms provide by most object-oriented languages we determined that there are several ways to do that:

• By using the file system - each component is a folder in the system source code. The depth of the folder mapping can be controlled.

• By using the class organization mechanisms provided by the language - for example, we use namespaces in C#, modules in JavaScript or packages in Java. Again the depth of the names-pace/module/package mapping can be controlled.

• By using a subsystem organizational mechanism provided by the language - for example, the .NET ecosystem uses projects to organize a solution into different parts.

• Custom mapping - you manually specify each class that is part of a component.

After defining the ways to map modules to classes, we need to have a way to choose the correct one. Like most other questions in software engineering, the answer to this particular issue is - it depends. There is no perfect formula for selecting the best way to do this type of mapping. The developer in charge of the refactoring should choose the most appropriate mapping for the particular system. For example using a project to component mapping in a .NET solution that only has one project makes no sense. It is better to use a namespace to component or file system to component mapping.

So what does this mean for the BCH web tool? For starters BCH only provides a file system to component and custom mapping options. Of course, the custom mapping options is enough to imitate any other mapping option, but it requires the user to do a lot of manual mapping work. Our recommendation here is to highly consider the implementation of an easy to use, out of the box solutions for the other two options.

For the purpose of this master thesis and the evaluation of our approach, we wrote an automatic tool to generate configuration files for the project to component and namespace to component mapping. This allows us the circumvent the current limitation of the BCH tool. We will not look into our configuration generation tool, because of its simplicity and almost no standalone research value.

(30)

Chapter 6

Evaluation

In this chapter, we describe the validation process for our idea. We perform several experiments which aim to prove that our approach has a positive effect on the SIG Maintainability Model. Additionally, we use an expert panel to show the value of our proposed refactorings and illustrate how to order refactorings based on Pareto Efficiency.

6.1 Validation setup

In this section, we describe the experimental setup we are using. We first introduce all of our tar-get systems and then provide more information about the SIG Maintainability Model, expert panel evaluations, and Pareto front construction.

6.1.1 Targeted systems

We conduct all our experiments on several open source systems, hosted on GitHub. • AutoMapper1

• Humanizer2

• ExperimentSample3 • MoreDotNet4

• Open Judge System(OJS)5 • Opserver6

• ShadowSocks Windows7

We choose these specific systems because we aim at small and medium sized systems from different domains(libraries, web applications, desktop applications), to see how our approach performs under various conditions.

More information about the type, number of modules and number of components for each system can be found in Table6.1. To see how our systems compare to each other based on their community size, we constructed a treemap (Figure6.1). In this way, the reader can visualize the variety of system we selected for our experiment.

1_{https://github.com/AutoMapper/AutoMapper} 2_{https://github.com/Humanizr/Humanizer} 3_{https://github.com/Teodor92/ExperimentSamples} 4 https://github.com/Teodor92/MoreDotNet 5_{https://github.com/SoftUni-Internal/OpenJudgeSystem} 6_{https://github.com/opserver/Opserver} 7_{https://github.com/shadowsocks/shadowsocks-windows}

(31)

EVALUATION

Table 6.1: Type and size of target systems

System Type Modules Count Communities Count LOC

AutoMapper library 179 7 4580

ExperimentSamples test application 10 3 48

Humanizer library 158 6 4097

MoreDotNet library 29 4 1932

OJS web application 428 25 13434

Opserver web application 584 8 10540

Shadowsocks desktop application 114 8 6452

Figure 6.1: System comparison based on community size

6.1.2 Gathering dependency empirical data

Before we start with the validation experiments, we gather empirical data about the dependencies of our projects.

The rationale behind this is to analyze which types of dependencies are most common and to validate if our algorithm takes into account enough code information.

The following dependency data is collected: • Property dependencies.

• Field dependencies. • Method dependencies. • Constructor dependencies. • Input parameters dependencies. • Return value dependencies. • Custom attribute dependencies. • Custom exceptions dependencies.

(32)

EVALUATION

6.1.3 Evaluation using the SIG Maintainability Model

To illustrate that our approach has a positive effect on the SIG Maintainability Model, we conduct the following validation experiment:

• Measure the code metrics for Component Independence and Component Balance, before the refactoring. This is done using an internal SIG tool that produces precise metrics for the SIG Maintainability Model.

• Apply the move module refactorings.

• Measure the code metrics, after the refactorings.

• Analyze the changes in the code metrics and see if the refactorings were beneficial. If they were not - discuss what was the reason behind that.

We execute the above steps on all of our targeted system described in section6.1.1.

6.1.4 Evaluation using an expert panel

We want to prove that our tool provides refactorings that have practical value to the developer community. So in our opinion, the best way to access this is to consult experienced developers and see if the refactorings indeed have value.

Our interview setup was done in the following way.

• Each developer gets a non-altered copy of the system that must undergo refactoring. • Each developer gets a list of all the recommended refactorings.

• Each developer can decide if a refactoring is valid, it is not valid or if it is partially valid(needs more than a move module refactoring).

• Additionally, the developer can provide feedback on each refactoring if necessary.

There is no time limit for the participant, and the can take as long as they like to analyze the test system and the proposed refactorings. After they are done, we aggregate all the received data and see what the practical usefulness of our approach is.

6.1.5 Ordering refactoring using Pareto efficiency

As mentioned in previous chapters we use the ideas behind Pareto efficiency to chose optimal refac-torings. We define optimal refactorings as refactorings that have the biggest external dependency improvement and the lowest chance to affect Component Balance.

We use two metrics to construct our Pareto fronts - External dependency improvement and Relative module size(RMS).

Using these metrics, we get values in the range of 0 to 1 which we use to construct our Pareto fronts. For both metrics, a bigger value is better.

6.2 Results

In this section, we present all the empirical data gathered and the results of our validation experiments.

6.2.1 Initial refactoring data

Before we show our results lets first look at the initial data for our evaluation systems. As discussed in the previous chapter we chose only medium and small sized system to make it easy for us to evaluate the correctness of the proposed refactorings. More information about each test system can be found in section6.1.1.

(33)

EVALUATION

We rank our refactorings based on the improvement they introduce in our system and the chances of them affecting Component Balance. We can see the number of refactorings per system in Table6.2. From experience we have with our proof-of-concept refactoring tool, we observed that refactorings that provide a meager percentage of improvement tend to be more hard to implement and more likely to be incorrect or hard to apply. For this reason, we disregard the refactorings with the lowest percent of external dependency improvement.

Table 6.2: Refactorings count for each experiment system Name Total refactorings Top refactorings

AutoMapper 87 23 Humanizer 25 17 MoreDotNet 6 6 OJS 157 34 Opserver 100 20 Shadowsocks 33 17

6.2.2 Empirical data regarding dependencies

We extracted the number of all types of dependencies in our system. A numerical representation of our results can be found in Table6.3. It is worthwhile to mention that we exclude the following dependencies from the results:

• Dependencies from system libraries - these are library part of the System namespace.

• Dependencies from third party libraries - for example libraries like JSON.NET or EntityFrame-work.

An interesting observation is to see the dependency type percentage for each system. We visualize this using a stacked bar chart found in Figure6.2.

As seen from our data, the most common dependencies in our systems are the property, filed and method dependency. And the least common ones are the custom exception, custom attribute and return value dependencies. Additionally, the top three dependencies make up around 90.9% of all dependencies in our systems. In contrast, the bottom three dependencies make up around 2.1% of all dependencies.

Table 6.3: Dependencies count, for each project by type

Name Func. Field Prop. Constr. Attr. In-Param Ret.v. Except.

AutoMapper 1036 641 2131 237 0 692 188 9 ExperimentSamples 11 0 16 11 0 2 0 0 Humanizer 767 762 179 108 2 154 67 1 MoreDotNet 38 27 0 0 0 25 0 0 OJS 1559 2325 10487 420 160 394 121 3 Opserver 1685 2322 5351 230 179 271 91 12 Shadowsocks 882 5856 429 120 0 102 16 13

6.2.3 SIG maintainability model evaluation

The results from our experiments with the SIG Maintainability Model can be found in Table6.4. We present only the changes in the maintainability scores as we believe this to be the most significant result of this experiment. The two metrics that we observe for changes are Component Independence and Component Balance. This is to be expected as our approach should affect only those two metrics. As seen in Table 6.4, the improvement in Component Independence ranges from 0.1 to 0.7 stars. Additionally, in some cases, we see no improvement(MoreDotNet) or even worsening of the start rating.

(34)

EVALUATION

(35)

EVALUATION

For the Component Balance, we see changes ranging from 0.1 star growth, through no improvement at all and even a negative impact on the metric(ExperimentSystem).

Table 6.4: Star rating changes in the SIG Maintainability model after refactoring

System CB change CI change

AutoMapper 0 +0.2

Experiment Sample -0.4 +0.7

Humanizer 0 +0.1

MoreDotNet 0 0

Open Judge System +0.1 +0.1

Opserver 0 +0.1

ShadowSocks-windows +0.1 -0.2

6.2.4 Evaluation using an expert panel

The overall performance from the evaluation using an expert panel was quite diverse. We aggregated the data from all participants in two ways:

• By using simple averages and presenting the results as percentages. The results from that aggregation can be seen in Table 6.5.

• By using the majority rule. For example, if three out of the five experts say that a refactoring in not good, we consider that refactoring to have a no good score. The results from this aggregation can be seen in Table6.6.

Table 6.5: Expert panel results. Geometric mean aggregation.

System Valid Partially valid Invalid Useful %

AutoMapper 47.01% 14.17% 36.49% 61.18%

Sample Experiment System 66.67% 26.63% 6.66% 93.33%

Humanizer 29.17% 41.81% 27.90% 70.98%

MoreDotNet 0 13.33% 86.43% 13.33%

Open Judge System 38.24% 13.17% 48.14% 51.40%

Opserver 20.00% 27.87% 50.36% 47.87%

ShadowSocks 31.00% 34.86% 33.61% 65.86%

Table 6.6: Expert panel results. Majority rule aggregation.

System Valid Partially valid Invalid Useful %

AutoMapper 47.83% 21.74 % 30.43% 69.57%

Sample Experiment System 66.67% 33.33% 0 100.00%

Humanizer 29.41% 41.18% 29.41% 70.59%

MoreDotNet 0 16.67% 83.33% 16.67%

Open Judge System 38.23% 14.71% 47.06% 52.94%

Opserver 20.00% 40.00% 40.00% 60.00%

ShadowSocks 31.25% 31.25% 37.50% 62.50%

We see that the system with the lowest Usefulness score is MoreDotNet, and the systems with the highest one are the SampleExperiment system and the Humanizer library. This is true for both the averages aggregation and the majority rule aggregation.

(36)

EVALUATION

6.2.5 Ordering refactoring using Pareto efficiency

We plotted all our refactorings data from all of our projects to highlight the Pareto fronts. The fronts can be seen on Figure6.3.

The refactorings on the Pareto front range from 1 to 9, which is not all of our possible refactorings. It is important to mention that we do not drop the refactorings that are below the Pareto front. The main idea here is to order the refactorings optimally, and so the refactoring below the front will just have a lower ranking.

(37)

EVALUATION

(38)

Chapter 7

Discussion

In this chapter, we discuss the results of our evaluation, reflect on the usefulness of our approach, point out several flaws and present any threats to the validity of our experiments.

7.1 Discussion per test system

In this section, we discuss the results for each of our tests systems and give a rationale for any anomalies.

AutoMapper

The AutoMapper project is a popular .NET library for mapping POCO classes to one another. The system does not rely on conventions.

The recommended refactorings produced from our tool have a positive effect on the SIG Maintain-ability Model, as we see a +0.2 increase in the Component Independence metrics.

As seen in the expert panel results the useful refactorings percentage is relatively high - between 60 and 70%, which in comparison to the other results is good. This indeed shows that our initial speculation was correct and our refactoring approach is most useful for systems that do not rely on conventions.

Sample experiment system

Although the Sample experiments system was used only for a proof of concept, it is interesting to see if the developers see the recommended refactorings as valuable. This code base was developed with no context in mind.

From the perspective of the SIG Maintainability Model, our refactorings produce a 0.7 stars increase in the Component Independence metric, but the trade-off is a 0.4 decrease in the Component Balance metric. The reason behind this is the removal of one of the system components after our refactorings are applied. This may affect the Component Balance negatively, but the refactoring is logical and should be implemented.

The expert panel results also seem promising. On average 2 out of all the three suggested refactor-ings were considered valid, and the remaining one was either consider partially valid or not valid. The partially valid/non-valid refactoring is due to it becoming non-needed after the other two refactoring were applied.

Humanizer

Humanizer is an open source library for converting .NET types into human readable text. The library does not rely on any conventions.

The validation using the SIG Maintainability Model yelled positive results as our refactorings increased by the Component Independence with 0.1 stars. The Component Balance was not affected.

Extending Actionability in Better Code Hub