CoCoNut: A New Dynamic Approach to Code Generation

(1)

Bachelor Informatica

CoCoNut: A New Dynamic

Approach to Code Generation

Rick Teuthof

June 15, 2020

tica

—

Universiteit

v

an

Ams

terd

am

(2)

(3)

Abstract

Compilers are important to provide machine code readable to an arbitrary architecture. Creating compilers by hand can be a tedious task because of having to duplicate code of common code patterns. For this purpose, CoCoNut has been created. CoCoNut is a meta-compiler framework built to provide an easier experience for building actual meta-compilers. It provides the automatic generation of compiler passes and an intermediate representation according to a user specified domain specific language. According to this domain specific language an abstract syntax tree is generated. We research the implementation of a dynam-ically typed alternative to the already existing statdynam-ically typed framework. We discuss the implementation details of both backends and discuss their differences. We research the dif-ferences between the two as well as the implementation details the dynamic backend allows to provide a more streamlined experience for the user. To validate the dynamic approach, we perform a qualitative and quantitative analysis on the dynamic backend and compare this to the static backend. From this analysis, we can conclude that the performance between the two backends is similar, but for larger scale trees, the dynamic backend is more efficient.

(4)

(5)

Introduction

Code is used in almost all technology nowadays. A lot of this code is compiled, rather than interpreted. Therefore, this code needs a suitable compiler in order to run the code on a particular machine. For these compilers (henceforth referred to as ‘target compilers’ to prevent confusion), it is essential that they are easy to design and error-free. Something that aids the development of target compilers are so-called meta-compilers. These are compilers that generate code that aids the construction of target compilers.

For this project, we explore a compiler framework called CoCoNut (Compiler Construction in a Nutshell [1]–[3]. This framework generates code that aids the development of target compilers. The user can specify how this compiler is designed using a Domain Specific Language (or DSL). The syntax of the DSL is specifically designed for the CoCoNut meta-compiler. The code gen-erated by the meta-compiler provides files and functions that are used for the resulting target compiler. This target compiler in turn generates an Abstract Syntax Tree (or AST). This AST is used as an intermediate representation that contains all the nodes of the to be compiled code. Currently, the CoCoNut framework uses a statically typed approach to the generated code of the given AST specified by the DSL. This means that for every node type in the specified DSL, a data structure is generated, with dedicated functions for replacement, traversals, etc.

The goal of this project is to devise a different solution to the current statically typed imple-mentation. Essentially, this means trying to come up with a more dynamic approach. Thus we explore the necessary extensions and modifications to the CoCoNut framework in order to imple-ment this new dynamic approach. For this dynamic approach, we preserve the DSL designed for CoCoNut, but, the AST generated by the target compiler consists of dynamically typed nodes. Using a dynamic implementation of traversals, we discuss how to traverse the generated AST These traversals describe the transformations and analysis of the AST. Besides traversals there are passes, which function like traversals on a node, but in an isolated way. These traversals and passes are handled by a phase driver, which determines at what points certain traversals and passes should be executed.

In this thesis we first discuss the general compiler design and implementation. Next we describe the current state of CoCoNut and look at its overarching structure. Then, we describe our own dynamic implementation of the data structures, the traversal functionality and its phase driver. Keeping the implementation details in mind, we discuss the thought processes behind

(8)

(9)

(10)

CHAPTER 2

Background

2.1 Compiler Design

In this section we will briefly discuss the general design of compilers. When compiling source code, there are usually a certain few phases. The first of these common phases is parsing and tokenisation, also known as lexical analysis. The source file is read line by line and each line will be turned into tokens and added to an intermediate representation, which is usually an AST. Each of these tokens con-tains the data necessary for the compiler to understand what kind of token it is. Some types of tokens are keywords, character strings, variables and constants. CoCoNut uses the open-source lexer gen-erator Flex [4] to execute this task.

The next step is actually parsing the tokens, also known as syntactic analysis. Each of the tokens is analysed and the validity of the source code is determined. Compilers like GCC [5] and Clang [6] throw errors at the exact location the error was found in case of invalid syntax. As with lexer generators, syntactic analysis can be automated by usage of a parser generator. CoCoNut makes use of the open source parser generator GNU Bison [7].

When done parsing, an AST is generated as an intermediate rep-resentation of the source code. This intermediate reprep-resentation can now be traversed through using compiler passes. A pass that traverses through the entire IR is called a traversal. A common traversal is semantic analysis, which scans the tree and checks the validity of their identifiers and types. Another is creating a sym-bol table to store data of various entities such as objects, classes, variables, functions, etc. Besides these traversals, optimisations can be performed on the intermediate representation to produce higher quality compiled code.

The final step of the compilation process is code generation, in which the intermediate representation is converted into code that can be

interpreted by the target (virtual) machine. Figure 2.1: Common

com-piler pipeline, where yellow denotes the code in the

(11)

in-2.2

CoCoNut

In this Section we will discuss the current state of the CoCoNut project. We will show some of its key implementation details and allude to improvements to certain drawbacks this typed implementation carries.

The CoCoNut metacompiler consists of two layers. The first of these layers is the frontend, which creates all the entities according to the DSL specification. These entities consist of the following • Phases: The stages of the compiler that can contain, traversals or passes. These are also called actions. Each of the actions is called in the phase driver, which controls the entire flow of the target compiler. The phase driver is generic for any backend and is thus shared with the new dynamically typed backend. It determines which actions to call at a specific point in the compiler.

• Passes: Transformations applied to (sub-)trees in isolation. The user specifies a root node type and the pass will be applied on each of the nodes of that type without traversing into the next node of its kind.

• Traversals: These provide the user to specify how the data in the tree should be analysed or transformed. For each traversal, nodes are divided into three categories:

– Traversal nodes, which are handled by the user.

– Pass-through nodes, which are necessary to traverse in order to get to the traversal nodes and are handled by the meta-compiler.

– Nodes that fall into neither of the former two categories, which are simply be ignored during the traversal.

It is determined which type each node is for a given traversal by using a reachability matrix. This is a matrix of booleans that signifies if a child node is reachable given a specific traversal and a specific node.

• Enums: These function as user defined types. For example, the type of a binary operation can be defined using an enum.

• Nodes: These entities form the AST that the target compiler traverses through. They contain the data that can be analysed or transformed. Nodes Contain pointers to children nodes or attributes of any primitive type or a link to some other node.

• Nodesets: These consist of multiple possible nodes, but during runtime it is only one type of node at a time. for example, an expression node can either be a constant, a variable or a binary operation, but during runtime, it is only one of these types. When a child node is expected to be in a certain nodeset, it can freely be replaced with a node of another type within that nodeset.

All of these entities form the intermediate representation that will be used by the backend to generate the compiler framework code.

The second layer is the backend, which actually generates the code for the target compiler. Be-sides the aforementioned core components, The CoCoNut backend has a few other components. One of these components is a consistency checker. This provides warnings and error messages if the AST somehow became corrupt or invalid during a traversal. The AST is initialised and cleaned before and after the traversal process respectively. It does this with generated construc-tors and destrucconstruc-tors for the AST. It also has a dedicated copy traversal, for copying (sub-)trees.

(12)

Figure 2.2: Abstract overview of the several stages in the CoCoNut framework, where blue denotes user defined files, green are generated files, white are programs, yellow are linked libraries and red is the final executable. The red outline shows the files generated by the work from this thesis. (Source: Damian Fr¨olich [3])

As the current coconut framework is statically typed, each node specified by the user has its own dedicated data structure containing its children and attributes. Each of these structures are defined in their own file containing forward declarations to the structures of the children. These structures are defined in Figure 2.3.

1 typedef struct BinOp { 2 struct Expr *left; 3 struct Expr *right;

4 BinOpType op;

5 } BinOp;

Figure 2.3: Example of CoCoNut node data structure

(13)

1 struct BinOp *create_BinOp( 2 struct Expr *left, 3 struct Expr *right,

4 BinOpType op) {

5 struct BinOp *res = mem_alloc(sizeof(struct BinOp)); 6 res->left = left;

7 res->right = right; 8 res->op = op; 9 return res; 10 }

Figure 2.4: Example of CoCoNut node data structure constructor

The way this is implemented has a few benefits. First, the structure is simple to work with, as it only contains pointers to other node structures. Second, as each structure is defined in their own file, changes made to the AST will only affect the files containing the changed structures. This is implemented using a hashing mechanism.

When traversing through the children of a node, the current traversal is determined with the function ‘trav current’, which simply returns the type of the current traversal. Based on this type, the next function to be called is determined. In Figure 2.5 we can see an example of this.

1 BinOp *_trav_BinOp(struct BinOp *node, struct Info *info) {

2 if (!node) return node;

3 switch (trav_current()) { 4 case TRAV_Print: 5 Print_BinOp(node, info); 6 break; 7 default: 8 trav_BinOp_left(node, info); 9 trav_BinOp_right(node, info); 10 break; 11 } 12 }

Figure 2.5: Example of traversing a node.

We can see that for the ‘Print’ traversal, the function ‘Print Binop’ is called. This function should be provided by the user and contains the functionality for the ‘Print’ traversal.

When traversing through the children of a node, a function comparable to the one shown in 2.6 is called.

(14)

1 void trav_BinOp_left(struct BinOp *node, struct Info *info) {

2 if (!node)

3 return;

4 void *orig_node_replacement = node_replacement;

5 node_replacement = NULL; 6 if (!node->left) 7 return node; 8 switch (node->left->type) { 9 case NS_Expr_Var: 10 _trav_Var(node->left->value.val_Var, info); 11 break; 12 case NS_Expr_BinOp: 13 _trav_BinOp(node->left->value.val_BinOp, info); 14 break; 15 case NS_Expr_Num: 16 _trav_Num(node->left->value.val_Num, info); 17 break; 18 case NS_Expr_UnOp: 19 _trav_UnOp(node->left->value.val_UnOp, info); 20 break; 21 } 22 23 if (node_replacement != NULL) { 24 switch (node_replacement_type) { 25 case NT_Var: 26 node->left->value.val_Var = node_replacement; 27 node->left->type = NS_Expr_Var; 28 break; 29 case NT_BinOp: 30 node->left->value.val_BinOp = node_replacement; 31 node->left->type = NS_Expr_BinOp; 32 break; 33 case NT_Num: 34 node->left->value.val_Num = node_replacement; 35 node->left->type = NS_Expr_Num; 36 break; 37 case NT_UnOp: 38 node->left->value.val_UnOp = node_replacement; 39 node->left->type = NS_Expr_UnOp; 40 break; 41 default: 42 print_user_error("traversal-driver",

43 "Replacement node for BinOp->left is not a

node type of nodeset Expr."); ,→ 44 break; 45 } 46 } 47 node_replacement = orig_node_replacement; 48 }

(15)

We can see two core aspects of this function. The first is that when passing through a nodeset (in this case ‘Expr’,) the type of the node will be compared to each of the nodes in the nodeset and based on that the correct traversal function will be called. This is something the dynamic backend will not have to deal with at all, since it can directly call functions based on its child node type. This is demonstrated in Figure 3.15.

Secondly, it is necessary for the typed backend to provide a mechanism that allows for nodes to be replaced with nodes of another type. This is a drawback of the typed backend and it is something we will try to improve in the dynamic backend.

In general, the traversal mechanism in the statically typed backend generates a lot of code because of the inherent type checking by usage of switches. This is something we aim to improve in the dynamic backend, as demonstrated in Figure 3.15.

As seen in the work of Fr¨olich [3], the coconut framework makes use of a phase driver to handle the calls to actions. Actions consist of phases, passes and traversals. This traversal driver creates start functions that start each of these kinds of actions. The phase driver is generic and can be used for any backend. This includes the dynamic backend we developed.

2.3 SAC (Single Assignment C Framework)

For the compiler construction course at the University of Amsterdam, we used a compiler frame-work created by the SAC development team [8]. The goal of this project is to design a dynamic implementation of a compiler framework. The SAC framework is designed with this philosophy in mind. Therefore we took inspiration from this framework by looking at the best parts of it and modifying it to create a potentially better solution and improve on its implementation. This framework uses the data structure defined in Figure 2.7.

1 struct NODE {

2 nodetype nodetype;

3 struct SONUNION sons; 4 struct ATTRIBUNION attribs; 5 };

Figure 2.7: Node Data structure used in the SAC framework

The SAC framework uses a struct to store the children and attributes of each individual node. This is, however, very memory inefficient, as the size of each of these ‘UNION’ structs is now equal to the number of nodes times the size of the pointer to the struct it contains. This is why we redesigned this in the new dynamic implementation for CoCoNut.

(16)

1 struct SONUNION {

2 struct SONS_N_EXPRLIST *N_exprlist; 3 struct SONS_N_BINOP *N_binop; 4 struct SONS_N_UNOP *N_UNOP; 5 struct SONS_N_NUM *N_num; 6 struct SONS_N_ROOT *N_root; 7 struct SONS_N_VAR *N_var; 8 };

9

10 struct ATTRIBUNION {

11 struct ATTRIBS_N_EXPRLIST *N_exprlist; 12 struct ATTRIBS_N_BINOP *N_binop; 13 struct ATTRIBS_N_UNOP *N_UNOP; 14 struct ATTRIBS_N_NUM *N_num; 15 struct ATTRIBS_N_ROOT *N_root; 16 struct ATTRIBS_N_VAR *N_var; 17 };

Figure 2.8: Examples of the SONUNION and ATTRIBUNION Structs

With each of the underlying structs defined in Figure 2.9.

1 struct ATTRIBS_N_BINOP 2 { 3 binop Op; 4 }; 5 struct SONS_N_BINOP 6 { 7 node *Left; 8 node *Right; 9 };

Figure 2.9: Examples of the individual node data structs

(17)

1 node *TBmakeBinop(binop Op, node *Left, node *Right) { 2 node *this;

3 this = MakeEmptyNode();

4 NODE_TYPE(this) = N_binop;

5 this->sons.N_binop = MEMmalloc(sizeof(struct SONS_N_BINOP)); 6 this->attribs.N_binop = MEMmalloc(sizeof(struct ATTRIBS_N_BINOP));

7 NODE_TYPE(this) = N_binop; 8 BINOP_LEFT(this) = Left; 9 BINOP_RIGHT(this) = Right; 10 BINOP_OP(this) = Op; 11 return this; 12 }

Figure 2.10: Example of a constructor in the SAC framework

The version of the traversal mechanism in the SAC compiler is comparable to the one designed in the dynamic backend. A simplified version is shown in Figure 2.11. We can see here that there’s an extra functionality that allows the user to call a specific function before and after a traversal through a node. This is, however, not a feature we have adopted in CoCoNut.

1 node *TRAVdo(node *arg_node, info *arg_info) { 2 if (pretable[travstack->traversal] != NULL) {

3 arg_node = pretable[travstack->traversal](arg_node, arg_info);

4 } 5 6 arg_node = travstack->funs[NODE_TYPE(arg_node)](arg_node, arg_info); ,→ 7 8 if (posttable[travstack->traversal] != NULL) {

9 arg_node = posttable[travstack->traversal](arg_node, arg_info);

10 }

11

12 return (arg_node); 13 }

(18)

(19)

CHAPTER 3

The dynamic backend

We tackled several problems during the construction of the dynamic back-end. In this chapter we go over the several problems and their solutions and discuss their up- and downsides.

3.1 Design Philosophies

We design the dynamic backend with a few key design philoso-phies in mind. The first is that it should be truly dynamic. There should only be a few core functions that call each other with specific parameters, instead of explicitly defined functions for each kind of node or traversal. This should result in less code being generated. Exceptions to this rule are core traversals, which are the copy, free and check traversals. These traversals copy (sub-)trees, free the allocated data structures and check for compatible node types respectively.

Besides being dynamic, the framework should be easy for the user to understand and use. This means that we provide pre-generated functions the user should write their own implementations in. This mitigates the problem of having to add new files and functions whenever they edit the DSL specification, which might be undesirable from a user experience standpoint. In addition to this, the dynamic backend will, in contrast with the static backend, generated less files. It only generates one file for each core functionality. This will be explained in greater detail in Section 3.2

Lastly, the framework should be error-free. This is of course a must for any application and therefore we provide dynamic consistency checks that aid the user in quickly discovering errors in their code. Memory management is also very important, as we should not allow any memory leaks in the target compiler framework.

(20)

il-3.2

File Structure

The dynamic backend has a different approach to the file structure in comparison to the static backend. Instead of generating dedicated files for each node- and traversal type, we instead dedicate a single file to a feature at a time. In Figure 3.2 we show the dependency structure of the core and generated include files.

generated/src/vtables.c ccn/vtables_core.h actions_core.h ccngen/actions.h ccn/ast_core.h ccn/trav_core.h ccngen/ast.h ccn/types.h

ccngen/enum.h stdbool.h stdint.h ccngen/trav.h

Figure 3.2: Include dependency graph

We will explain this figure from bottom to top, as that is the order of inheritance. At the bottom, we have all includes dedicated to types. These types include:

• Forward declarations to the ‘Node’ and ‘Trav’ data structures. • Types for booleans and specialised integers.

• Enum types, which are generated according to the DSL specification.

Next, we have the AST generation functionality and the traversal functionality. They handle the creation of the AST and its corresponding compiler traversals. They will be thoroughly discussed in Sections 3.3 and 3.4 respectively. These files are then in turn included by the files containing declarations for actions. These include all passes and traversals according to the DSL specification. Finally, these actions are then inherited by the virtual table files. Their function

(21)

3.3 Node Data Structure

The first problem was devising the data structure necessary to store the data of each node. For this we drew inspiration from the SAC compiler framework, as discussed in Section 2.3. The design of this function is shown in Figure 3.3.

1 typedef struct NODE {

2 NodeType nodetype;

3 union NODE_DATA data; 4 struct NODE **children;

5 long int num_children;

6 } Node;

Figure 3.3: Core node data structure

There are a few key points to take note of here. First, we can see that every node has a pointer to an array containing its children. This will be explained in Figure 3.5. Besides the pointer to the array, the amount of children a node has is also stored. This is because since we are working with a pointer to an array, we cannot know the size of the array, as it is dynamically allocated on the heap and the compiler has no way to know its size.

An example of the ‘NODE DATA’ union is shown in Figure 3.4.

1 union NODE_DATA {

2 struct NODE_DATA_ROOT *N_root;

3 struct NODE_DATA_EXPRLIST *N_exprlist; 4 struct NODE_DATA_UNOP *N_unop;

5 struct NODE_DATA_BINOP *N_binop; 6 struct NODE_DATA_VAR *N_var; 7 struct NODE_DATA_NUM *N_num; 8 };

Figure 3.4: Example of node data union

This implementation is a small improvement over the SAC implementation. The node struct makes use of a union of pointers to data structs. This means that the size of the data is now the size of only one time the size of the pointer to the struct.

(22)

1 struct NODE_DATA_BINOP {

2 union NODE_CHILDREN_BINOP {

3 struct NODE_CHILDREN_BINOP_STRUCT {

4 Node *left, *right;

5 } binop_children_s; 6 7 Node *binop_children_a[2]; 8 } binop_children; 9 10 BinOpType op; 11 };

Figure 3.5: Example of node data structure

At first glance, there’s only a few differences between the structure defined in 3.5 and the static implementation as shown in 2.3. Essentially, we applied an abstraction layer on top of the statically typed structure. In addition to this abstraction layer, we provided functionality for accessing children nodes in different ways. The first of these ways is accessing the children directly, with the usage of user access macros, which will be shown in Figure 3.6. The second is accessing the children nodes as an array, such that traversal of all children could be made generic. This is shown in the traversal function for node children in Figure 3.13. This implementation works, as a struct is a contiguous array of data allocated on the heap, with addresses aligned in the way they are defined, according to the ISO C99 standard [9]. As we are dealing with only data of pointers to nodes, we can simply access this struct as if it were an array. This is why we implemented it as a union of both the struct containing the data and a pointer to an array of these children. As this is a union, it does not take up any extra space in memory. In order to be able to dynamically access the children of a node, we had to give each node a pointer to its array of children, as accessing the children directly without this pointer would require an extra check for the type of the node, as each data struct is unique to each type of node. Along with this pointer we allocated an integer that stores the amount of children a node can have, such that we can perform loops over the amount of children.

A downside of this approach is that it might not be portable to any compiler, as not every compiler might handle structs the same way. However, we have decided to expect the user to use a compiler following the C99 standard.

We have combined the children and attributes into one data structure. As the user should not be accessing the data structures directly, there is no need for the children and attributes to be handled separately.

For conveniently accessing the data each node holds, we use macros to mitigate the need to access struct members directly. These macros are defined in Figure 3.6. These macros allow the user to access the data from a node without having to write a lot of code. They simply provide an abstraction layer for the user for convenience.

(23)

1 #define NODE_TYPE(n) ((n)->nodetype) 2 #define NODE_CHILDREN(n) ((n)->children)

3 #define NODE_NUMCHILDREN(n) ((n)->num_children) 4 5 #define BINOP_LEFT(n) ((n)->data.N_binop->binop_children.binop_children_s.left) ,→ 6 #define BINOP_RIGHT(n) ((n)->data.N_binop->binop_children.binop_children_s.right) ,→

7 #define BINOP_OP(n) ((n)->data.N_binop->op)

Figure 3.6: Data access macros, with examples for a specific node

The base constructors for nodes are defined in Figure 3.7.

1 // Constructor for empty node

2 Node *node_init() {

3 Node *node = (Node *)mem_alloc(sizeof(Node)); 4 return node;

5 }

Figure 3.7: Base node constructor

The individual constructors for each node follow the template in Figure 3.8. First a blank node is initialised, for which the data for its type will then be allocated and assigned. Next its type is assigned and it will get a pointer to its children, which can then be accessed in the children traversal function shown in Figure 3.13. The number of children of the node is determined in the generation process of the constructor code, as each node specified in the DSL gets its own array of children nodes, we can take the size of that array as the amount of children. Finally all of its children pointers and attributes will be assigned to its data values.

1 Node *node_init_binop(Node *left, Node *right, BinOpType op) { 2 Node *node = node_init();

3 node->data.N_binop = mem_alloc(sizeof(struct NODE_DATA_BINOP));

4 NODE_TYPE(node) = NT_binop; 5 NODE_CHILDREN(node) = node->data.N_binop->binop_children.binop_children_a; ,→ 6 NODE_NUMCHILDREN(node) = 2; 7 BINOP_LEFT(node) = left; 8 BINOP_RIGHT(node) = right; 9 BINOP_OP(node) = op;

(24)

One improvement over the statically typed framework is the lack of need for replacement func-tions. in the statically typed framework, each node has its own type. This means that if a specific traversal needs to replace a node of a specific kind for a node of a different kind, it needs to change the properties of the parent node, as the parent node expects its child to be of a specific type. In the dynamically typed framework, however, this is not necessary, as each node has the same type. Instead of these replacement functions, we need dynamic consistency checks. If a specific node expects a node in nodeset ‘Expr’, then a replaced node must also be a node in nodeset ‘Expr’.

A drawback of the dynamic implementation is that each time a change is made to the DSL specification, the entire framework must be recompiled. But, considering that the dynamic backend generates only one file for the AST code and therefore the compilation process is less complex than for the statically typed backend.

3.4 Traversals

For the traversal functionality, one of the largest concerns was how to traverse the AST efficiently. This means that certain nodes should not be traversed during the traversal, as traversing through certain nodes would not be necessary for the given traversal. We determine which functions should be traversed through by means of a function pointer table. This table consists of an array unique to each traversal, containing a pointer to the traversal function appropriate for the target node. We use function pointers as an efficient method to access the next function in the traversal. In higher level languages these tables are known as ‘virtual method tables’, but as we are bound to C, we emulate this functionality with standard C arrays.

For the traversals, we define two kinds of tables. The first is a subtable defined for each traversal, which contains an amount of pointers to functions equal to the amount of specified nodes. An example of these tables is defined in Figure 3.9.

1 const TravFunc print_vtable[_NT_SIZE] = {

2 &trav_error, &print_root, &print_exprlist, &print_unop, 3 &print_binop, &print_var, &print_num,

4 };

Figure 3.9: Example of function pointer table

The second is the general table containing a pointer to each of the previously mentioned subtables for each specified traversal. An example of these tables is defined in Figure 3.10.

1 const TravFunc *trav_vtable[_TRAV_SIZE] = {

2 error_vtable, print_vtable, free_vtable, copy_vtable, check_vtable,

3 };

Figure 3.10: Example of main traversal function pointer table

These traversal function pointer tables consist of four specific kinds of functions. The first of these functions is a function called only in case of errors. It is defined in Figure 3.11.

(25)

1 Node *trav_error(Node *arg_node) { 2 print_user_error("traversal-driver",

3 "Trying to traverse through node of unknown

type."); ,→

4 return arg_node;

5 }

Figure 3.11: Traversal error function, called in case of invalid traversal.

The second of these functions is the user traversal function. Each of these functions should be defined for all nodes specified per traversal and contain all of the desired user functionality for the resulting target compiler. An example of one of these functions is shown in Figure 3.12, which is a basic print of the node data.

1 Node *print_binop(Node *arg_node) {

2 printf("binop op=%d\n", BINOP_OP(arg_node));

3 arg_node = trav_children(arg_node);

4 return arg_node;

5 }

Figure 3.12: Example of user defined traversal function

The third of these functions is the traversal function through each child node. These have been made dynamic by accessing the children struct as an array, as seen in Figure 3.5. This function has a mandatory check for the existence of the node and its children. If either of these pointers is NULL it will simply return and continue with the traversal through the next node. It is defined in Figure 3.13.

1 Node *trav_children(Node *arg_node) {

2 assert(arg_node != NULL);

3 for (int i = 0; i < NODE_NUMCHILDREN(arg_node); i++) {

4 NODE_CHILDREN(arg_node)[i] = trav_opt(NODE_CHILDREN(arg_node)[i]); ,→ 5 } 6 return arg_node; 7 }

(26)

1 Node *trav_return(Node *arg_node) { return arg_node; }

Figure 3.14: Traversal function for ignored nodes

The user traversal functions and the children traversal function call the following two functions: The first of these functions is the main traversal function. This function should be called by the user. If the given node is NULL, then it calls the error function, as for this function it is mandatory that the given traversal node exists.

The function first determines which traversal function should be used. It does this by indexing the traversal function pointer table at the current traversal and the given node type for the appropriate function pointer. Then the function simply calls the function that pointer points to. It is defined in Figure 3.15.

1 Node *trav(Node *arg_node) {

2 assert(arg_node != NULL);

3 TravFunc trav_func = trav_vtable[TRAV_TYPE][NODE_TYPE(arg_node)];

4 return trav_func(arg_node);

5 }

Figure 3.15: Core traversal function for mandatory child nodes

As we can see, in contrast with the traversal function in the statically typed backend as shown in Figure 2.5, This implementation significantly lessens the amount of code generated. Instead of having to check the types of the current traversal and node, it is now sufficient to index the function pointer table at the indices at the current traversal and node. This allows for a truly dynamic traversal mechanism.

The second function simply adds a NULL-check to the mandatory traversal function and is defined in Figure 3.16.

1 Node *trav_opt(Node *arg_node) {

2 if (arg_node != NULL) {

3 return trav(arg_node);

4 }

5 return arg_node;

6 }

Figure 3.16: Core traversal function for optional child nodes

The trav and trav opt functions can be used by the user to signify whether a node is expected to exist during a given traversal.

(27)

using the function defined in Figure 3.15. Afterwards, the traversal is popped and the syntaxtree is returned. It is defined as in Figure 3.17.

1 Node *trav_start(Node *syntaxtree, TraversalType trav_type) {

2 trav_push(trav_type);

3 syntaxtree = trav(syntaxtree);

4 trav_pop();

5 return syntaxtree;

6 }

Figure 3.17: Traversal start function

The push function creates a new traversal data structure, whose function is to provide a stack of traversals, such that traversals can be called within an traversal that is already being executed. The implementation is defined in Figure 3.18. First it allocates a new traversal data structure. Then it assigns the current type and sets the pointer to the previous current traversal, which is a global variable, to the current traversal. Then, the new current traversal is the newly allocated traversal. Finally, it handles the allocation of the traversal attributes, which will be discussed in more detail in Section 3.6.

1 void trav_push(TraversalType trav_type) {

2 Trav *trav = (Trav *)mem_alloc(sizeof(Trav));

3 trav->trav_type = trav_type;

4 trav->prev = current_traversal;

5 current_traversal = trav;

6 InitFunc init_func = trav_data_init_vtable[trav_type];

7 init_func(trav);

8 }

Figure 3.18: Traversal push function

The pop function is defined analogously to the push function in Figure 3.19. It essentially reverses the actions done by the push function.

(28)

1 void trav_pop() {

2 if (current_traversal == NULL) {

3 print_user_error("traversal-driver",

4 "Cannot pop of empty traversal stack."); 5 return;

6 }

7 Trav *prev = current_traversal->prev;

8 FreeFunc free_func = trav_data_free_vtable[TRAV_TYPE];

9 free_func(current_traversal);

10 mem_free(current_traversal);

11 current_traversal = prev;

12 }

Figure 3.19: Traversal push function

3.5 Passes

The traversal start function was previously defined in Figure 3.17. The start function for passes are defined in Figure 3.20.

1 Node *pass_start(Node *syntaxtree, PassType pass_type) {

2 PassFunc pass_func = pass_vtable[pass_type];

3 return pass_func(syntaxtree);

4 }

Figure 3.20: Pass start function

As previously shown in Figure 3.10, traversal functions are determined by usage of a function pointer table. These tables are used to determine the function to be called by the phase driver. This same method is applied to passes as well. It is defined in Figure 3.21.

1 const PassFunc pass_vtable[_PASS_SIZE] = { 2 &pass_error,

3 &pass_createast, 4 };

Figure 3.21: Example of pass function pointer table

The pass functions can do various things. An example of these pass functions is shown in Figure 3.22. It creates an AST from various nodes. Usually this is done automatically using a lexer-and parser generator.

(29)

1 Node *pass_createast(Node *arg_node) { 2 Node *var = node_init_var("somevar");

3 Node *exprlist2 = node_init_exprlist(var, NULL); 4 Node *num1 = node_init_num(42);

5 Node *num2 = node_init_num(22);

6 Node *binop = node_init_binop(num1, num2, BO_add); 7 Node *exprlist1 = node_init_exprlist(binop, exprlist2); 8 Node *root = node_init_root(exprlist1);

9 return root; 10 }

Figure 3.22: Example of a pass function

Usually, passes like lexical and syntactic analysis are standard. As discussed before, this reads code and transforms it into an intermediate representation, which in our case is the generated AST. The compiler passes can then be applied to this intermediate representation.

3.6 Traversal attributes

A useful feature of a compiler framework is to allow the user to define their own variables for a given traversal. This specifies data that is defined ‘globally’ in the traversal. In the SAC framework this was implemented by having a locally defined ‘INFO’ struct for each traversal, which is then created once at the start of the traversal and then passed to each function in the traversal. We have implemented this by adding this functionality to the code generator DSL specification and expanding the traversal stack of the statically typed framework. We implemented the traversal data structure in the same way as the node data structure, with the user being able to define their own data in the DSL specification file. Its implementation is defined in Figures 3.23, 3.24 and 3.25.

1 typedef struct TRAV { 2 struct TRAV *prev;

3 TraversalType trav_type;

4 union TRAV_DATA trav_data;

5 } Trav;

Figure 3.23: Core traversal data structure

1 union TRAV_DATA {

(30)

1 struct TRAV_DATA_PRINT {

2 int nodecount;

3 };

Figure 3.25: Example of traversal data structure

In contrast to nodes which are only able to have a specific set of data types as their attributes, the user is free to use whatever data type they wish for the traversal attributes. The user can specify a header file specifying the user data structure, which is then automatically included in the traversal header file.

The traversal determines which constructor and destructor to use by usage of a set of dedicated function pointer tables. Their implementation is defined in Figures 3.26 and 3.27.

1 const InitFunc trav_data_init_vtable[_TRAV_SIZE] = {

2 &trav_init_error, &trav_init_print, &trav_init_return, 3 &trav_init_return, &trav_init_return,

4 };

Figure 3.26: Function pointer table for traversal attribute constructors

1 const FreeFunc trav_data_free_vtable[_TRAV_SIZE] = {

2 &trav_free_error, &trav_free_print, &trav_free_return, 3 &trav_free_return, &trav_free_return,

4 };

Figure 3.27: Function pointer table for traversal attribute destructors

We can see that for traversals that do not have a specified traversal data structure, it will simply not create or destroy any traversal data and instead return, as shown in Figures 3.30 and 3.31. For each traversal with specified data, the following constructors and destructors are generated:

1 Trav *trav_init_print(Trav *trav) {

2 trav->trav_data.TD_print = mem_alloc(sizeof(struct TRAV_DATA_PRINT));

,→

3 trav = print_init_trav_data(trav);

4 return trav;

(31)

1 void trav_free_print(Trav *trav) {

2 print_free_trav_data(trav);

3 mem_free(trav->trav_data.TD_print);

4 }

Figure 3.29: Example of traversal data destructor

The functions ‘print init trav data’ and ‘print free trav data’ should be defined by the user for each traversal and contain all the initialisations and freeing of the data specified by the user. Traversals that do not have their own attributes naturally do not have constructors and destruc-tors for their attributes and instead have pointers to functions that only return:

1 Trav *trav_init_return(Trav *trav) { return trav; }

Figure 3.30: Traversal data return constructor

1 void trav_free_return(Trav *trav) {}

Figure 3.31: Traversal data return destructor

When the user has initialised their desired traversal attributes, they can use the access macros to conveniently access the traversal attribute data. An example of one of these access macros is defined in Figure 3.32.

1 #define PRINT_NODECOUNT (trav_current()->trav_data.TD_print->nodecount)

Figure 3.32: Example of traversal data user access macro

This macro can be used as if it is a variable and can be read from and written to from anywhere within the current traversal.

(32)

done, but only for one specific node. Once all children nodes are tested, it will traverse into those children.

1 Node *check_exprlist(Node *arg_node) {

2 if (EXPRLIST_EXPR(arg_node)) {

3 if (!type_is_expr(EXPRLIST_EXPR(arg_node))) {

4 print_user_error("consistency checking", "Child \"expr\" of

node \"ExprList\" has nonallowed type.");

,→

5 }

6 }

7 if (EXPRLIST_NEXT(arg_node)) {

8 if (NODE_TYPE(EXPRLIST_NEXT(arg_node)) != NT_exprlist) {

9 print_user_error("consistency checking", "Child \"next\" of

node \"ExprList\" has nonallowed type.");

,→ 10 } 11 } 12 arg_node = trav_children(arg_node); 13 return arg_node; 14 }

Figure 3.33: Example of check traversal function

In Figure 3.34 we can see how the type of a node is compared to that of a nodeset. It simply tests each of the values of the nodeset. Since this is a string of ‘or’ operators, it is short-circuited once the type of the node is found.

1 bool type_is_expr(Node *arg_node) { 2 return ( 3 NODE_TYPE(arg_node) == NT_var || 4 NODE_TYPE(arg_node) == NT_binop || 5 NODE_TYPE(arg_node) == NT_num || 6 NODE_TYPE(arg_node) == NT_unop 7 ); 8 }

(33)

CHAPTER 4

Evaluation

In this chapter we will discuss the qualitative and quantitative analysis we have performed on the resulting framework. We will first perform a qualitative analysis to prove the current implementation works and then we will perform a quantitative analysis to compare it to the static framework.

4.1 Qualitative Analysis

For the qualitative analysis we will validate our approach with a simple compiler traversal, to demonstrate the functionality of the framework.

Our approach is as follows. The root phase contains only four subphases. These are:

• LoadProgram, calls the CreateAST pass which creates an AST based on a specified number of integer nodes.

• PrintBefore, calls the Print traversal which prints each integer in the AST before the transformation.

• Transform, calls calls the ConstantFolding traversal which transforms each binary operation node into a single integer node.

• PrintAfter, calls the Print traversal once more, which now prints the sum of all integer nodes in the AST.

We will now describe each of the compiler passes in this example. First, we describe the Cre-ateAST pass, as demonstrated in Figure 4.1. Here, the number of nodes is globally defined for testing purposes. First it creates an array of nodes which each get an integer value equal to its number (i.e. the first integer created gets a 1, the second gets a 2, etc.) Next, we generate all binary operation nodes. The first BinOp node gets two integer nodes assigned, while the other nodes each get the previously defined BinOp and a next integer node in the list of nodes. BinOp nodes all have the summation operator, to showcase the sum of numbers. Lastly, the root BinOp is assigned to an expression node, which is then assigned to the root node.

(34)

1 Node *pass_createast(Node *arg_node) {

2 int numbinops = numnodes - 1;

3

4 Node *nodes[numnodes];

5 for (int i = 0; i < numnodes; i++) {

6 nodes[i] = node_init_num(i + 1);

7 }

8

9 Node *binops[numbinops];

10 binops[0] = node_init_binop(nodes[0], nodes[1], BO_add);

11 for (int i = 0; i < numbinops - 1; i++) {

12 binops[i + 1] = node_init_binop(binops[i], nodes[i + 2],

BO_add); ,→

13 }

14

15 Node *expr = node_init_exprlist(binops[numbinops - 1], NULL); 16 Node *root = node_init_root(expr);

17

18 return root; 19 }

Figure 4.1: The CreateAST pass

For the print traversal, we only print the integer values, as the other nodes aren’t interesting to show in this example. it is defined in Figure 4.2.

1 Node *print_num(Node *arg_node) {

2 printf("num value=%lu\n", NUM_VALUE(arg_node)); 3 return arg_node;

4 }

Figure 4.2: The Print traversal function

Finally, we use a very simple constant folding traversal, as defined in 4.3. Here we test the type of the left and right child in the BinOp node. If they are both integer values, they can be added together, thus a new integer node with the sum of their values is created and returned.

(35)

1 Node *constantfolding_binop(Node *arg_node) {

3 Node *left = BINOP_LEFT(arg_node); 4 Node *right = BINOP_RIGHT(arg_node); 5

6 if (NODE_TYPE(left) == NT_num && NODE_TYPE(right) == NT_num) { 7 Node *new_node = node_init_num(NUM_VALUE(left) +

NUM_VALUE(right)); ,→ 8 node_free(arg_node); 9 arg_node = new_node; 10 } 11 12 return arg_node; 13 }

Figure 4.3: The constant folding traversal function

If we run this traversal for 10 nodes, we get the following output:

1 [CCN] start RootPhase 2 [CCN] *start LoadProgram 3 [CCN] *end LoadProgram 4 [CCN] *start PrintBefore 5 num value=1 6 num value=2 7 num value=3 8 num value=4 9 num value=5 10 num value=6 11 num value=7 12 num value=8 13 num value=9 14 num value=10 15 [CCN] *end PrintBefore 16 [CCN] *start Transform 17 [CCN] *end Transform 18 [CCN] *start PrintAfter 19 num value=55 20 [CCN] *end PrintAfter 21 [CCN] *start Cleanup 22 [CCN] *end Cleanup 23 [CCN] end RootPhase

(36)

4.2 Quantitative Analysis

For the quantitative analysis we will look at the execution time of a ”dummy” traversal of both the static backend and the dynamic backend through a perfect m-ary tree with various values of m. An m-ary tree is a tree of which each node has no more than m nodes. A perfect m-ary tree is a special case of an m-ary tree in which all leaf nodes are at the same depth. We define the breadth of the m-ary tree as the amount of children a node can have and the depth of the tree is the number of layers until the leaf nodes.

An example of one of these trees is shown in Figure 4.5.

Figure 4.5: Example of an m-ary tree with breadth 2 and depth 3

For this experiment, we kept the DSL specification constant between the two backends, but the implementation differs. In our example implementations we will use a breadth of 2 and a depth 3. The DSL is dependant on the breadth of the tree and therefore we generate it for each breadth value. In Figure 4.6 we show an example of the DSL specification we used for a breadth of 2 children nodes.

(37)

1 start phase RootPhase { 2 prefix = R, 3 actions { 4 LoadProgram; 5 Traverse; 6 } 7 }; 8 phase LoadProgram { 9 prefix = LP, 10 actions { 11 pass CreateAST; 12 } 13 }; 14 phase Traverse { 15 prefix = T, 16 actions { 17 TraverseAST; 18 } 19 }; 20 traversal TraverseAST { 21 prefix = TA, 22 travdata = { 23 int numnodes = 0 24 } 25 };

26 root node Root {

27 children { 28 TreeNode child 29 } 30 }; 31 node TreeNode { 32 children { 33 TreeNode child0, 34 TreeNode child1 35 } 36 };

Figure 4.6: The DSL specification used in the experiment for a breadth of 2

We are dealing with only two kinds of nodes. The first is the root node, which has only one child of the second type of node, which is TreeNode. Each TreeNode node has children either equal to the breadth of the tree or 0, as a leaf.

First, we look at the static implemenation of the AST creation. In order to dynamically create the tree for a variable breadth and height, we chose to recursively create the tree. We specify the depth of the tree as a global variable and use a helper function create layer to generate two

(38)

1 int depth = 3; 2

3 TreeNode *create_layer(int gen) {

4 TreeNode *newlayer = create_TreeNode();

5 if (gen < depth) { 6 newlayer->child0 = create_layer(gen + 1); 7 newlayer->child1 = create_layer(gen + 1); 8 } 9 return newlayer; 10 }

11 Root *pass_CreateAST_entry(Root *syntaxtree) { 12 Root *root = create_Root();

13 root->child = create_layer(0); 14 return root;

15 }

Figure 4.7: Implementation of the static AST creation pass

The dummy traversal only keeps track of the amount of nodes traversed. It is shown in Figure 4.8. As we have to explicitly traverse through each child, this traversal is dependant on the breadth of the tree.

1 struct Info {

2 int numnodes;

3 };

4 Info *TraverseAST_createinfo(void) {

5 Info *info = (Info *)malloc(sizeof(Info));

6 info->numnodes = 0;

7 }

8 void TraverseAST_freeinfo(Info *info) { 9 printf("numnodes: %d\n", info->numnodes);

10 free(info);

11 }

12 void TraverseAST_Root(Root *node, Info *info) {

13 trav_Root_child(node, info);

14 free_Root_tree(node);

15 }

16 void TraverseAST_TreeNode(TreeNode *node, Info *info) {

17 info->numnodes++;

18 trav_TreeNode_child0(node, info);

19 trav_TreeNode_child1(node, info);

20 }

Figure 4.8: Implementation of the static AST dummy traversal

For the dynamic backend, we can implement the creation pass in a much cleaner way. As we allow the user to assign nodes directly to the array of children, we can assign all children using

(39)

1 Node *pass_createast(Node *arg_node) { 2 Node *layer0 = node_init_treenode();

3 for (int var0 = 0; var0 < NODE_NUMCHILDREN(layer0); var0++) { 4 Node *layer1 = node_init_treenode();

5 for (int var1 = 0; var1 < NODE_NUMCHILDREN(layer1); var1++) { 6 Node *layer2 = node_init_treenode();

7 for (int var2 = 0; var2 < NODE_NUMCHILDREN(layer2); var2++) {

,→

8 Node *layer3 = node_init_treenode();

9 NODE_CHILDREN(layer2)[var2] = layer3; 10 } 11 NODE_CHILDREN(layer1)[var1] = layer2; 12 } 13 NODE_CHILDREN(layer0)[var0] = layer1; 14 } 15 return layer0; 16 }

Figure 4.9: Implementation of the dynamic AST creation pass

In Figure 4.10 we show the dynamic implementation of the dummy traversal. Here we also only keep track of the amount of nodes passed through. As we can traverse all children with the dedicated function described in 3.13, the dummy traversal function is not dependant on the breadth of the tree.

1 Trav *traverseast_init_trav_data(Trav *trav) {

2 TRAVERSEAST_NUMNODES = 0;

3 return trav;

4 }

5 void traverseast_free_trav_data(Trav *trav) { 6 printf("numnodes: %d\n", TRAVERSEAST_NUMNODES);

7 }

8 Node *traverseast_root(Node *arg_node) {

10 return arg_node; 11 }

12 Node *traverseast_treenode(Node *arg_node) {

13 TRAVERSEAST_NUMNODES++;

15 return arg_node; 16 }

(40)

1

2

3

4

5

6

7 Depth

10

4

10

3

10

2

10

1

10

0

10

1

10

2

Execution time (s)

Static

Breadth 1

Breadth 2

Breadth 3

Breadth 4

Breadth 5

Breadth 6

Breadth 7

Breadth 8

Breadth 9

Breadth 10

1

2

3

4

5

6

7 Depth

Dynamic

Figure 4.11: Results of running the experiment for both backends

From the results we can deduce that for low breadths, the execution time of the static backend is shorter than for the dynamic backend. In contrast, for higher breadths, the execution time of the static backend is significantly higher for the highest tested depths, going up to as much as 4.5 times higher in execution time. To better illustrate this, we look at the values for the depth of 7 in Figure 4.12.

(41)

2

4

6

8

10 Breadth

10

4

10

3

10

2

10

1

10

0

10

1

10

2

Execution time (s)

Execution time for a depth of 7

Static

Dynamic

2

4

6

8

10 Breadth

0

10

20

30

40

50

60 Execution time for a depth of 7

Static

Dynamic

Figure 4.12: Results of the experiment for the depth of 7 and a variable breadth (On a logarithmic and linear scale respectively)

From this we can conclude that the dynamic backend is more efficient for large scale compilers, while the static backend might be more suited for smaller scale compilers.

(42)

(43)

CHAPTER 5

Related work

In this chapter we will briefly discuss some of the related works that either inspired or is com-parable to our work.

5.1 Rose Compiler

Rose compiler [10] is an analysis tool that aids source-to-source transformation by usage of an AST.

The user provides a binary executable or source code to the Rose Compiler. The supported source code languages consist of, but is not limited to, C, C++ [11], Fortran [12] and OpenMP [13]. The provided code is then parsed and processed, which results into an intermediate representation in the form of an AST. This AST can either be analysed or transformed by traversing through the AST and applying transformations. The user is free in the transformations applied to the AST. After the transformation process is done, the backend generates source code based on the applied transformations to the AST. The result can either be the generated source code or a binary executable compiled from the generated source code.

This is comparable to the target compilers CoCoNut aims to help development for. The user provides code, which is transformed into new code based on user defined traversals. In Figure 5.1 we can see an example of one of these traversals.

(44)

1 class f2cTraversal : public AstSimpleProcessing { 2 public:

3 virtual void visit(SgNode *n); 4 };

5 void f2cTraversal::visit(SgNode *n) { 6 switch (n->variantT()) {

7 case V_SgSourceFile: {

8 SgFile *fileNode = isSgFile(n);

9 translateFileName(fileNode); 10 } break; 11 case V_SgProgramHeaderStatement: { 12 ... 13 } break; 14 default: 15 break; 16 } 17 }

Figure 5.1: Rose Compiler example traversal (Source)

The Rose Compiler traversal functionality works in a comparable way to the static CoCoNut traversal functionality. It uses switches to determine the type of node in the AST and applies traversal functions based on that type of node. Where it differs to CoCoNut is that it is written in C++, it makes use of virtual functions to allow for polymorphic inheritance, as discussed in Section 6.4.

5.2 CINCLE

CINCLE, or ”Compiler Infrastructure for New C/C++ Language Extensions” [14], is a compiler-based infrastructure written in C++ that allows the user to specify their DSL for a source-to-source transformation, comparable to Rose Compiler. It is especially catered towards stream parallelism. Examples of stream parallelism frameworks are MPI [15] and OpenMP [16]. The goal of CINCLE is to simplify the processes of creating high-level parallelism abstractions using standard C++. Like CoCoNut, it is divided into modules that provide the functionality of the framework. CINCLE is divided up into a front-end, a middle-end and a back-end. The front-end, like CoCoNut, makes use of Flex and Bison to parse a DSL specification to cre-ate an intermedicre-ate representation to pass through to the middle-end. The middle-end provides modules that can be provided by the user. Unlike CoCoNut however, the semantic analysis which is done by the CoCoNut frontend is done by the middle-end in CINCLE. The user can provide their own semantic analysis of their DSL specification. This is something that is done automat-ically in the CoCoNut frontend. The CINCLE middle-end also supports optional optimisations as another module. Lastly, the CINCLE back-end contains the actual code transformation mod-ules. It also provides a source-to-binary functionality which calls the user’s compiler of choice to compile the transformed code.

It makes use of an AST with one single type of node containing the data. This is also comparable to the dynamic backend CoCoNut now provides. The node data structure implementation is demonstrated in Figure 5.2. Like the CoCoNut dynamic backend, it provides an integer value for the type and an integer for the number of children nodes. In contrast to the CoCoNut dynamic backend however, it only has a pointer to an array of children nodes. This restricts the

(45)

the children array. The CINCLE data structure also has a pointer to the parent node. This is something we did not adopt in CoCoNut as it would require too much data to keep track of when creating or replacing nodes.

1 struct tree_node {

2 int type;

3 char *content;

4 long int childs_n;

5 struct tree_node *node_up; 6 struct tree_node **node_down;

7 unsigned long int start_line;

8 unsigned long int start_column;

9 unsigned long int end_column;

10 unsigned long int end_line;

11 12 }

(46)

(47)

CHAPTER 6

Discussion

6.1 Evaluation

As the construction of a large-scale compiler would be very time-consuming and beyond the scope of this project, we have only been able to run very small-scale examples of traversals and passes. For a good validation of the compiler framework, one could rewrite a large-scale compiler in CoCoNut and compare the performance of the original and the resulting compiler. For the didactic aspect of the compiler framework, we could measure the satisfaction of students using the static and dynamic backend.

6.2 Planned Features

Due to time constraints, we have not been able to implement certain planned features. The first of these features consists of enhanced dynamic consistency checks, which checks if a certain type of node is mandatory or disallowed at a certain point during the compilation process. An extra feature would be consistency and invariance rules. This would entail that a certain node should always be present, for example a symbol node for nodes that require it.

The second is binary- and textual serialisation. This was a feature for the static backend, but the implementation would require an entirely new serialisation design, as the current design is incompatible with the new dynamic implementation.

Finally, an extra feature to specify a single user pass or traversal to be generated could be useful for when new passes or traversals are added to the DSL. In this case, other files would not be overwritten.

6.3 Code Generation and User Contributed Code

Code generation, while convenient, has some inherent flaws. One of these flaws is the need of regenerating files when the DSL specification is altered. As a convenient feature for the user, we allow them to generate the source files containing the necessary functions for the specified compiler passes. However, if the user regenerates these files, they would lose their progress. We have mitigated this problem somewhat by only generating code the user needs to add to if they

(48)

intelligently adapts the file where necessary. This is however beyond the scope of this project, but could be interesting for future research.

6.4 Viability of C

This project focuses on the implementation of abstract tree nodes in a polymorphic way. While we came up with methods of achieving this in the C programming language, other languages like C++ [11] or newer languages like Rust [17] could be better suited for this project. For example, the initial node structure can, instead of being given a union of structs, be inherited by sub-classes for each node. This can look as follows:

1 class Node { 2 public:

3 std::vector<Node*> children; 4 };

5

6 class BinOp : public Node { 7 public: 8 Node* left; 9 Node* right; 10 }; 11 12 ...

Figure 6.1: Example of the C++ implementation of the node structure (Pseudo code)

The inherited functions can in turn contain virtual functions that contain the traversal functions which are determined during runtime. With this in mind, rewriting the framework in a C++ compatible way could be interesting for future research.

6.5 Attribute Bit Structs

For the storage of attributes, we have deliberated over how to efficiently store the data of at-tributes. One of these methods is the usage of so-called ‘bit structs’. Bit structs allow the user to specify the amount of bits a specific data field in a struct uses. However, these bit structs come with a cost. Bit structs are more costly on the processor, as the processor has to apply masking operations to the bit structs in order to acquire the requested data in the code. Therefore it can be an optional flag to let the user decide if they want to use it.

6.6 Ethical Remarks

A critical concern in our society is whether the usage of code actually is ethical or not. Code can be used in several malicious ways, both intended and unintended. Therefore it is necessary for programmers to take responsibility for the code they write. For this project, however, it is not necessarily clear how far the scope of responsibility reaches. One could argue that a framework that aids the construction of compilers could very well cause malicious code to be designed using the constructed compiler. In contrast, the compiler framework might merely be the enabler for the malicious acts and should therefore not be responsible of the actions of third parties.

(49)

CHAPTER 7

Conclusions

In this thesis we presented a dynamic approach to the existing CoCoNut framework. We suc-cessfully designed it in such a way that it does not generate a lot of code in comparison to the existing framework, it is easy to use and understand and therefore keeps the high-productivity philosophy as initially presented and it mitigates the need for any auxiliary functions.

CoCoNut is designed with a high accessibility in mind, which makes it suited for courses on compiler construction. The new dynamic backend introduced in this thesis allows students to choose their preferred programming style and allows for flexibility within the course. It also allows students to compare each other’s results and learn from their implementations.

The dynamic backend provides an abstraction layer over the original statically typed data struc-tures of the static backend and this allows for a more dynamic implementation of the framework, which includes a dedicated children traversal function and easier replacement of nodes.

From the quantitative analysis we can conclude that the performance between the two backends is similar, but for large trees, the dynamic backend is more efficient than the already existing static backend.

(50)

Acknowledgements

I’d like to thank dr. Clemens Grelck for his guidance, support and the helpful feedback and discussions. Furthermore I’d like to thank Damian Fr¨olich for his help with maintaining the GitHub repository and helping me understand the frontend and the static backend when they were unclear. Finally, I’d like to thank everyone who took the time to read and reviewed my thesis.

(51)

Bibliography

[1] L. Coltof, Coconut: A metacompiler-based framework for compiler construction in c high productivity, traversal optimization, and ast serialization, 2017.

[2] M. Timmerman, Coconut: A metacompiler-based framework for compiler construction in c scalability, modularity, space leak detection and garbage collection, 2017.

[3] D. Fr¨olich, Coconut: A phase driver for a compiler construction framework increasing con-sistency and agility, reducing boilerplate and errors. 2019.

[4] Vern Paxson, Flex, version 2.6.4, May 6, 2017. [Online]. Available: https://github.com/ westes/flex.

[5] R. M. Stallman et al., “Using the gnu compiler collection,” For GCC version, vol. 4, no. 2, 1988.

[6] C. Lattner, “Llvm and clang: Next generation compiler technology,” in The BSD confer-ence, vol. 5, 2008.

[7] C. Donnelly and R. M. Stallman, Bison: The YACC-compatible Parser Generator: Septem-ber 2003 Bison Version 1.875. GNU, 2003.

[8] C. Grelck et al., “Sac compiler construction framework,” SAC Development Team, 1994. [Online]. Available: http://www.sac-home.org.

[9] “ISO/IEC 9899:1999 Programming languages — C,” International Organization for Stan-dardization, Geneva, CH, Standard, Dec. 1999.

[10] D. Quinlan and C. Liao, “The ROSE source-to-source compiler infrastructure,” in Cetus users and compiler infrastructure workshop, in conjunction with PACT, Citeseer, vol. 2011, 2011, p. 1.

[11] “ISO/IEC 14882:1998 Programming languages — C++,” International Organization for Standardization, Geneva, CH, Standard, Sep. 1998.

[12] “ISO/IEC 1539:1991 Information technology — Programming languages — FORTRAN,” International Organization for Standardization, Geneva, CH, Standard, Jul. 1991.

[13] L. Dagum and R. Menon, “Openmp: An industry standard api for shared-memory pro-gramming,” IEEE computational science and engineering, vol. 5, no. 1, pp. 46–55, 1998. [14] D. J. Griebler et al., “Domain-specific language & support tools for high-level stream

parallelism,” 2016.

[15] M. P. Forum, “Mpi: A message-passing interface standard,” USA, Tech. Rep., 1994. [16] L. Dagum and R. Menon, “Openmp: An industry-standard api for shared-memory

pro-gramming,” IEEE Comput. Sci. Eng., vol. 5, no. 1, pp. 46–55, Jan. 1998, issn: 1070-9924. doi: 10.1109/99.660313. [Online]. Available: https://doi.org/10.1109/99.660313. [17] N. D. Matsakis and F. S. Klock, “The rust language,” ACM SIGAda Ada Letters, vol. 34,

CoCoNut: A New Dynamic Approach to Code Generation

Bachelor Informatica