Automated feedback on naming in programming assignments

(1)

Bachelor Informatica

Automated feedback on

nam-ing in programmnam-ing assignments

Sander Meester

June 8, 2018

Supervisor(s): dhr. drs. M.S. Stegeman

Inf

orma

tica

—

Universiteit

v

an

Ams

terd

am

(2)

(3)

Abstract

To educate students on the quality of code and especially the quality of their own programs it is essential that they receive feedback on their written code. Currently, this feedback is given by course lecturers and teaching assistants (TAs), which is a time consuming and poorly scaling process. We present a tool that recognises possible improvements on the naming style in the code and gives feedback based on guidelines from professional literature, which the student can use to improve the quality of their code. It was evaluated with the help of both TAs and students. The feedback of the tool was in line with the feedback that the TAs would give to students and the number of names not conform found guidelines decreased notably when students used the tool.

(4)

(5)

Introduction

1.1 Context

Understanding the importance of the quality of code starts at introductory programming courses. To teach the students of these courses how to improve the quality of their programs it is essential that they receive feedback on the quality of their code. This in contrast to whether it performs the given task correctly. However, to determine the quality of the code a teacher or teaching assistant (TA) has to look into the code of each student and write individual feedback and improvement

points, which is a time consuming process.

Since the number of students in the area of computer science is rising, this process will take more and more time each year. For this reason we investigated the possibility to automate the process of giving feedback to students on their code quality. This automation will be performed in the form of a tool that can extract a students style from his or her code and give feedback on the discrepancy between their code and the guidelines found in professional literature. A student will then be able to receive immediate feedback at any moment on their code quality without the need for a teacher.

Multiple books written on the subject of good quality coding emphasise the important role that chosen names of variables, functions, and arguments have on the quality of code. (Martin, 2008; McConnell, 2004; Hunt and Thomas, 1999). Naming is an essential part of code quality since almost all elements in a piece of code contain self-defined names. Choosing good names for your variables, functions and even files can make your code readable without the need of comments and declare in an instant the purpose of your code (Martin, 2008; Hunt and Thomas, 1999). For these reasons, this study will focus on that part of code quality.

Many tools exist that give feedback on aspects of code style. The choice of names, however, is something that no existing tool gives feedback on. For example, tools that check if code is PEP81compliant focus mainly on whitespace and indenting to improve readability and give no improvements on the choice of a name. Our contribution to the field of automated feedback tools are two novelties. We aim to validate the feedback given to the student based on literature guidelines and to give feedback in such a way that a student learns from his or her mistakes and understands why the chosen names need improvement.

There are three different aspects for choosing the naming guidelines that can be used to evaluate the student code. The first one is general guidelines, which are set for programming in general or one specific programming language. The second one is course specific guidelines, which are usually set by the course teacher and can differ from general guidelines. The last possibility are guidelines set by internal consistency. If a student does not follow any set guidelines, but defines their own that are consistent throughout the code, it could be said that he or she defined his own guidelines. Internal consistency might improve the quality, but we could only check whether the

(8)

code is consistent, not whether good names are used consistently. Course specific guidelines can be interesting for a certain teacher, but will make the tool less useful for all teachers that have their own course guidelines. General guidelines can be inferred from professional literature and can form a reference validated by the programming community. For these reasons, this study will only take general guidelines on naming into account.

Feedback from the tool will be based on the difference between the naming the student uses and the naming guidelines found in professional literature. To improve the quality of the code the students must be able to assess three different things. They need to know what good naming is, what the difference is between their naming and good naming, and how to improve their naming so that it becomes good (Sadler, 1989; Stegeman et al., 2016). Therefore, the feedback will inform the student of a guideline when a name is found not conform that guideline, why that name is not conform the guideline, and a possible improvement to make it conform.

An example would be when a student uses four different variables for four different players, e.g. player1 to player4. A guideline says: ’Do not use numbers in names’. The tool would assess that these four variables are used that differ just by number, and the following feedback would be given:

Try t o n o t u s e numbers i n names . O c c a s i o n s found : − p l a y e r 1 on l i n e number 1 − p l a y e r 2 on l i n e number 2 − p l a y e r 3 on l i n e number 3 − p l a y e r 4 on l i n e number 4 S i n c e m u l t i p l e names a r e u s e d t h a t d i f f e r j u s t by number , t r y t o t h i n k whether you c o u l d u s e a l i s t o r d i c t i o n a r y h e r e .

1.2 Research goal

The goal of this project can be divided into three parts. First we have to look into what good naming consists of. Second we have to find a way to automatically analyse student code to retrieve all names in the code that are not conform the guidelines found in the previous step. The third part consists of generating the right feedback on these retrieved names that is designed in such a way that the student can improve the quality of their naming. The effectiveness of this feedback will be evaluated and validated. This will result in a tool that uses the found guidelines in combination with the names the tool found in the students code to generate effective feedback to the student.

The next chapter of this thesis will contain the literature study on guidelines for naming, followed by a section on effective feedback and how it is constructed. In chapter three we explain how names were extracted from student code and how they were compared to the guidelines. Chapter four contains a description of the evaluation done for validating feedback and the tool. Afterwards we reflect on the project and end with conclusions and potential future work.

(9)

CHAPTER 2

Related work

2.1 Naming quality metrics

This section describes the result of the literature study on current naming guidelines, which will answer the question on what good naming consists of. It concludes with which guidelines will be implemented in the prototype.

In their books ’Clean Code’ by Martin (2008) and ’Code complete’ by McConnell (2004) they define indications of good naming. Additionally, the PEP8 Style Guide for Python Code is a very well-established style guide for Python coding conventions. It indicates naming guidelines in a more concrete way than the books named above and might be easier to use as a reference for feedback. Using these three references we constructed a list of items that define what good naming consists of, as found in referenced professional literature, divided into three groups.

1. Syntactic aspects. These are defined by the characters in the name, not its meaning, and can be checked by analysing just the name itself.

2. Semantic aspects. Defined by the meaning or language of the name. Extracting these from the code will require a form of Natural Language Processing (NLP).

3. Context dependent aspects. These are defined by their use in code. To check, a method to extract comparable meaning from code is necessary.

Each group contains its own guidelines, as shown below. 1. Syntactic aspects.

(a) Avoid numbers in names (Martin, 2008; McConnell, 2004).

(b) Avoid single letter names (Guido van Rossum, 2001; Martin, 2008; McConnell, 2004). (c) Names are spelled correctly (McConnell, 2004).

(d) Multi-words names are split up using underscores or using camelCase. The opinions variable names are divided between camelCase (Guido van Rossum, 2001) or underscores (Martin, 2008) or either (McConnell, 2004), but for function names underscores are the convention (Guido van Rossum, 2001; Martin, 2008; McConnell, 2004).

(e) Avoid encodings: don’t add type information to the name (Martin, 2008; Hunt and Thomas, 1999).

(f) Avoid names in words that are often misspelled (McConnell, 2004).

(g) Do not use boolean names with a negative in it as those are harder to read, for example notFound in a negation becomes not notFound, which is quite unclear (McConnell, 2004).

(10)

(h) Use pronounceable names: Human brains simply cope better with names that can be pronounced in your head (Martin, 2008).

2. Semantic aspects.

(a) Do not use multiple languages in one program (McConnell, 2004).

(b) Do not use names that are so general that they can be used for more than one purpose, for example temp says nothing about what it does (McConnell, 2004).

(c) A boolean variable should have a name that can only be true or false. For example, found or succes are good, but status less so (McConnell, 2004).

(d) Avoid disinformation: Don’t name a function intCollect when meaning InternationalCollect, as int already has a meaning other than the abbreviation of international, namely

integer (Martin, 2008).

(e) Prevent ambiguity: Don’t use names that can mean multiple things, or use multiples names that mean the same, for example do not use both delete and remove as function names (Martin, 2008; McConnell, 2004; Hunt and Thomas, 1999).

3. Context dependent aspects

(a) Use names that reveal its intention: A name should tell you how it is used and what it does (Martin, 2008; McConnell, 2004; Hunt and Thomas, 1999).

2.2 Related tools

The following section describes published tools that give automated feedback on code. It explains how they were constructed, how feedback was composed and what issues the authors encountered, which we can learn from when we create our own automated feedback tool.

Rees (1982) describes one of the first automated assessment tools that analysed the quality of written code. He intuitively selected ten measurements to assess from code, including two about naming, quoted below:

• “Length of Identifiers: Calculating the average length of all the programmer-defined identifiers gives a very approximate indication of the use of meaningful names. Readability is assumed to increase with longer identifier lengths.” (Rees, 1982)

• “Variety of Identifiers: This measure consists of the number of different programmer-defined identifiers used in the program. (Different declarations of the same name are not included in the count.) A correct program which employs considerably fewer identifiers is likely to be more readily understandable. However, a low figure also indicates extensive use of standard or anonymous types, absence of constants, the use of a single name for several purposes, etc., all militating against ’good’ style. Of the ten measures, this is probably the least intuitively satisfying measure.” (Rees, 1982)

Instead of giving feedback on these measurements, Rees decided to transform the measurement into a mark given. He tested his tool by using it in two different PASCAL courses, one for students with no programming experience and one with experience. When marking the students programs, his tool indicated better code styling was used by more experienced programmers, which is as expected. The only formal evaluation of his tool was done by examining a tenth of the programs marked and checking whether they were in line with what Rees himself would have awarded. He found that making the tool accessible to students resulted in students using the tool for immediate feedback during programming, which contributed to higher overall quality of the students code. In his study Rees did analyse naming used in students programs, his feedback however was only a mark. The students were not explained how the mark was constructed.

Truong et al. (2004) created a framework for the static analysis of java programs written by students. Their framework was able to evaluate both the quality and the difference between the student code and the model solution. To evaluate the quality, they analysed a number of software

(11)

engineering metrics like total number of variables and number of logic decisions by counting all occurrences found in the code. The feedback given to the student consisted of a list of found values, with the possibility for a teacher to set a specific accepted value. A limitation of the tool is that it does not perform semantic analysis. The feedback given also does not include suggestions for a student to improve their code.

Singh et al. (2013) implemented an automated method for providing feedback on programming assignments of students. By comparing the code of the student to a reference implementation, combined with an error model (both set by the teacher) their automated system was able to give students insight in how erroneous their solution was, possible corrections, and feedback on what the student did wrong. They found it was difficult to validate the correctness of their feedback, since there are many improvement that could be considered correct. To ensure feedback validity of our own tool, a validation process is needed by comparing our feedback to what course teachers consider good feedback.

2.3 Design principles for automated feedback tools

As seen in the previous section, multiple attempts have been made to automatically assess students code. As a result, studies have been done on which factors lead to functional tools and what can be learned from encountered issues.

Pieterse (2013) argues that programming courses can profit from the use of automated assessment of the assignments given. She states multiple success factors that are linked to automated assessment, collected from feedback of researchers that designed and used automated tools of their own.

She starts with stating that the test data used to validate the tool must be chosen carefully. The design is highly influenced by the results of the initial tests. Furthermore she states that the feedback given is essential for the improvement of the student and its role for the tool may therefore not be underestimated. If students are using the tool to receive feedback, they should also be able to submit their code multiple times. She also states that the tool should be easy to use and require no additional skills not related to the course to test their program. As a final note she states that the need for personal guidance can not be fully removed by the use of automated assessment.

Opposed to the success factors stated above, Pieterse also discusses some relevant issues that arise with using an automated tool to assess programming assignments.

The first issue is that a teacher should be aware that the workload of the course instructors will not be replaced by the tool, but at most partly taken away. One must also not forget that the use of the tool will result in new tasks for a lecturer, as feedback on the tool from students and testing it’s validity also cost time. Every new assignment could result in new issues with the tool that beforehand could not be anticipated.

As a second, the creativity of the student might be restricted when using an automated assessment tool. It is heavily influenced by the design of the tool, but a strictness on the design of the code might restrain the growth of student programming skills, since solutions that are also correct but different than what the automated program expected might still be assessed erroneously.

She raises two more issues, increase in plagiarism and malicious intents. When students are aware that their program is being automatically evaluated students tend to plagiarise more. Student can also try to trick or misuse the tool to execute malicious code on the lecturers computer where the tool is ran. (Pieterse, 2013).

2.4 Feedback

The last section of this chapter describes what feedback should consist of to achieve maximum effectiveness, according to literature. This should give us insight it how effective feedback is

(12)

constructed.

Edwards (2003) indicates that when given direct feedback on their work, while working on their assignment, scored higher and has less defects in their code. Furthermore, Hattie and Timperley (2007) explain that giving feedback to students is a valid method to promote learning. Feedback is especially effective if a student simply has a different interpretation of given task, instead of not understanding the task. It also states that feedback in the form of praising is rarely effective and therefore should not be given by the tool.

Sadler (1989) defines what feedback must achieve if we want the student to learn from it. A student has to gain insight in what is considered good on the given subject, how his work compares to what is considered good, and how to improve his work so that it reaches the level considered good (Stegeman et al., 2016; Sadler, 1989). The feedback given by the tool should cover as much of these three items as possible.

2.5 Summary

In conclusion, professional literature contains naming style guidelines that define whether a used name is good. These guidelines will be used to base the feedback of the tool on the students code on.

The different tools described in literature give no feedback on naming (Singh et al., 2013; Truong et al., 2004), or give feedback in the form of a mark for the entire code, without telling the student how this mark was constructed and what could be improved to raise the quality (Rees, 1982). Useful design principles and common pitfalls however can be learned from previous

attempts. (Pieterse, 2013).

Lastly, it is stated that feedback is an effective method to promote learning (Hattie and Timperley, 2007; Edwards, 2003)) and what feedback must achieve of to maximise the effectiveness of it.

(13)

CHAPTER 3

Method

In this project we aim to answer the question on how an automated feedback tool is created that gives feedback on naming in programming assignments such that the student can improve the quality of his or her code.

To create such a tool we defined a list of naming guidelines in section 2.1. Using this list we have a frame of reference we can use to compare the naming in student code to. When a student runs the tool over their program, the tool will check every name in the program against the guidelines. Afterwards, the tool will output feedback to the student on the names used that conflict with the guidelines.

We will perform two evaluation rounds on the prototype of the tool. One to validate the correctness of the feedback and one to test the effectiveness of the feedback the tools gives.

For the prototype, we will focus on the following guidelines: • Avoid numbers in names.

• Avoid single letter names.

• Do not use both underscores and camelCase in a name. • Do not use boolean names with a negative in it.

• Avoid encodings: don’t add type information to the name.

• Names are spelled correctly and Multi-words names are split up using underscores or using camelCase.

• Do not use multiple languages in one program.

These are chosen since of all these guidelines can be either true or false when checked with a name. Whether a name contains a number can be true or false and nothing in between. Other guidelines require forming an opinion that is hard to grasp by an automated system. For example “Do not use names that so are general that they can be used for more than one purpose” is difficult

since there is no way to detect when a name is ‘too’ general.

After collecting all names, we can check every name against each guideline listed above.

3.1 Design of the tool

3.1.1 Technical design choices

We chose to use Python as the programming language to implement the prototype for. The reasons being it is commonly used in programming assignments at our institution (University of Amsterdam). Another benefit of Python is the ast library. Using the ast library we can generate and traverse an abstract syntax tree (AST) from the students code. An abstract syntax tree holds

(14)

the structure of a piece of code in a tree representation, with each node representing an element of the code. For example, a function has a node with its definition and has a link to a function body node. We can traverse this tree and collect the names from the nodes that hold a name.

We chose to use an AST to analyse the programs structure and names since an AST can efficiently attain a structural representation of a piece of code, which is both well-defined and easily understood.

Below is a piece of example code with its corresponding AST. def t e s t f u n c t i o n ( t e s t a r g ) : t e s t v a r = 10 print ( t e s t v a r ) Module Body FunctionDef - lineno: 1 - name: ’test function’

- args: ’test arg’

FunctionBody Assign - lineno: 2 Targets Name - lineno: 2 - id: ’test var’ - Ctx: Store() Value Number - lineno: 2 - n=10 Expression - lineno: 3 Value Call - lineno: 3 Function Name - lineno: 3 - id: ’print’ - Ctx: Load() Args Name - lineno: 3 - id: ’test var’ - Ctx: Load()

We can traverse this AST to iterate over all nodes in the tree, initially adding a name to a list when encountering a Name node, but in the first implementation of the the tool we found that when detecting all names from the Name nodes, also imported python function names were included. For example, the print() function would also be added to function names used in the code, as seen in the graph above. This would force the student to code in English, as imported python function names are English and the tool would give feedback if multiple languages are used. Also, the student has no control over those names, so feedback on it would not achieve anything. To solve this issue, we changed detecting all names from the Name nodes to detecting them from the Name nodes with a store operation and FunctionDef nodes. Only variables are used as a store, so no standard Python function get included. The imported Python functions also get no dedicated FunctionDef node, so only the self-written function names are included. The FunctionDef nodes also contain the arguments, which therefore could also detected using this new method. In the example above, this would mean only test name and test function would be found.

(15)

The tool is unable to create an AST if there is a syntax error in the code of the student. In this case, the tool tells the student their is a syntax error in their code so they understand why the tool does not run and exits.

3.1.2 Detecting corrigible names

Each name extracted from the code has be checked against each of the guidelines described in the beginning of this chapter.

The first three guidelines are checked with regular expressions, where each name is checked for occasions of wrong characters (e.g. numbers or both underscores and capital letters). Using the Python re package we can check against a self defined regular expression. For example (re.match(0(_{? = . ∗ [0 − 9]$)}0_{, name)) returns True if name contains a number. Also using the re}

package we can substitute a number in a name with that number +1 (or +n to be precise). Using this method we check if multiple names exist that differ only by a number. For example we are able to give feedback that numbers should be avoided and it might be beneficial to use a list if the code contains player1 to player4.

Figure 3.1: An example of the feedback the tool gives on numbers in names.

Guidelines four and five are checked by whether a name contains an item in a selected set of words. For the fourth guideline this is a list of words that indicate negative like ‘not’ and ‘none’. For the fifth guideline this a list of all Python types. After creating those lists, we can check if a name contains one of the items in the list and give feedback if it does.

The last two guidelines are checked with the use of dictionaries. For the detection of the languages used in the code there is one English and one Dutch dictionary. If a word is only present in one of the dictionaries, that language is used in the code. I chose to use dictionaries instead of Pythons Natural Language Toolkit language detection package, since that package needs sentences to function properly, while I can only use single words. For the detection whether a name is spelled correctly and Multi-words names are split up using underscores or using camelCase, we check all words in a name after splitting the name on capital letters and underscores. All words that remain should be present in one of the dictionaries. If it is not, it is either incorrectly spelled, or consists of multiple words that should have been split up.

Common used abbreviations were added to both dictionaries, such as tmp, max, min, temp, init, vars and num. These abbreviation are common practice and it’s meaning is clear when found in code.

The resulting tool can be run on a file and will then output the feedback to the terminal. This feedback will consist of a list of possible improvements found in the code of the student, along with examples of non guideline conform names found in the code.

3.2 Evaluation

To check whether the given feedback from the tool can be used by students to improve the quality of the names used in their code, we have evaluated the output given by the tool. One evaluation was done by course teachers and teaching assistants (TA) that grade and provide feedback to

(16)

student assignments. TAs were asked to write feedback on code of a student, which could then be compared by the feedback from the tool on that code.

A second evaluation was done with the help of students from introductory programming courses. They were asked to improve code based on the feedback given by the tool.

3.2.1 Teaching assistant feedback evaluation

To get information of how teaching assistants (TAs) give feedback on students code, we performed a student code evaluation round. Knowledge of how TAs give feedback gives us a reference to compare to improve our tool’s feedback, since it should be in line with TA feedback. Before starting the evaluation, we gave the TA a written set of guidelines on paper and asked them for permission to record the evaluation.

The guidelines:

• Avoid single letter names. • Avoid numbers names.

• Avoid encodings: don’t add type information to the name. • Do not use both underscores and camelCase in a name. • Names are spelled correctly.

• Multi-words names are split up using underscores or using camelCase.

Then, using their own interpretation of these guidelines, the TA went through the printed out student files and marked every name that in their opinion should be improved.

We occasionally inferred reasoning why a TA noted a certain improvement point, especially when it deviates from the set guidelines.

After going through the file, we asked the TA to write down a maximum of three things he or she would like to give to the student as feedback, which gave us a notion of how to focus the feedback of the tool.

This process was done with two different TAs and 5 student files for each TA. The files are found in Appendix A. These files were randomly selected from a dataset of anonymous student code, with the condition that no two pieces of code come from the same course assignment.

At the end of the process, the question was asked whether they give feedback if a student uses multiple languages in a program, and if they do, let them write down which feedback that would be.

3.2.2 Student evaluation

In this evaluation we tested whether the given feedback achieves a measurable increase in the quality of the code, measured by the amount of non guidelines conform names in the code. This will answer the question whether the student is able to identify names that need improvement after interpretation of the feedback and is able change the name so that it no longer displays characteristics that the fool gives feedback on.

To achieve this, we performed an evaluation round on the feedback from the tool with the help of programming students.

The student was given a piece of code and was asked to look through the code focusing on the names used. After inspection, the student was also given the feedback produced by the tool (on paper) and was then asked to improve the code to his or her own interpretation of the feedback. An example of the feedback is shown below.

Of this code we counted beforehand all the occasions of names that are not conform at least one of the following guidelines.

(17)

Figure 3.2: An example of the feedback the tool gives which was given to the student

• Avoid encodings: don’t add type information to the name. • Names are spelled correctly.

• Multi-words names are split up using underscores or using camelCase. • Do not use multiple languages.

All of these items can be objectively checked, with the last one only being counted as 0 or 1 (yes or no). A name can count for multiple guidelines, as a student might improve a name on one guideline, but not improve it on another.

After the improvements done by the student, we counted the occasions again. The feedback given to the student only gives feedback on one name per guideline (as the tool standard does). In the ideal case the student is able, based on the feedback on similar names, to detect names that also need improvement and improve those as well. This also tested the student’s understanding of the feedback given by the tool.

For every name in the code that was still not conform the guidelines after improvements are done, we asked the student whether he or she simply missed it, or was not aware that the name needed improvement based on the feedback. We also asked whether the student understood everything in the feedback.

Two files were be evaluated. Both files are adaptations of submitted student files, not written by the students that improve the code. Some code was removed or shortened to shorten the evalu-ation time. They were handpicked from the student code data-set and can be found in Appendix B.

The functionality before and after the improvements was not checked, even though the functionality should be identical before and after improvements are done. The reason for this is that the goal of this project is not to improve the functionality of the student code, but the quality of the code by improving the names used.

(18)

(19)

CHAPTER 4

Results

After evaluating the tool in its first prototype as described in the previous chapter, we can use this evaluation to reflect on our list of naming guidelines and how they compare to real world evaluation. It can also tell us how we could improve the feedback our tool gives. After each evaluation slight changes were made to the tool. The results of each evaluation are described in the following sections. Prior to these results, we show the amount of names not conform the list of guidelines found in student code.

4.1 Statistical analysis

To validate that corrigible names can be found in student files we used frequency statistics. We ran a preliminary version of the tool over 535 student files, counting the occasions that a name not conform a certain guideline was found.

Type Occasions

Number in name 1293

Single letter as name 1159

Both underscores and camelCase in a name 502 Negative boolean variable name 0

Encoding in name 320

Misspelled name (including wrong split-up) 1917 Multiple languages used in program: 265

Table 4.1: Occasions of corrigible names found in 535 student programs from 149 students

4.2 Teaching assistant feedback evaluation results

Below is the summarised feedback the TAs give related to every guideline. • Avoid numbers in names.

- When names with numbers were used in student code, both TAs would always give feedback on the student that different numbers make no clear distinction. When 2 or more names were used that differ only by number, they would give as feedback that a list or array would be the better option here.

- When encountering single letter names the TAs would tell the student that its meaning is simply not clear. They made an exception when a student used x, y or z when denoting coordinates, n when denoting an amount or single letter variables as for loop iterators.

(20)

• Avoid encodings: don’t add type information to the name.

- On this point both TAs diverged from tool feedback, as they found that adding a type to a name is not by definition wrong and could even add useful information to a name. For example, the variable random list was fine in their opinion.

• Names are spelled correctly.

- Both TAs gave feedback on every name clearly incorrectly spelled, but they would not give feedback when a clear abbreviation was used. One TA also would give feedback on names that were correctly spelled but quite long as a result. In her opinion short, abbreviated names were better than long, correctly spelled ones. The tool is unable however to detect all possible abbreviations.

• Multi-words names are split up using underscores or using camelCase.

- Every name containing both underscores and capital letters was given feedback on. The TAs would give as feedback that either is fine, but both is certainly not.

• Do not use multiple languages.

- When a student used more than one language in their or her code, the TAs would give the feedback that either language is fine, but that choosing one and sticking to it is essential.

4.2.1 Improvements done to the tool

A few changes to the tool were made after the evaluation to prevent feedback on names that are considered correct. These false positives were found when the discrepancy between TA and tool feedback was checked.

• Occasions were a single letter variable was used as a for loop counter, were no longer added to the list of corrigible names even though they conflicted with a guideline, since they are standard practice. Detecting when a variable was used as a for loop counter could be done by inspecting the parent node of the variable in the AST,

• Single letter variables x, y, z and n were also not added, since its meanings are common knowledge.

4.3 Student evaluation results

4.3.1 Evaluation with student one

In file 1 as shown in Appendix B, 28 corrigible names were counted before improvement done by the student. After the student went over the file with feedback from the tool, 4 corrigible names were counted. This file is also shown in Appendix B.

Names that the tool still gave feedback on were names with 1 and 2, e.g. counter1 and counter2) The student said that he did not see the added benefit of using for example a tuple here, and in his opinion the one and two were clear distinctions here

Also, the function name with Monopoly in it (simuleer potje Monopoly) was still given feedback on by the tool, since it has both uppercase letters and underscores, but in this case it is a name, so a uppercase letter is correct, so it was a false positive not corrected by the student here)

In file 2, 9 corrigible names were counted before improvement done by the student. After the student went over the file with feedback from the tool, 4 corrigible names were counted.

Names that the tool still gave feedback on were names with 1 and 2, e.g. counter1 and counter2) The student said that he did not see the added benefit of using for example a tuple here and in his opinion the one and two were clear distinctions here.

4.3.2 Evaluation with student two

(21)

Names that the tool still gave feedback on were names: - simuleer potje monopoly (she did change Monopoly to monopoly, but kept the spelling of the board-game, which is correct in my opinion) - gevonden lijst (in her opinion lijst (the Dutch word for list) does add valuable information here and is not unnecessary extra info)

In addition to changing the names, the students also changed the flow of the code from duplicating the same code multiple times for each name with a different number, to using a for-loop to do the same for all items in the list, as seen in appendix A. It is however unclear if this was caused by the tool or would have been done by the students on their own accord. Nonetheless, it is a clear improvement in the code.

4.3.3 Improvements done to the tool

The students mentioned in the evaluation that it would be nice to see all names that were found to be corrigible by the tool, instead of just an example. To achieve this, the tool was changed to to output all corrigible names found, with an option to silence the tool to only one example.

4.4 Final version of the tool

The feedback outputted by the final version of the tool on the first of the student evaluation files (as found in appendix B) is shown below.

(22)

(23)

CHAPTER 5

Discussion

In this project we aimed to create an automated feedback tool capable of delivering feedback to students on the quality of naming in code.

Our results shows that students were able to improve the non-conform names in the code. This indicates that they are able to understand what is considered good quality naming and how to improve their names to achieve this good quality in the code. From which we can conclude that the feedback of the tool satisfies the three feedback requirements set by Sadler (1989), as stated in the related work section.

Pieterse (2013) stated success factors for automated feedback. During the development of the tool, these factors were taken into consideration. It is constructed in such a way that it is easy to use by students, without the need for additional knowledge. It is a lightweight tool that only takes a few seconds to run and therefore can easily be run every time the student would like to receive feedback. This tool is not intended to be used as a replacement for teacher feedback. It gives complementary feedback during the development process of the student that adds no extra strain on the workload of teachers. Her issue on lowered creativity of the student is however not tested and could be a result of using the tool.

Her issues on plagiarism and malicious intents are relevant when the program of the stu-dent is automatically assessed by a program run by the lecturer, but are not applicable to our tool in its current design. The tool will be ran by students on their own computer, so malicious intents by students to trick or misuse the tool will be ineffective. An increase in plagiarism as noted to happen by Pieterse when using automated tools should not arise with our feedback tool. Student programs will still be manually evaluated by course teachers, since the tool is only intended to teach the student a better naming style during the development process. In the final version of the tool we chose for a high verbosity, where every name not conform to the found naming guidelines was output by the tool. Both high and low-verbosity have their advantages. No proof is given that a high verbosity teaches students more about code quality and how to improve their naming. Even though the students stated that they preferred to receive all found occasions as feedback, this might result in the students learning less from the feedback. A student might simply run the tool very often in a trial-and-error method to prevent wrong names, instead of learning from the feedback and thinking critically why the current names are wrong. Removing all examples of non-conform names might force the student to really think about the names and why some are wrong. However, it might lead to frustration when the tool keeps giving feedback because a single name is not correct and the student is unable to pinpoint which name needs improvement. A study into the effects of different verbosity might give us insight into which is the better approach.

During the TA evaluation we found that it was difficult to keep the TAs focused on the scope of the project as planned. We informed them of the guidelines and gave the instruction to give feedback on the code using their interpretation of the guidelines. When the TAs went through

(24)

the code, however, they often gave feedback on what the name meant instead of its syntactic contents. For example feedback stating that a name contained not enough information, or its meaning was unclear. Even though this is valid feedback, it was not implemented in the tool and could therefore not be used. The problem caused by this is that the TA sometimes moved on after that form of feedback was given, while the name was also not conform one of the guidelines. We then had to either accept that a TA missed a feedback point because he or she focused on other forms of feedback, or tell the TA this kind of feedback was outside of the scope, nudging the TA in a direction. This nudging might cause the TA to give feedback that would normally not be given, which can change the outcome of the evaluation.

To solve this, we could have used an extra student file the TA gives feedback on and discussed the feedback given on that file before starting the actual validation. This way we could inform the TA what kind of feedback is within the scope and what kind is not without pointing them towards certain names that in our opinion need feedback.

In the student evaluation we were forced make a few assumptions due to the nature of the test materials. The code was not written by the student itself. First, this setup is not how the tool is intended to be used, since the tool is designed to be used by students to improve their own code. Second, code not written by the student itself might be harder to understand, or might be written in a style the student would never write.

We also performed no validation whether the student would not have improved the naming in the code without the help of the tool, since no control group was present in the evaluation. To improve our evaluation we could have asked student to first hand in a couple of their assignments we could then choose from to use in the evaluation with them. To check whether students would not have improved their code without the tool we could have added a preliminary improvement round were the student would be asked to improve the code before we gave them the feedback of the tool. Furthermore, the size of both the student and TA groups used to perform the validation was too low to draw significant conclusions.

5.1 Future work

To better validate the effectiveness of the tool, a more extended evaluation should be done. Ideally, we would test the tool with two different groups, one that will use the tool (test group) and one that will not (control group). Over multiple assignments, the test group with the tool should produce notably better quality names in code, which could be assessed by lecturers or teaching assistants that normally grade in a course.

We assumed that the students being able to improve non-conform names indicates increased knowledge of naming quality. To test whether this assumption is valid we could stop student from using the tool halfway through the evaluation. If the student learned from using the tool what good good quality names consist of they should still achieve better results than the control group even without using the tool.

The next step in the development of the tool would be to add support for the guidelines not currently included. A major guideline is whether a name explains how it is used and what it does. To add this, a method of extracting the context and meaning of code is necessary, which is a difficult but interesting subject of study.

The tool could be extended to other languages than just Python. The work done in this project can form a base on how a tool in another language would be created, but the actual tool would be have to be written from scratch since core functions of this tool only work in Python. The naming guidelines and feedback given by the tool are reusable across programming languages.

(25)

5.2 Conclusion

In this project we collected a list of guidelines that define good naming. A subset was selected based on whether they could be evaluated as true or false, with the least amount of subjectivity as possible. The statistical analysis of student code shows that ample amount of non-guideline conform names can be found in student code, which validates that an automated feedback tool on naming does indeed have its use. For this subset, we created a prototype version of an automated feedback tool that was able to extract all names from a student file and formulate feedback based on the difference between student names and the guidelines on naming. This prototype shows that chosen guidelines can be used to generate automated feedback on the code of a student. A pilot study shows a possible method of testing the effectiveness of the feedback, which for our prototype showed positive results. The feedback from the tool was in line with TA feedback and students were able to notably decrease the amount of names used in the code that conflicted with the list of guidelines. The positive results of student improvement show that automated feedback tools could have a positive impact on programming education.

(26)

(27)

Bibliography

Stephen H Edwards. Using test-driven development in the classroom: Providing students with automatic, concrete feedback on performance. In Proceedings of the international conference on education and information systems: technologies and applications EISTA, volume 3. Citeseer, 2003.

Nick Coghlan Guido van Rossum, Barry Warsaw. Pep 8 – style guide for python code. July 2001. John Hattie and Helen Timperley. The power of feedback. Review of educational research, 77(1):

81–112, 2007.

Andrew Hunt and David Thomas. The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999. ISBN 0-201-61622-X. Robert C. Martin. Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall PTR,

Upper Saddle River, NJ, USA, 1 edition, 2008. ISBN 0132350882, 9780132350884.

Steve McConnell. Code Complete, Second Edition. Microsoft Press, Redmond, WA, USA, 2004. ISBN 0735619670, 9780735619678.

Vreda Pieterse. Automated assessment of programming assignments. In Proceedings of the 3rd Computer Science Education Research Conference on Computer Science Education Research, CSERC ’13, pages 4:45–4:56, Open Univ., Heerlen, The Netherlands, The Netherlands, 2013. Open Universiteit, Heerlen. URL http://dl.acm.org/citation.cfm?id=2541917.2541921. Michael J. Rees. Automatic assessment aids for pascal programs. SIGPLAN Not., 17(10):33–42, October 1982. ISSN 0362-1340. doi: 10.1145/948086.948088. URL http://doi.acm.org/10. 1145/948086.948088.

D Royce Sadler. Formative assessment and the design of instructional systems. Instructional science, 18(2):119–144, 1989.

Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. Automated feedback generation for introductory programming assignments. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’13, pages 15–26, New York, NY, USA, 2013. ACM. ISBN 978-1-4503-2014-6. doi: 10.1145/2491956.2462195. URL http://doi.acm.org/10.1145/2491956.2462195.

Martijn Stegeman, Erik Barendsen, and Sjaak Smetsers. Designing a rubric for feedback on code quality in programming courses. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research, Koli Calling ’16, pages 160–164, New York, NY, USA, 2016. ACM. ISBN 978-1-4503-4770-9. doi: 10.1145/2999541.2999555. URL http://doi.acm.org/10.1145/2999541.2999555.

(28)

(29)

(30)

(31)

APPENDIX A

Files evaluated in evaluation by TA

(32)

A.1 Monopoly realistisch.py

#Monopoly met 2 spelers en geld,

# voordeel van de speler die begint bepalen en opheffen

import random

import matplotlib.pyplot as plt

#Worp met twee dobbelstenen

def worp_met_twee_dobbelstenen(): dobbelsteen1 = random.randint(1,6) dobbelsteen2 = random.randint(1,6)

som_dobbelstenen = dobbelsteen1 + dobbelsteen2 return som_dobbelstenen

startgeld_speler_1 = 1500 startgeld_speler_2 = 1500

def simuleer_potje_Monopoly(startgeld_speler_1, startgeld_speler_2):

#Lijst met een 0 voor elk vakje op het bord, # wordt de lijst, met de straten die gekocht zijn

gevondenlijst = [] for n in range (0, 40):

gevondenlijst.append(0)

#lijst met prijzen van straten

bordwaarden = [0, 60, 0 , 60, 0, 200, 100, 0, 100, 120, 0, 140, 150, 140, 160, 200, 180, 0, 180, 200, 0, 220, 0, 220, 240, 200, 260, 260, 150, 280, 0, 300, 300, 0, 320, 200, 0, 350, 0, 400] positie1 = 0 positie2 = 0 aantalstraten = 0 aantalstraten_1 = 0 aantalstraten_2 = 0

# er kunnen 28 straten gekocht worden

while aantalstraten < 28:

#positie van speler 1 op het bord

positie1 = positie1 + worp_met_twee_dobbelstenen()

# na een rondje weer aan het begin van het bord beginnen

if positie1 > 39:

positie1 = positie1 - 40

startgeld_speler_1 = startgeld_speler_1 + 200

# positie van speler 2 op het bord

positie2 = positie2 + worp_met_twee_dobbelstenen()

#na een rondje weer aan het begin van het bord beginnen

if positie2 > 39:

positie2 = positie2 - 40

(33)

#speler 1 koopt de straat waar hij op komt als die te koop is, # en hij het kan betalen

if bordwaarden[positie1] > 0 and gevondenlijst[positie1] == 0: if bordwaarden[positie1] <= startgeld_speler_1:

gevondenlijst[positie1] = 1 aantalstraten = aantalstraten + 1 aantalstraten_1 = aantalstraten_1 + 1

startgeld_speler_1 = startgeld_speler_1 - bordwaarden[positie1]

#speler 2 koopt de straat waar hij op komt als die te koop is, # en hij het kan betalen

gevondenlijst[positie2] = 1 aantalstraten = aantalstraten + 1 aantalstraten_2 = aantalstraten_2 + 1

startgeld_speler_2 = startgeld_speler_2 - bordwaarden[positie2]

#verschil in het aantal straten in bezit van beide spelers na 1 potje

delta = aantalstraten_2 - aantalstraten_1 return delta

# Veel potjes monopoly doen om het gemiddelde verschil in bezit te berekenen

def

simuleer_groot_aantal_potjes_Monopoly(startgeld_speler_1,startgeld_speler_2):

,→

N_potjes = 10000 deltatot = 0

for i_potje in range (0, N_potjes):

delta = simuleer_potje_Monopoly(startgeld_speler_1,startgeld_speler_2) deltatot = deltatot + delta

gemiddelde = float(deltatot) / N_potjes return gemiddelde

# verschil in startgeld bepalen waarbij de spelers gemiddeld # allebij even veel straten kopen

def evenwicht(startgeld_speler_1,startgeld_speler_2):

#op zoek naar een bedrag dicht bij het evenwicht, met stappen van 50 euro

startgeldlijst = [] while simuleer_groot_aantal_potjes_Monopoly( startgeld_speler_1, startgeld_speler_2 ) < 0: startgeld_speler_2 = startgeld_speler_2 + 50 startgeldlijst.append(startgeld_speler_2)

#Vanaf het net bepaalde bedrag verder gaan, maar in stappen van 25 euro

if simuleer_groot_aantal_potjes_Monopoly(startgeld_speler_1, startgeldlijst[len(startgeldlijst)-2]) ** 2 < simuleer_groot_aantal_potjes_Monopoly(startgeld_speler_1, startgeldlijst[len(startgeldlijst)-2] + 25) ** 2: ,→ return startgeldlijst[len(startgeldlijst)-2] elif simuleer_groot_aantal_potjes_Monopoly(startgeld_speler_1, startgeldlijst[len(startgeldlijst)-2]+25) **2 < simuleer_groot_aantal_potjes_Monopoly(startgeld_speler_1,

(34)

startgeldlijst[len(startgeldlijst)-2] + 50) ** 2: ,→ return startgeldlijst[len(startgeldlijst)-2] + 25 else: return startgeldlijst[len(startgeldlijst)-1] voordeel_speler_1 = simuleer_groot_aantal_potjes_Monopoly( startgeld_speler_1,startgeld_speler_2 )

print "Monopoly simulator: twee spelers, 1500 euro startgeld, 10000 potjes"

print "Gemiddeld heeft speler 1 %.2f meer straten in bezit als \

alle straten verdeeld zijn" % (voordeel_speler_1)

uitkomst_evenwicht = evenwicht(startgeld_speler_1,startgeld_speler_2) - 1500 print 'Monopoly simulator: 2 spelers'

print 'Als we speler 2 %d euro meer startgeld meegeven hebben beide spelers \

(35)

A.2 basejump.py

#Let op: Bij checkpy moet je de grafiek die up-popt meteen wegklikken, anders #gaat ie time out door de een of andere reden.

import numpy as np import matplotlib.pyplot as plt def basejump(): dt = 10**-4 h = 0 h_lw = 0 G = 6.674 * 10**-11 M = 5.972 * 10**24 v = 0 v_lw = 0 t = 0 t_lw = 0 xi = 0.24 m = 72 time = [] height = [] velocity = [] timelw = [] heightlw = [] velocitylw = [] burj = 828 for i in np.arange(0,10**2,dt):

#Berekeningen voor zonder luchtweerstand

r = 6371 * 10**3 + (burj - h) a = G * M / r**2 v = v + a * i h = h - v*i t = t + i #Met luchtweerstand (lw) r_lw = 6371 * 10**3 + (burj - h_lw) a_lw = (G*M*m/r_lw**2 - xi*v_lw**2) / m v_lw = v_lw + a_lw*i h_lw = h_lw - v_lw*i t_lw = t_lw + i

#Mbv de if-statements hieronder hoeft de loop pas te stoppen als de #de basejumper zonder lw ook beneden is.

if h >= -burj + 100: time.append(t) height.append(h) velocity.append(v*3.6) if h_lw < -burj + 100: break timelw.append(t_lw)

(36)

heightlw.append(h_lw) velocitylw.append(v_lw)

return time, height, velocity, timelw, heightlw, velocitylw

#Hieronder maak ik 2 aparte grafieken waarin de tijd met de hoogte en snelheid #uitgezet wordt.

plt.suptitle('Basejumper')

plt.title('Met luchtweerstand (blauw) en zonder luchtweerstand (groen)') plt.subplot(122) #snelheid

plt.plot(basejump()[0],basejump()[2]) plt.plot(basejump()[3],basejump()[5],'g-') plt.xlabel('tijd (s)')

plt.ylabel('snelheid (km/u)') plt.subplot(121) #hoogte plt.plot(basejump()[3],basejump()[4],'g-') plt.plot(basejump()[0],basejump()[1]) plt.ylabel('hoogte (m)') plt.xlabel('tijd (s)') plt.show() timelw = basejump()[3] time = basejump()[0] print (time[len(time)-1])

(37)

A.3 temperatuur.py

# Dag 2.1: functies die de vragen beantwoorden over de bestanden van het ECA

# functie: nummer omzetten in juiste datum

# parameters: datum = getal wat moet worden omgezet in de datum

def omzetter(datum):

maanden = ["januari", "februari", "maart", "april", "mei", "juni", \

"juli", "augustus", "september", "oktober", "november", "december"] jaar = int(datum[0:4])

maand = maanden[int(datum[4:6]) - 1] dag = int(datum[6:8])

hele_dag = "%d %s %d" % (dag, maand, jaar) return hele_dag

# functie: de minimum en maximum temp uit de lijsten halen # parameters: nvt

def max_en_min():

input_file_max = open('DeBiltTempMax.txt', 'r') input_file_min = open('DeBiltTempMin.txt', 'r')

# voor de max

max_temp = 0 temp = 0 teller = 0

for line in input_file_max: teller = teller + 1 data_opgeknipt = line.split(',') if teller > 19: temp = float(data_opgeknipt[3]) temp = temp / 10 # hoogste temperatuur if temp > max_temp: max_temp = temp datum_warm = data_opgeknipt[2] datum_warm = omzetter(datum_warm) # voor de min teller = 0 temp = 0 min_temp = 0

for line in input_file_min: teller = teller + 1

(38)

if teller > 19: temp = float(data_opgeknipt[3]) temp = temp / 10 # laagste temperatuur if temp < min_temp: min_temp = temp datum_kou = data_opgeknipt[2] datum_kou = omzetter(datum_kou) input_file_max.close() input_file_min.close()

return max_temp, min_temp, datum_kou, datum_warm

# functie: langste periode negatieve temperatuur # parameters: nvt

def koud():

input_file_max = open('DeBiltTempMax.txt', 'r') teller = 0

temp = 0

langste_periode = 0 periode = 0

for line in input_file_max: teller = teller + 1 data_opgeknipt = line.split(',') if teller > 19: temp = float(data_opgeknipt[3]) temp = temp / 10 datum = float(data_opgeknipt[2]) if temp < 0: periode = periode + 1 if periode > langste_periode: langste_periode = periode laatste_dag = data_opgeknipt[2] laatste_dag = omzetter(laatste_dag) else: periode = 0 input_file_max.close()

return langste_periode, laatste_dag

# functie: grafiek van de zomerse en tropische dagen # parameters: nvt

(39)

input_file_max = open('DeBiltTempMax.txt', 'r') L_t_dagen = [] L_jaar = [] L_z_dagen = [] teller = 0 temp = 0 prev_jaar = 1901 z_dagen = 0 t_dagen = 0

for line in input_file_max: teller = teller + 1 data_opgeknipt = line.split(',') if teller > 19: temp = float(data_opgeknipt[3]) temp = temp / 10 jaar = int(data_opgeknipt[2][0:4]) if temp > 25: z_dagen = z_dagen + 1 if temp > 30: t_dagen = t_dagen + 1

# data in lijst zetten per jaar

if jaar != prev_jaar: L_jaar.append(prev_jaar) L_z_dagen.append(z_dagen) L_t_dagen.append(t_dagen) z_dagen = 0 t_dagen = 0 prev_jaar = jaar # voor 2015 L_jaar.append(prev_jaar) L_z_dagen.append(z_dagen) L_t_dagen.append(t_dagen)

plt.plot(L_jaar, L_z_dagen, 'y-', label = 'zomerse dagen') plt.plot(L_jaar, L_t_dagen, 'r-', label = 'tropische dagen') plt.xlabel('jaar')

plt.ylabel('dagen')

plt.title('Pittig warme dagen in NL') plt.legend()

plt.savefig("temp.png") input_file_max.close() return

(40)

# parameters: nvt

def hittegolf():

input_file_max = open('DeBiltTempMax.txt', 'r') teller = 0 zom_teller = 0 zom_golf = 0 trop_teller = 0 trop_golf = 0 eerste_keer = 0

for line in input_file_max: teller = teller + 1 data_opgeknipt = line.split(',') if teller > 19: temp = float(data_opgeknipt[3]) temp = temp / 10 jaar = int(data_opgeknipt[2][0:4]) if temp >= 25: zom_teller = zom_teller + 1 zom_golf = zom_teller if temp >= 30: trop_teller = trop_teller + 1 trop_golf = trop_teller else: trop_teller = 0 else: zom_teller = 0

# golf moet 5 dagen boven de 25 en 3 dagen boven de 30 zijn

if zom_golf >= 5 and trop_golf >= 3 and eerste_keer == 0: golf_jaar = jaar

eerste_keer = 1 input_file_max.close() return golf_jaar

maxi, mini, dag_min, dag_max = max_en_min() tijd, dag = koud()

warm()

print "De hoogste temperatuur was %.1f graden Celcius, en werd gemeten op %s."

\

,→

% (maxi, dag_max)

print "De laagste temperatuur was %.1f graden Celcius, en werd gemeten op %s."

\

,→

% (mini, dag_min)

(41)

met als laatste dag %s." % (tijd, dag)

print "Het eerste jaar van de eerste hittegolf is %d." % hittegolf()

A.4 montecarlo.py

# de waarde van een integraal bepalen mbv de montecarlo methode

import math

import random

# de maximale x en y waarden van de functie

x_min = 0 x_max = math.pi y_min = 0 y_max = 1

# voor 3 functie de methode toepassen

def functie_1(x): return math.pow(x, x + 0.5) def functie_2(x): return math.tan(math.cos(math.sin(x))) def functie_3(x): return math.sin(x**2)

# hier wordt de montecarlo functie gedefinieerd

def montecarlo(func, x_1, y_1, x_2, y_2): N = 200000

# in de lijsten bewaren we de random x en y waarden die goed of fout zijn

negatief_goede_X = [] negatief_goede_Y = [] positief_goede_X = [] positief_goede_Y = [] slechte_X = [] slechte_Y = []

# genereer random waarden en bepaal of die goed of fout zijn

for i in range(0, N):

random_x = random.random() * (x_2 - x_1) + x_1 random_y = random.random() * (y_2 - y_1) + y_1 y_value = func(random_y) if random_y < 0: if random_y > y_value: negatief_goede_X.append(random_x) negatief_goede_Y.append(random_y) else: slechte_X.append(random_x) slechte_Y.append(random_y) if random_y > 0: if random_y < y_value:

(42)

positief_goede_X.append(random_x) positief_goede_Y.append(random_y) else:

slechte_X.append(random_x) slechte_Y.append(random_y)

# bepaal de waarde van de integraal

aantal_negatief_goed = len(negatief_goede_X) aantal_positief_goed = len(positief_goede_X)

integraal = ((float(-(aantal_negatief_goed - aantal_positief_goed))/N) * ((x_2-x_1)*(y_2-y_1)))

return integraal

# roept de functie aan

integraal_waarde = montecarlo(functie_1, x_min, y_min, x_max, y_max)

# plot de punten

plt.xlim([x_min, x_max]) plt.ylim([y_min, y_max])

plt.plot(negatief_goede_X, negatief_goede_Y, 'go') plt.plot(positief_goede_X, positief_goede_Y, 'go') plt.plot(slechte_X, slechte_Y, 'ro')

(43)

A.5 randomwiskunde.py

#calculating average amount of random numbers (0-1) needed, to have sum > 1 #importing needed libraries

import random

#creating lists

randomlist = [] poginglist = []

#defining function randomwiskunde

def randomwiskunde():

#loop to repeat process 100.000 times

for i in range(0,1000001): som = 0

pogingen = 0

#add random number (0-1), till sum > 1

while som < 1:

x = random.random() som = som + x

pogingen = pogingen + 1

#if sum > 1 reached, append sum, and amount of random numbers needed, # to differend lists

randomlist.append(som) poginglist.append(pogingen) total = 0

#adding all different sums of random numbers together

for i in poginglist: total = total + i

#adding all ammounts of random numbers of of different sums together

aantal = len(poginglist)

#calculating average of random numbers(0-1) needed to sum up to 1

average = float(total)/aantal return average

#print result

(44)

(45)

APPENDIX B

Files evaluated in evaluation by student

B.1 Student evaluation file original

def worp_met_twee_dobbelstenen():

dobbelsteen1 = random.randint(1,6) dobbelsteen2 = random.randint(1,6)

# bereken de som van beide dobbelstenen

s = dobbelsteen1 + dobbelsteen2 return s startgeld_speler_1 = 1500 startgeld_speler_2 = 1500 startgeld_speler_3 = 1500 startgeld_speler_4 = 1500

def simuleer_potje_Monopoly(startgeld_speler_1, startgeld_speler_2, startgeld_speler_3, startgeld_speler_4):

#Lijst met een 0 voor elk vakje op het bord, # wordt de lijst met de straten die gekocht zijn

gevondenlijst = [] for n in range (0, 40):

gevondenlijst.append(0)

bordwaarden = [0, 60, 0 , 60, 0, 200, 100, 0, 100, 120, 0, 140, 150, 140, 160, 200, 180, 0, 180, 200, 0, 220, 0, 220, 240, 200, 260, 260, 150, 280, 0, 300, 300, 0, 320, 200, 0, 350, 0, 400]

# Initaliseer de posities van de spelers

positie1 = 0 positie2 = 0 positie3 = 0 positie4 = 0

# Initaliseer de straten van de spelers en het totaal

total_aantal_straten = 0 aantalstraten_1 = 0

(46)

aantalstraten_2 = 0 aantalstraten_3 = 0 aantalstraten_4 = 0

while total_aantal_straten < 28:

positie1 += worp_met_twee_dobbelstenen()

if positie1 > 39: positie1 -= 40

startgeld_speler_1 += 200

#na een rondje weer aan het begin van het bord beginnen

# speler 1 koopt de straat waar hij op komt, # als die te koop is en hij het kan betalen

gevondenlijst[positie1] = 1 aantalstraten += 1

aantalstraten_1 += 1

startgeld_speler_1 -= bordwaarden[positie1]

# speler 2 koopt de straat waar hij op komt als die te koop is, # en hij het kan betalen

gevondenlijst[positie2] = 1 total_aantal_straten += 1 aantalstraten_2 += 1

(47)

(48)

B.2 After improvements by student one

dobbelsteen1 = random.randint(1,6) dobbelsteen2 = random.randint(1,6)

som = dobbelsteen1 + dobbelsteen2 return som

start_geld_speler = [1500, 1500, 1500, 1500]

def simuleer_potje_Monopoly(start_geld_speler):

#Lijst met een 0 voor elk vakje op het bord, wordt de lijst met de straten die gekocht zijn

,→

gekochte_straten = [] for n in range (0, 40):

gekochte_straten.append(0)

bordwaarden = [0, 60, 0 , 60, 0, 200, 100, 0, 100, 120, 0, 140, 150, 140, 160, 200, 180, 0, 180, 200, 0, 220, 0, 220, 240, 200, 260, 260, 150, 280, 0, 300, 300, 0, 320, 200, 0, 350, 0, 400]

positie = [0,0,0,0]

total_aantal_straten = 0 aantal_straten = [0,0,0,0]

while totaal_aantal_straten < 28: for i in range(len(positie)):

# positie van speler op het bord

positie[i] += worp_met_twee_dobbelstenen()

if positie[i] > 39: positie[i] -= 40

startgeld_speler[i] += 200

# Speler koopt de straat waar hij op komt als die te koop is en hij het kan betalen

,→

for i in range(len(positie)):

if bordwaarden[positie[i]] > 0 and gevondenlijst[positie[i]] == 0: if bordwaarden[positie[i]] <= start_geld_speler[i]:

gevondenlijst[positie[i]] = 1 total_aantal_straten += 1 aantal_straten[i] += 1

(49)

B.3 After improvements by student two

dobbelstenen = []

dobbelsteen[0] = random.randint(1,6) dobbelsteen[1] = random.randint(1,6)

totaal = dobbelsteen[0] + dobbelsteen[1] return totaal

startgeld_spelers = [1500, 1500, 1500, 1500]

def simuleer_potje_monopoly(startgeld_spelers):

#Lijst met een 0 voor elk vakje op het bord, wordt de lijst met de straten die gekocht zijn

,→

gevonden_lijst = [] for n in range (0, 40):

gevonden_lijst.append(0)

bord_waarden = [0, 60, 0 , 60, 0, 200, 100, 0, 100, 120, 0, 140, 150, 140, 160, 200, 180, 0, 180, 200, 0, 220, 0, 220, 240, 200, 260, 260, 150, 280, 0, 300, 300, 0, 320, 200, 0, 350, 0, 400]

posities = [0, 0, 0, 0]

total_aantal_straten = 0 aantal_straten = [0, 0, 0, 0]

while totaal_aantal_straten < 28: for i in range(len(posities)):

# positie van speler op het bord

posities[i] += worp_met_twee_dobbelstenen()

if posities[i] > 39: posities[i] -= 40

startgeld_spelers[i] += 200

# Speler koopt de straat waar hij op komt als die te koop is en hij het kan betalen

,→

for i in range(len(posities)):

if bordwaarden[posities[i]] > 0 and gevondenlijst[posities[i]] == 0: ,→ if bordwaarden[posities[i]] <= start_geld_spelers[i]: gevondenlijst[posities[i]] = 1 total_aantal_straten += 1 aantal_straten[i] += 1

(50)

Automated feedback on naming in programming assignments

Bachelor Informatica