Eindhoven University of Technology
MASTER
Evaluating different test insertion policies during the final test of a semiconductor production process
Bingel, C.T.
Award date:
2020
Link to publication
Disclaimer
This document contains a student thesis (bachelor's or master's), as authored by a student at Eindhoven University of Technology. Student theses are made available in the TU/e repository upon obtaining the required degree. The grade received is not published on the document as presented in the repository. The required complexity or quality of research of student theses may vary by program, and the required minimum study period may vary in duration.
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
NXP Semiconducters
Evaluating different test insertion policies during the final test of a semiconductor
production process
By:
C.T. Bingel ID: 0903044
Mechanical Engineering Dynamics and Control
Commission:
prof. dr. ir. I.J.B.F. Adan prof. dr. F.C.R. Spieksma Q.V. Dang PhD ir. J. Verstraten
December 3, 2020
NXP Graduation thesis
Preface
At this point in time, I am about the finish my studies at the University of Technology Eindhoven.
I started a little over six years ago in the fall of 2014 with a bachelor in mechanical engineering.
After four years of personal and academic development, I finished the bachelor’s program and started the master’s program manufacturing systems engineering. The master’s program included an international internship in Milan and concluded with my graduation project at NXP. Now after a very eventful and challenging 2020, I am about to graduate.
I would like to thank my supervisor Jan Verstraten from NXP, for the opportunity and su- pervision he provided during my graduation project. I would like to thank Ivo Adan for his mentorship during my masters and the supervision of my graduate project. Finally, I would like to thank my family, girlfriend, friends, and fellow students for guidance and support during the years.
Christian Bingel December 2020
Abstract
This thesis focuses on the final test of the production process of integrated circuits. The final test is the last quality check before distribution. The goal of this thesis is to make a model that can evaluate different test insertion policies. The two policies are: a single insertion policy and a split insertion policy. The single insertion policy tests each integrated circuit once per test program. The split insertion policy tests each integrated circuit twice per test program, in which one of the tests is tested with a higher test parallelism. The goal of the model is to evaluate if the extra insertion decreases the total test time enough to be worth the extra changeovers and hardware.
The thesis tackles this problem with an optimal scheduling model. The model will schedule a fixed set of test lots on both test insertion policies to evaluate important performance parameters.
Scheduling models are often NP-hard problems, this means that the solution space grows exponentially relative to the problem size. Three modeling methods will be evaluated to optimize the test schedule: a mixed-integer linear problem, a basic genetic algorithm, and a hybrid genetic algorithm with local search algorithms. The mixed-integer linear problem is an exact method and the basic genetic algorithm and the hybrid genetic algorithm with local search algorithms are heuristic methods. Performance tests have been carried out using each modeling method to assess the accuracy of the models and computing times.
The modeling method that is most promising is the hybrid genetic algorithm with local search algorithm. The performance of this method is a sweet spot between accuracy and computing time.
With this model, three integrated circuit products from NXP, which are currently being tested in Kuala Lumpur, are evaluated on its insertion policy. The model is also used to plot contour plots in which shows the area when splitting the insertion policy is financially viable. These contour plots can be used to evaluate future NXP products on their insertion policy choice.
NXP Graduation thesis CONTENTS
Contents
1 Introduction 2
1.1 The integrated circuit production process . . . 2
1.2 Integrated circuit testing . . . 3
2 Problem definition and research questions 5 2.1 Problem Definition . . . 5
2.2 Research questions . . . 7
3 Methodology 8 3.1 Approach . . . 8
3.2 Scheduling Methods . . . 9
3.3 Scope . . . 11
4 Exact method: Mixed-integer linear programming 12 4.1 Optimization problem: semiconductor final test . . . 12
4.2 Model implementation . . . 15
4.3 Model performance . . . 15
5 Heuristic method: Genetic Algorithm 17 5.1 Algorithm overview . . . 17
5.2 Chromosome definition . . . 18
5.3 Chromosome fitness assessment . . . 18
5.4 Chromosome selection . . . 20
5.5 Chromosome mutation . . . 21
5.6 Model implementation . . . 22
5.7 Hyperparameter factorial design . . . 22
5.8 Model Performance . . . 23
5.9 Model limitations . . . 24
6 Heuristic method: Genetic algorithm with local search algorithms 26 6.1 Algorithm overview . . . 26
6.2 Objective function . . . 28
6.3 Local search algorithms . . . 28
6.4 Model implementation . . . 33
6.5 Hyperparameter tuning . . . 33
6.6 Model Performance . . . 34
6.7 Validation . . . 35
7 Results 40 7.1 Actual products . . . 40
7.2 Split insertion financial viability . . . 51
8 Conclusion and recommendations 56 8.1 Conclusion . . . 56
8.2 Recommendations . . . 57
A Business case NXP 59
B Full empirical data of the changeovers times 60
1 Introduction
The birth of a semiconductor can be traced back to the invention of the rectifier (AC-DC converter) in 1874. Almost a century later in 1959, the bipolar integrated circuit (IC) was invented by Kilby of Texas Instruments and Noyce of Fairchild Semiconductor in the US. This invention had a major impact on the history of semiconductors, and it marked the dawn of the IC era. Since then the semiconductor industry has grown into a 500 billion dollar industry, in which semiconductors can be found in every single electronic device. Semiconductor manufacturers can be divided into three types: fabless, pure-play, and integrated device manufacturer (IDM). Fabless manufacturers design and sell semiconductor devices while outsourcing their fabrication to specialized manufactures.
A pure-play manufacturer produces semiconductor devices under contract from other companies without designing them. An IDM does all three of these components, it designs, manufactures, and sells semiconductor devices. NXP Semiconductors N.V. is a global IDM headquartered in Eindhoven. NXP used to be part of Philips which was called Philips Semiconductors at the time.
The company was sold to a consortium of private equity investors in 2006, at which point the company’s name was changed to NXP. NXP focuses on the manufacturing of integrated circuits for automotive, telecommunications, internet of things (IoT), and several other markets.
1.1 The integrated circuit production process
The production process of integrated circuits is complex and involves many steps. The production process can be divided into two parts: the front-end process and the back-end process. The front-end process refers to the process that forms the transistors directly on the silicon wafer.
The front-end process starts with a clean wafer and ends when the dies are fabricated on the wafer. After the wafer is patterned with dies the back-end process starts. The back-end process is the process from which the patterned dies are turned into the final product. Figure 1.1 shows the division between the front-end process and the back-end process.
Figure 1.1: Semiconductor manufacturing process
The figure shows only a small part of the back-end process. The full back-end process consists of the following steps:
1. Wafer probing: quality check of the dies on the wafer for production mistakes.
2. Wafer grinding: grinding the wafer to the desired thickness.
3. Wafer sawing: sawing the wafer into individual dies.
4. Die bonding: attaching the dies to a lead frame.
5. Wire bonding: connecting the dies to the lead frame with wires.
6. Molding: encapsulating the die and lead frame in an epoxy compound to protect it.
NXP Graduation thesis Introduction
7. Trim and form: cut lead frame into individual products and bent metal connections to the desired geometry.
8. Final test: test final products before packaging and shipping.
This study will focus on the testing part of the back-end process, in specific the final test.
1.2 Integrated circuit testing
Every integrated circuit (IC) goes through two types of tests before it can be sold and used to build a product: wafer test and final test. The wafer test (or probing) is the test process that checks for fabrication mistakes at the front-end process. A probe card is used to connect the dies on the wafer with the tester. Small probes touch down on the dies and test the dies on fabrication mistakes. Figure 1.2 shows a schematic and a picture of a probe card.
(a) A schematic overview of a probe card (b) A picture of probes touching down on a wafer Figure 1.2: An example of a probe card
The second type of test will be the last test on the product before it is shipped to its customers.
This test is called final test. Final test takes place after the products are assembled. The purpose of final test is to ensure that no products have been damaged during wire-bonding and packaging.
The final products are loaded into a load board, this serves as a connection tool between the products and the tester. Figure 1.3 shows an example of a load board with space for eight products.
Figure 1.3: An example of a load board
Each time a product is inserted into the load board and tested is called a test insertion. Products are often tested multiple times, depending on the product type and purpose. Common tests are temperature tests, in which the product is tested at a very high temperature and a very low temperature. This study focuses on a test insertion policy analysis in the final test of the integrated circuit production. Testing accounts for approximately five percent of the total expenses at NXP, so it often is an area where improvement needs to be made in order to decrease the production costs and increase the total profits of NXP.
NXP Graduation thesis Problem definition and research questions
2 Problem definition and research questions
This section takes a closer look at the problem at hand within NXP and discusses the research questions that will tackle the problem. The section explains what the different insertion policies are and how the test parallelism is determent. Finally, the importance of different changeovers is discussed.
2.1 Problem Definition
During the final test of the integrated circuit (IC) production process, the products are tested on manufacturing mistakes that might occur at the assembly process. When products are being tested, a lot of products need to be inserted into the tester. A lot consists of 100-15000 IC products. A lot is inserted at the test handler and the products are tested in batches. The batch size is also called test parallelism. The current research will revolve around the test insertion policies in the final test. The easiest insertion policy is to insert the product once per test program and let the tester run the whole test program. However, engineers at NXP found out that splitting the test program into two insertions could decrease the total testing time a lot.
This could be achieved by increasing the test parallelism of the second insertion. The first thing that needs to be explained is how the maximum test parallelism is determined. Each integrated circuit has a certain number of connection pins. These are connection points on the back or the side of an integrated circuit. These connection pins need to be tested on manufacturing mistakes during the assembly process. Semiconductor testers have a limited number of connections to which the pins can be connected. Divide the total connections at the tester by the connection pins on the integrated circuit to determine the maximum test parallelism. Figure 2.1 shows how this works.
Figure 2.1: An example of how the maximum test parallelism is determined
At one point in a test program, all the connection pins need to be connected to the tester, but for a significant portion of the test program, this is not the case. For that part of the test program, only a subset of the pins has to be connected, so this part of the test program can be done with higher parallelism. This means that a single test can be split into two tests, where the second test is being done at higher parallelism. This results in a lower net test time. Figure 2.2 shows how a single insertion policy works, the lot is tested once per test program with low parallelism.
Full pin means that all the connection pins are connected to the tester, this results in a low test parallelism. The Chevron bar represents a single test. Figure 2.3 shows the same test program but split up into two separate tests. The second test is a min pin test, this means that the minimal number of connection pins are connected to the tester resulting in the highest possible test parallelism. The split insertion strategy results in a lower total net test time.
Full pin test on a low test parallelism
Figure 2.2: A single insertion test policy
Full pin test on a low test parallelism Min pin test on a high test parallelism
Figure 2.3: A split insertion test policy
The split insertion policy has also downsides. Splitting a test program into two means that the total number of tests is doubled. This means that if a product is temperature tested, which normally happens with a hot temperature test and a cold temperature test, the product now has to be inserted four times instead of two times. This means more changeovers, which could result in higher cycle times. Due to the higher number of changeovers, the testers have higher idle times which results in a decrease in machine efficiency. With a split insertion policy, the second test is being done at higher parallelism. In order to achieve this, extra load boards need to be developed due to the fact that the load boards are different than the full pin test load boards. More sockets need to be added to the load board due to the increase in parallelism.
This increases the complexity of the load boards. The development and production of these load boards result in the test costs going up.
The changeover times between the different test programs are important for insertion policy problem, because they cause the most downtime for the testers. The split insertion policy has four test programs: cold and hot min pin test and cold and hot full pin test. This means that on this policy there are 16 possible changeovers with 16 different changeover times. Within those 16 possibles, there are three types of changeovers. The first type is a changeover in which the next test program is the same as the current test program. This changeover takes the shortest amount of time, because nothing has to be changed on the tester. The second type is a temperature changeover on the same load board, for example, a cold min pin to hot min pin. This changeover takes longer due to the temperature change of the tester. Depending on the current test program and the next test program, the tester needs to cool-down or warm-up. The final type is a changeover in which the load board needs to be changed. This happens when a switch is being made from a min pin test to a full pin test or vice versa. This type of changeover has the longest duration of all the changeovers.
Table 2.1 summarizes the trade-off of using each of the insertion policies.
Table 2.1: The trade-off between the different insertion policies Performance parameter Single insertion policy Split insertion policy
Net test time Higher Lower
Number of tests per test program 1 test 2 tests
Changeovers Less More
Time consuming changeovers Limited hardware change
so short changeovers Changeovers more time consuming due more hardware change
Tester efficiency Higher Lower
Load board usage Minimal load boards required More and complexload boards needed
NXP Graduation thesis Problem definition and research questions
2.2 Research questions
The goal of the research will be to make a model that can evaluate different insertion policies.
With the model, a comparison will be made with the two insertions policies for several products that are currently running in the test and assembly plant in Kuala Lumpur. The goal is to obtain insight into the effects of the different insertion policies on cycle time, total test time, costs, and changeover rate.
Main Research question
• What is the best insertion policy in final test for the evaluated NXP products?
Sub-research questions
• What is the best modeling method that can evaluate the insertion policy?
• What are the different changeover times between the different test programs?
• What is the difference in the total test time for the evaluated NXP products on the different insertion policies?
• What is the difference in costs for the evaluated NXP products on the different insertion policies?
• What is the relation between the tester capacity and the insertion policy choice?
3 Methodology
This section describes the methodology of the research study. This section starts with evaluating different modeling approaches. It shows the problem definition of the scheduling problem and evaluates different scheduling methods.
3.1 Approach
There are several methods to obtain insight into which insertion policy would be preferable over the other. Three of those are listed below.
1. Excel calculation 2. Simulation model 3. Scheduling model
An Excel calculation has the advantage of being easy to make and easy to understand if NXP wants to pick up the project after completion. The biggest downside of this method is that the complexity of the analysis is limited. A simulation model of the insertion policy problem can be tackled in various ways. The most used simulation method in manufacturing modeling is the discrete-event simulation. The advantage of this model is that the model is relatively easy to make. However, the biggest downside is that discrete-event simulation needs a fixed sequence of events, and the sequence of which lot is tested on which machine is an unknown parameter in the insertion policy problem. The last modeling method that is taken into consideration is a scheduling model. This model could schedule a fixed set of lots on a fixed set of machines. The model would optimize this schedule based on for example the fastest makespan of those lots.
This kind of model seems very suitable for the problem at hand. This model could compare a fixed set of lots on both of the insertion policies and find out which one would finish faster.
Now with the modeling method set, the scheduling problem needs to be defined. The scheduling model needs to be able to schedule a set of lots to achieve the lowest makespan of all scheduled jobs. This scheduling model also needs to comply with the following constraints:
• Lots can be scheduled on a fixed number of testers.
• Each lot is tested on several test programs in a predefined sequence. The single insertion strategy tests the cold test first and then the hot test, both tests are full pin tests. The split insertion tests in the following order: cold min pin test, hot min pin test, cold full pin test, hot full pin test.
• Before and after each test program there is a warm up or a cooling down time. These values depend on which test program comes next and are also called sequence-dependent changeover times.
• Switching a lot from one tester to another also takes a certain amount of time.
• Load boards are dependent on the test program and the test parallelism.
• The test jobs are non-preemptive, which means that each test job cannot be interrupted.
With these constraints, the scheduling problem is defined. The scheduling problem needs several input parameters that need to be collected from several databases and sources. These are:
• Test times per test program;
• Lot sizes;
NXP Graduation thesis Methodology
• Test parallelism per test program;
• Changeover times between test programs;
• Hardware costs;
NXP has several databases and data sources where data can be collected. The unit-level data from the testers at the assembly and test plant in Kuala Lumpur can be collected from Optimal+(O+) and Tool Time Tracker (TTT). These databases collect all the data from the current testers and they also have tools for analysis. From these databases, the test times per test program, lot sizes, test parallelism, and changeover times can be collected. This tool can give an insight into the determination of the machine capacity. The GNTC in Nijmegen has specialists on the wafer and final test. They have a lot of knowledge about the costs of the complex load boards and other test materials.
3.2 Scheduling Methods
This section contains theoretical background on different scheduling models and shows which models are used in the literature.
Scheduling literature
Table 3.1 shows several studies that have characteristics of a final test scheduling problem. The scheduling literature is quite broad and extensive. Different case studies have been done in the last two decades about a lot of scheduling problems. The key is to find several studies that resemble key characteristics of the scheduling problem at hand and combine them into one. The literature shows two main modeling approaches: an exact approach or a heuristic approach. An exact approach like mixed-integer linear programming is very accurate but can consume a lot of computing time when the problem is complex. Heuristic approaches consume less computing time but sacrifice accuracy.
Table 3.1: Scheduling literature
Multi-machine scheduling Changeover modeling Sequence independent Exact approach Heuristic approach
(Agrawal, Pattanaik, & Kumar, 2012) x x
(Achkar, Cafaro, Méndez, & Cafaro, 2019) x x
(Lin & Floudas, 2005) x
(Xiong & Zhou, 1998) x x x
(Wolsey, 1997) x x
(Sang, Duan, & Li, 2018) x x x x
(Wang, Wang, Liu, & Xu, 2015) x x x x
(Wu & Chien, 2008) x x x x
(Novara & Henning, 2015) x x x
(Rodrigues, Dourado, & Szwarcfiter, 2014) x x x
(Zheng, Wang, & Wang, 2014) x x x x
(Lee, Chang, Wong, & Jang, 2011) x x x x
The literature shows a preference in using an exact method like mixed-integer linear programming (MILP) for solving scheduling programs, but when computing time is limited a heuristic approach is preferred for solving complex scheduling problems. However, due to the nature of scheduling problems, the solution space can grow exponentially with the difficulty of the problem. In optimization theory, these problems are classified as NP-hard. These NP-hard problems are very difficult to solve and often heuristic approaches are used instant. This paper will evaluate both methods and compare them on accuracy and computing time.
Exact method: Mixed-integer linear programming
This section will explain what a mixed-integer linear program is and how it is used in scheduling theory. Integer programming or mixed-integer programming is a mathematical optimization program which restrict some or all or the variables to be integer. Often the objective function is linear and thus are these program referred to as mixed-integer linear program. The basic form of a mixed-integer linear program is shown below:
minx fTx
subject to: Ax ≤ b Aeqx = beq lb≤ x ≤ ub xi ∈ R xj ∈ Z xk∈ B
(3.1)
The basic form is very similar to a normal linear program however, in this case, the variables can be a set of real values, integers, and/or Boolean values (1,0). Mixed-integer linear programming is a popular method in solving job shop scheduling problems. For example, out of the 40 research papers published in the Journal of Scheduling in 2014, 14 use mixed-integer linear programming, more than any other technology (Ku & Beck, 2016). The advantage of a mixed-integer linear program is that with the integer variables, certain constraints can be easily defined. For example, if certain jobs need to be scheduled in a particular sequence.
Heuristic method: Genetic algorithm
This section will explain how a heuristic approach, like a genetic algorithm, is used in scheduling theory. A genetic algorithm is a metaheuristic that is inspired by the process of natural selection.
Genetic algorithms are often used for NP-complete or NP-hard optimization problems to find high quality near optimum solutions. The advantage of a genetic algorithm is that the solution takes a lot less computing time than a mixed-integer linear program. Figure 3.1 shows an example of a genetic algorithm. The key of a genetic algorithm is to simplify the solution of the program to a chromosome structure, this can be a single array or multi-column array. A single rows of these chromosomes are often called genes. With this chromosome structure, the fitness of the solution can be calculated, with the optimum solution being having the highest fitness.
NXP Graduation thesis Methodology
Figure 3.1: An example of a genetic algorithm
The algorithm starts with generating an initial population of chromosomes. This initial generation is assessed on its fitness and parents are selected from the initial population. The selection of parents can be done with several methods. In the literature the two most often used selection methods are the ranked selection (also called roulette wheel selection) and the binary tournament.
The ranked selection ranks the population-based on its fitness, it then picks the parents based on that rank. The higher the rank the higher the probability of picking that chromosome. The binary tournament method is different from the ranked selection. This method picks two chromosomes at random from the population and picks the fittest to be the parent.
A parent will mutate its chromosome by exchanging genes with an other parent. These mutations can be done in several ways but it is always important to keep the structure of the chromosome correct. This is needed to comply with the constraints set in the scheduling problem. After the mutation, the new chromosomes are the next generation of the algorithm. This process continues until the last generation is reached or if there is a different stop criterion activated.
3.3 Scope
The scope of the project will be limited to final test. The focus will be on several current products that are being tested on the UP1600 tester of Teradyne. The selected products are used in automotive and are mainly tested in two temperature tests, a hot test, and a cold test. The input variables of the model should be defined in a way that the model can be used for future products as well. Retests and other stochastic behaviour of the system is out of scope.
4 Exact method: Mixed-integer linear programming
This section evaluates an exact method of modeling the scheduling problem. The section investigates if a mixed-integer linear program will be a suitable modeling method to schedule the insertion policies at final test. The exact method will be evaluated on model performance and computing time.
4.1 Optimization problem: semiconductor final test
After evaluating papers that share key characteristics with the insertion policy problem, an opti- mization problem is defined. The optimization problem needs to include a multi-machine(testers) set-up, the changeovers between test programs, and sequence-dependent test programs. The subsection below will start with the mathematical formulation and follows with the objective function with the constraints and lastly, the section will end with an explanation of the constraints.
Mathematical formulation notations
j = a lot with a predefined amount of tests o= test program
m = the tester on which the job is processed
jo= operation (j, o), which means that job j is on test program o
jom= operation (j, o, m), means that operation (j, o) is processed on tester m cj = completion time of job j
cjo = completion time of operation (j, o) bjo = beginning time of operation (j, o) pjo = processing time of operation (j, o)
So1o2 = sequence-dependent changeover time between test program o1 and o2
xjomt =
(1 =if operation (j, o, m) is processed at time t 0 =otherwise
zjom =
(1 =if operation (j, o) is selected to be processed on tester m 0 =otherwise
N = the number of lots to be processed T = time horizon under consideration TDj = tardiness of job j
wj = weight factor put on a lot for the importance of meeting its due date Oj = set of tests programs for job j (amount of insertions)
A= set of all tests to be evaluated, i.e., {(j, o) | j ∈ {1, 2, . . . , N}, o ∈ Oj, }
M = set of testers
∆M t = available quantities of testers at time t
NXP Graduation thesis Exact method: Mixed-integer linear programming
µ= an arbitrary large number αjomt =
(1 =if operation (j, o) begins being processed on tester m at time t 0 =otherwise
βjomt =
1 =if operation (j, o) is processed on tester m at time t and is not being processed at time t + 1
0 =otherwise SQj101j202 =
(1 =if operation (j1, o1) before operation (j2, o2) 0 =otherwise
Objective function with constraints
Minimize Cmax (4.1)
Subject to:
T
X
t=1
xjomt− pjozjom= 0 ∀(j, o) ∈ A, ∀m ∈ M (4.2)
bjo=
T
X
t=1
X
∀m∈M
tαjomt ∀(j, o) ∈ A (4.3)
cjo=
T
X
t=1
X
∀m∈M
tβjomt+ 1 ∀(j, o) ∈ A (4.4)
cj > cjo ∀(j, o) ∈ A (4.5)
Cmax> cj ∀j ∈ {1, 2, . . . , N } (4.6)
cj1o1− bj2o2 + µ · SQj1o1j2o2 > 0 ∀(j1, o1) ∈ A, ∀(j2, o2) ∈ A (4.7)
cj1o1 − bj2o2+ µ · (1 − SQj1o1j2o2) 6 0 ∀(j1, o1) ∈ A, ∀(j2, o2) ∈ A (4.8)
SQjo1jo2 = 1 ∀j ∈ {1, 2, . . . , N }, ∀o1, o2 ∈ Oj if o2> o1 (4.9)
T
X
t=1 N
X
j=1
xjomt6 1 ∀o ∈ Oj ∀m ∈ M (4.10)
X
∀m∈M
zjom = 1 ∀(j, o) ∈ A (4.11)
X
∀m∈M
xjomt6 ∆Mt ∀(j, o) ∈ A, ∀t ∈ {1, 2, . . . , T } (4.12)
xj1o1mt+ xj2o2m(t+So1o2) 6 1
∀t ∈ {0, . . . , T }, ∀(j1, o1) ∈ A, ∀(j2, o2) ∈ A, ∀m ∈ M (4.13)
T
X
t=1
αjomt= zjom ∀(j, o) ∈ A, ∀m ∈ M (4.14)
T
X
t=1
βjomt= zjom ∀(j, o) ∈ A, ∀m ∈ M (4.15)
xjomt= αjomt ∀(j, o) ∈ A, t = 1 (4.16)
xjomt− xjom,t−1= αjomt− βjom,t−1 ∀(j, o) ∈ A, ∀m ∈ M, t ∈ {2, 3, . . . , T } (4.17)
xjomt= βjomt ∀(j, o) ∈ A, ∀m ∈ M, t = T (4.18) Equation (4.1) is the objective function of the MILP model, which minimizes the makespan. The makespan is the total scheduling time of all the test jobs. The objective function needs to be subjected to several constraints in order to comply with the problem definition. Equation (4.2) shows the constraint that requires the total time taken to process a certain job must be equal to the processing time of that job. This constraint states that the sum of xjomt must equal the processing time when the job is allocated on machine m.
Constraints are needed to define the beginning and the completion times of the jobs, these are set in Equation (4.3) and Equation (4.4). These constraints determine the moment in the horizon when the job starts and finishes, using the integer definitions of α and β. A constraint is needed to define the completion of all test programs of a lot. This marks the finishing time of the lot, this constraint is shown in Equation (4.5). Finally, the makespan constraint shows that the makespan must be higher or equal to all the finishing times of the lots. This is expressed in Equation (4.6).
The lots need to be scheduled in a predefined sequence, this requirement is set with the constraints in Equation (5.2). Due to the fact that every lot is tested multiple times with different test programs, a constraint is necessary to make sure a lot is not being processed on another machine before it finishes its current test program. This non-overlapping requirement is set with Equation (4.10). Each test job can only be allocated to one tester, this is constrained with Equation (4.11).
Equation (4.12) ensures that the sum of the total jobs being processed does not exceed the total amount of available testers.
After each test process, a changeover time needs to be added before starting with a new test process. These times are dependent on the current test program and the next test program.
Equation (4.13) demands that two jobs using the same tester must be separate for a time period of no less than the sequence-dependent changeover time.
Equations (4.14) and (4.15) show the non-preemptive constraints. The non-preemptive constraints will prevent that a process is being stopped and another process is being started on the same tester without the first process being finished. Preemptive scheduling is not considered in this
NXP Graduation thesis Exact method: Mixed-integer linear programming
scheduling problem. The final constraints set the relations between the integer variables in Equations (4.16), (4.17) and (4.18)
4.2 Model implementation
The mixed-integer linear program is implemented in Python with the Gurobi Optimizer. The Gurobi Optimizer is a commercial optimization solver for linear programming, quadratic program- ming, quadratically constrained programming, mixed-integer linear programming, mixed-integer quadratic programming, and mixed-integer quadratically constrained programming. With the Gurobi optimizer, the mixed-integer linear program can be implemented as an object-oriented problem instead of a matrix-oriented problem which is needed to solve the problem in different programming languages like for example MATLAB. This gives a great advantage with problems that have variables with multiple indices.
After implementation, the results are visualized in a horizontal bar chart in which on the vertical axis the testers are numbered and on the horizontal axis the time since the start of the schedule.
The lots are numbed in the boxes in each row and the horizontal length of each box shows the test time of that lot. Figure 4.1 shows an example result of this visualization. The figure shows the result of an example scheduling problem that consists of six lots on five testers, each lot has four test programs in the order: orange, green, red, blue.
Figure 4.1: An example result to show the result visualisation
4.3 Model performance
Several hypothetical cases are tested with the mixed-integer linear program to evaluated the performance of the program. Test process times are sampled from a Poisson distribution with a mean of 10 hours. The changeover times are one hour when a changeover occurs to the same test program and four hours when a changeover occurs between different test programs. The hypothetical cases are split into two insertion policies: a single insertion policy on which lots are tested once per temperature test and a split insertion policy in which the lots are tested twice per temperature test. The case study is limited to two temperature tests: cold and hot tests.
This means that the single insertion policy has two tests per lot and the split insertion strategy has four tests per lot. Table 4.1 shows the results of the single insertion policy and Table 4.2
shows the results of the split insertions policy.
Table 4.1: Test cases with a single insertion policy Number of
machines Number of
lots Number of
schedule jobs Makespan [h] Computing time [s]
2 2 4 23 1
3 3 6 25 1
3 4 8 33 6
4 5 10 31 30
5 6 12 33 73
Table 4.2: Test cases with a split insertion policy Number of
machines Number of
lots Number of
schedule jobs Makespan [h] Computing time [s]
2 2 8 52 15
3 3 12 52 195
3 4 16 67 6263
4 5 20 60 104352
5 6 24 58 143055
The results of the split insertion policy shows that when increasing the lots and subsequently the number of scheduled jobs above a certain threshold, the computing time blows up. This means that the mixed-integer linear program will not be a suitable method to answer the research questions. In the next chapter, a heuristic approach will be introduced.
NXP Graduation thesis Heuristic method: Genetic Algorithm
5 Heuristic method: Genetic Algorithm
Scheduling problems are known NP-hard problems. Meta-heuristics can be used to find near- optimal solutions for NP-hard problems. In this case, a genetic algorithm is used to solve the scheduling problem at the final test. The first genetic algorithm that is evaluated is a genetic algorithm with a random mutation.
5.1 Algorithm overview
Figure 5.1 shows a flowchart of the genetic algorithm. The first step is to generate an initial population of chromosomes and assess the fitness of the first generation of chromosomes. The next step is to select the chromosomes that will be used for mutation.
Start
Initial population
Fitness assessment
Chromosome selection
Mutation
Fitness assessment
Improvement?
Final solution
Stop no yes
Figure 5.1: Flow chart of the Genetic algorithm
This chromosome will be mutated with a predefined mutation rate. The new chromosome will be assessed on its fitness and stored. This process repeats itself until the next generation has the same number of new chromosomes as the initial generation. With the new generation, the cycle will repeat itself starting with the new selection for mutation. The best solution of each generation will be stored in order to assess the improvement over the generations. After a predefined number of generations that will not have an improvement with each best chromosome the algorithm stops and gives its best solution overall. The next subsection will elaborate on the important processes of the genetic algorithm.
5.2 Chromosome definition
It is important to define the chromosome in a way that it can assess the objective function and that it is compliant with all the scheduling constraints. The chromosome consists of a two by x array. The first column represents the lot numbers and the second column the tester allocation of each lot. Each lot occurs as often as the number of test programs of that lot. This means that if a lot is tested four times on different test programs, the lot number will occur four times in the first column of the chromosome. The lot numbers in the first column are fixed, the only thing that can change in the first column is the sequence of the lot numbers. The second column represents the tester allocation of the lot from the first column. Each position of the second column can take all the values in the set of available testers. Each row of the chromosome is called a genome. Table 5.1 shows an example of the chromosome structure.
Table 5.1: An example chromosome
Lot Testerallocation
1 2
3 1
2 2
2 1
4 1
3 2
4 1
1 2
The figure shows an example with four lots and each lot needs to be tested twice. The tester capacity of this example is two.
5.3 Chromosome fitness assessment
The chromosome needs to be decoded and translated into a schedule in order to assess the fitness of that particular chromosome. When the chromosome is translated into a schedule, key performance parameters can be tracked. The makespan, work in progress (WIP), changeovers, and the number of load boards used can be assessed from each chromosome. These performance parameters can be used to set a complex objective function. For example, an objective function that not only minimizes the makespan but also the WIP. Or an objective function that decreases the changeovers between different test programs and minimizes the number of load boards used.
It is important that during the decoding process all the constraints are taken into consideration.
The most important constraints implementations are elaborated in the next subsections.
NXP Graduation thesis Heuristic method: Genetic Algorithm
Sequence constraint
The sequence constraint states, that each lot needs to be tested in a predefined sequence. This constraint is set by the order in which each lot appears in the chromosome. The fitness function sets the first time a lot appears in the chromosome on test program one and the second time that lot appears in the chromosome on test program two, et cetera. Figure 5.2 shows an example of how this works.
Figure 5.2: Example of a decoded chromosome that shows the sequence constraint
The figure shows one lot that is tested on five test programs, the order in which it is tested is orange, green, red, blue, and pink. The fitness function automatically sets the correct order from the chromosome.
Non-overlapping constraints
It is important that the scheduling algorithm does not schedule lots at the same time on different testers. In order to achieve this, the fitness function tracks the finishing time of the last test job of the lot in question. If the test job is not finished yet the new test job of that lot will be tested after the completion time. A changeover time from switching between tester is also added.
Figure 5.3 shows an example of this constraint. The same lots are tested as the previous example, but now on two testers.
Figure 5.3: Example of a decoded chromosome that shows the workings of the non-overlap constraint
The figure shows that the constraint works correctly and the changeover time between different testers is added as well.
Changeovers times between test programs
Important input parameters are the changeover times between the different test programs. The number of different changeovers depends on the number of test programs each lot needs to complete. If a product has four test programs, the lots can encounter 16 different changeovers.
For this example, the changeovers are classified into two categories: a changeover between the same test program and a changeover between a different test program. The fitness function needs to correctly incorporate the changeover times between the test programs. Figure 5.4 shows an example of three lots that are tested on four test programs, the order of the test programs is:
orange, green, red, blue. In this example, the changeover time between the same test programs is one time unit and between different test programs eight time units.
Figure 5.4: Example of a decoded chromosome that shows the workings of different changeovers
The figure shows that the fitness function correctly applies the correct changeover times between the different test programs.
5.4 Chromosome selection
The chromosomes can be selected with different methods. The two methods that are evaluated are: the roulette wheel selection and the binary tournament selection. The roulette wheel method ranks the generation of chromosomes from the best fitness to the worst fitness. The best chromosome gets the rank that is equal to the total population and the worst chromosome gets the rank of one. The probability of picking a chromosome is shown in Equation 5.1.
Pi= Ri
PN j=1Rj
(5.1) The equation states that the probability of picking the chromosome i (Pi) out of the population(N), is the fraction of the rank of chromosome i (Ri) and the sum of the whole ranked population.
Figure 5.5 shows an example of a population with five chromosomes. The figure shows the rank of each chromosome and the corresponding probability of picking that chromosome. The results are visualized in a circular diagram, in which each chromosome are proportional to their corresponding probability. The method got its name due to this circular representation.
NXP Graduation thesis Heuristic method: Genetic Algorithm
Figure 5.5: An example of a roulette selection with five chromosomes
The second selection method that is evaluated, is the binary tournament selection. This method picks two chromosomes of the population at random and evaluates the fitness values of both chromosomes. The chromosome with the highest fitness value will be used for mutation.
5.5 Chromosome mutation
The next step with the selected chromosomes is to mutate the current chromosomes and generate a new generation of chromosomes. The mutation of the chromosomes is random with a predefined mutation rate. The chromosomes are mutated in pairs, the pairs from the current generation are called parents. With the mutation rate, an integer number of genomes are randomly picked from one of the parents. These genomes are randomly switched with the same number of genomes from the other parent. With this switch, it is taken into account that the genomes of the other parents have the same lot numbers. This condition is necessary otherwise, the scheduling problem would change and some lots would be tested too much. Figure 5.6 shows an example of this mutation. This example has a mutation rate of 0.5, this means that half of the genomes of the parent chromosomes are mutated. The figure shows the three picked genomes of each parent have the same lot numbers.
Figure 5.6: An example of the parent mutation
Figure 5.7 show the newly formed chromosomes from the mutated parents. These new chromo- somes are called children and they will be added to the next generation.
Figure 5.7: The children formed from the mutated parents
The selection and mutation process will continue until the predefined generation size is reached.
After that, the fitness of the new generation will be assessed and the process will repeat until the algorithm stop condition is reached.
5.6 Model implementation
The algorithm is implemented in a python model. The model is simplified in the pseudo-code and shown below
Algorithm 1: Genetic algorithm with random mutation Result: Best test schedule based on the objective function
1 Generate initial population
2 for all chromosomes in initial generation do
3 Fitness assessment
4 end
5 while Fittest chromosome not improved for "x" amount of generations do
6 Pick fittest chromosome of current generation and add to next generation
7 for Size of a generation -1 do
8 Pick 2 chromosomes from current generation with the selection method
9 Mutate the picked chromosome with mutation rate "r"
10 Fitness assessment of mutated chromosome
11 Add mutated chromosome to next generation
12 end
13 end
5.7 Hyperparameter factorial design
The algorithm has several parameters that can be tuned in order to improve the performance of the algorithm. The individual parameters influence the result of the algorithm. So to find the influence of the parameter and to find an indication of its optimal value a factorial design analysis can be done. Often several parameters need to be investigated so in order to keep the number of cases low, only three input values are chosen for each parameter. The three parameters that will be investigated are the generation size, the mutation rate, and the genome selection method.
Table 5.2 show the parameters with their input values.
NXP Graduation thesis Heuristic method: Genetic Algorithm
Table 5.2: Parameters and the input values for the factorial design analysis
Parameter Low value Medium value High value
Generation size 100 genomes 250 genomes 500 genomes
Mutation rate 0.2 0.5 0.8
Chromosome selection Ranked roulette wheel Binary tournament
With these eight input values, the total cases that need to be evaluated are 18. These 18 input parameters are evaluated with the same case. The evaluate the effect of input parameters, a difficult scheduling problem is chosen. The scheduling problem consists of 40 jobs that need to be scheduled on six machines, in this case, the makespan is the optimizer.
The input cases are compared with the average of all the cases. This shows the effect of each input parameter relative to the mean. Figure 5.8 shows the result in a horizontal bar chart.
The results show that a higher mutation rate and a generation size improves the scheduling algorithm relative to the other evaluated cases. As for the chromosome selection method, the binary tournament shows better performance relative to the ranked roulette wheel method.
Figure 5.8: The results of the factorial design analysis
5.8 Model Performance
To assess and validate the genetic algorithm model, the results of the mixed-integer linear program (MILP) are compared to the genetic algorithm (GA). The same cases that are used in the last chapter are used again to validate the genetic algorithm. Table 5.3 shows a comparison between the two methods. The table shows a clear improvement in computing time. The genetic algorithm is much faster in solving the problems.
Table 5.3: The GA performance compared with the MILP performance Number of
machines [h] Number
of lots [h] Number of
schedule jobs [h] Makespan
MILP [h] Computing
time MILP [s] Makespan
GA [h] Computing time GA[s]
2 2 4 52 15 52 1
3 3 12 52 195 52 1
3 4 16 67 6263 67 16
4 5 20 60 104352 61 15
5 6 24 58 143055 59 20
The genetic algorithm has no issues with the low complexity scheduling problems. With the more complex problems, more than 5 lots with 4 sequenced test programs, it finds very good solutions near the optimum makespan. Figure 5.9 shows a comparison between the optimal solution of the mixed-integer linear program and the near-optimal solution of the genetic algorithm.
(a) MILP optimum solution (b) GA near optimum solution
Figure 5.9: A comparison between the optimal solution of the MILP and the near optimal solution of the GA
5.9 Model limitations
The basic genetic algorithm has some limitations. When increasing the complexity of the problem, the basic genetic algorithm has trouble with consistently giving good solutions. For example a scheduling case with five testers and 15 lots that are tested on four test programs. This case has changeover times of four hours between different test programs and a changeover of one hour between the same test programs. This case is optimized for the makespan of all test jobs and the results are shown in the table below.
Table 5.4: The results of the complex case
Model Number of repetitions Best
makespan[h] Mean
makespan [h] Variance
makespan [h] Mean
computing time [s]
Basic GA 25 187.0 190.5 5.06 264
The table shows that there is quite a big difference in variance in the results. This means that the model has trouble consistently giving good solutions. Figure 5.10 shows the schedule of the best result. This figure shows that there are several lots that are "out of place". For example lot eight on tester five on the blue test program is clearly out of place.
NXP Graduation thesis Heuristic method: Genetic Algorithm
Figure 5.10: The best solution of the genetic algorithm, the black circles show lots that are out of place
With this result being the best of the 25 runs, the model clearly needs to be improved. It is not only desired that the model give consistent solutions, the objective function needs to be extended to include more performance optimizers. The next chapter will improve the current model to achieve these goals.
6 Heuristic method: Genetic algorithm with local search algo- rithms
This section elaborates on the improved genetic algorithm with the implementation of local search algorithms. A common problem with meta-heuristics, is that the solution can be stuck at a local optimum. Local search algorithms can be introduced in order to solve this problem. Local search algorithms make small changes to the solution based on certain performance parameters in order to "escape" the local optimum and find a solution near or on the global optimum. The workings of the local search algorithms will be explained and the model performance will be assessed.
6.1 Algorithm overview
When the number of scheduled jobs increases and the objective function becomes more complex, the random mutation will not be enough to consistently give accurate results. The algorithm needs to be modified to give more accurate and consistent results. To achieve these goals, local search algorithms are added to the genetic algorithm. Figure 6.1 shows the modified genetic algorithm. It starts similarly to the previous algorithm with the generation of the initial population and the fitness assessment of that population. This algorithm is a hybrid solution of both the random mutations and local search algorithms. The genetic algorithm starts with mutating the chromosomes randomly and after a predetermined generation number (the "switch point") the algorithm switches to the local search algorithms.
NXP Graduation thesis Heuristic method: Genetic algorithm with local search algorithms
Start
Initial population
Fitness assessment
Generation?
Chromosome selection
Mutation
Fitness assessment
Improvement?
Final solution
Stop
Chromosome selection
Local search algorithms
Fitness assessment Below switch point
no yes
Above switch point
Several iterations
Figure 6.1: Flow chart of the Genetic algorithm
The local search method starts with a chromosome selection. The same two chromosome selection methods will be evaluated for local search algorithms; the ranked roulette wheel selection method and the binary tournament selection method. The next step is to pick a local search algorithm.
The genetic algorithm picks one of five local search algorithms for its mutation. Each function
mutates the chromosome in a different way. Each chromosome iterates several times between different functions. The number of iterations is also a hyperparameter that can be tuned based on its performance. Before iterating the chromosome is assessed on its fitness and keeps the mutation if the mutation improves the fitness of the chromosome. If the mutation does not give an improvement in fitness, the original chromosome is kept for the iteration. After the predefined number of iterations, the chromosome is stored into the next generation.
6.2 Objective function
Up until this point, the genetic algorithm only optimized the makespan of all the scheduled test jobs. For the case study that investigates different insertion policy, it is necessary to take extra factors into account. The new objective function consists of four components:
1. The makespan minimization
2. The minimization of changeovers between different test programs 3. The minimization of the load boards used in the test schedule 4. The minimization of the lead time of individual lots
The makespan minimizes the total time necessary to finish all the test jobs in the schedule. The makespan is defined as the finishing time of the last test job in the schedule. The changeovers between different test programs cause the biggest downtime during testing. It is preferable to decrease the changeovers between different test programs to improve tester efficiency. The load boards used during testing are test program specific and they very expensive. It will be preferable to use less hardware to decrease test expenses. Deceasing the lead time of the lots is preferable to control the work in process (WIP) of the test floor. The lead time is defined as the time between the time a lot starts its first test program and the finishing time of its last test program.
The WIP is defined as the number of lots that are in the system. This means that when a lot starts with its first test job, the WIP increases by one and when a job is finished with its last test job, the WIP decreases by one. This fourth component can control the importance of WIP management of the test schedule.
With these four components, the fitness function is altered to incorporate the new objective function. The four components are translated to a single fitness value, in this case, the costs in us dollars. The makespan will be multiplied by the costs of running a tester and by the number of testers used. Each changeover between a different test program will be penalized with a penalty cost. The fitness function will track the number of load boards used in the schedule and multiplies each load board type with the depreciation and maintenance costs of that particular load board type. The last component is the lead time minimization of the individual lots. Due to the size difference in lots, this component is defined as the fraction between the lead time and the net processing time. This is necessary otherwise the comparison between small lot sizes and big lot sizes would not be fair. The sum of these fractions is multiplied with penalty costs. The cost components are input parameters and can be set by the user of the scheduling program.
A user might want to increase the tester efficiency by heavily penalizing changeovers or a user might want to decrease the WIP by penalizing the lot lead times.
6.3 Local search algorithms
The improved genetic algorithm includes five local search algorithms. This section elaborates in more detail on the different algorithms. Three of the five algorithms are "smart" mutations that mutate the genomes based on the fitness properties of the chromosome. The other two are random mutations to make sure every solution can be reached by the genetic algorithm.
NXP Graduation thesis Heuristic method: Genetic algorithm with local search algorithms
Randomly switches lots from one tester to another tester
The first local search algorithm switches the lots from one tester to another tester. The function picks a lot randomly and switches the lot with the same lot on another tester.
Figure 6.2: An example of the a random lot mutation
Figure 6.2 shows an example of the function. The function picks lot four that is scheduled to be tested on tester two. The function picks the same lot on another tester and switches the scheduled lots.
Randomly switches the lots within the same tester
This local search algorithm is very similar to the one discussed in the section above. This function picks a tester and switches different lots on that tester.
Figure 6.3: An example of a mutation on the same tester
Figure 6.3 shows an example of the local search algorithm that switches on the same tester. The example picks lot four that is scheduled on tester two and switches the lot with lot number two on the same tester.
Switches lots from testers with the most changeovers to another tester
This local search algorithm is the first of the three "smart" algorithms. The fitness function collects performance parameters from each of the chromosomes. One of these parameters is the changeover information of each of the testers. The fitness function collects the number of changeovers between different test programs because these are the most time consuming due
to hardware changes. The local search algorithm picks the tester with the most changeovers between different test programs and switches a lot on that tester with a different tester. This results in a total decrease of changeovers between different test programs, which decreases the total makespan and increases the tester efficiency. Figure 6.4 shows how the changeovers are being counted. The figure shows that when the next lot scheduled on the tester is not the same test program, the changeover is being counted.
Figure 6.4: A schedule with the changeovers outlined, tester four has the most changeover between different test programs
Switches the lots with the highest lead time/net process time fraction to another tester or within the same tester
This local search algorithm looks at the fraction between the lead time and the net process time of the individual lots. The lead time is defined as the time from the beginning of the first time a lot is tested until the time the lot is finished at its last test program. The net process time is defined as the sum of all the individual test program times of a lot. Reducing this fraction has several benefits. The most obvious benefit is that it would reduce the lead time which results in a decrease of WIP. The fitness function stores the lot number with the highest lead/net process time fraction. The local search algorithm uses this lot number to determine the last test job of that lot. This test job will find the lot with the highest fraction and places the lot earlier in the test schedule on the same tester at random.
Figure 6.5 shows a simple example of this function. The test schedule consists of four lots with two test programs each. The fractions are being tracked in the fitness assessment and the lot with the highest fraction is used as an input value for the local search algorithm. In Figure 6.5 the lot with the highest fraction is lot number one, the last test job of this lot is picked from the schedule and placed to be tested earlier on that tester.
NXP Graduation thesis Heuristic method: Genetic algorithm with local search algorithms
Figure 6.5: Caption
The effect on the chromosome from the example case is shown in Figure 6.6. The genome picked from its original place in the chromosome and moved to a location earlier in the chromosome.
The genome itself does not change, only the location of the genome in the chromosome.
Figure 6.6: The chromosome mutation of the algorithm example
Switches the lot that is most "out of place"
The last local search algorithm mutates lots that are "out of place". The function looks at the test programs that are being tested on each of the testers and calculates the percentage of each test program relative to all the tested programs on that tester. A lot will be picked that is being tested on the test program with the lowest percentage, this lot will be randomly moved to another tester.
Figure 6.7 shows an example of this local search algorithm. The red test program has the lowest percentage of tests relative to all the tests on testers one. This means that a lot will be picked that is doing a red test program test on tester one and that lot will be switched to another tester.
In this case, there is only one test job that is being tested on that test program and that is lot number one. That test job will be moved to another tester.
Figure 6.7: The figure shows which test job is picked by the local search algorithm
NXP Graduation thesis Heuristic method: Genetic algorithm with local search algorithms
6.4 Model implementation
The algorithm is implemented in a python model. The model is simplified in the pseudo-code and shown below.
Algorithm 2: Genetic algorithm with local search
Result: Best test schedule based on the objective function
1 Generate initial population
2 for all chromosomes in initial generation do
3 Fitness assessment
4 end
5 while Fittest chromosome not improved for "x" amount of generations do
6 Keep the fittest chromosome of the current generation for the next generation
7 if Generation number is below the switch point then
8 for Size of a generation -1 do
9 Pick 2 chromosomes randomly from the current generation
10 if Chromosome 1 has a higher fitness value than chromosome 2 then
11 Pick chromosome 1 for mutation
12 else
13 pick chromosome 2 for mutation
14 end
15 Mutate the picked chromosome with mutation rate "r"
16 Fitness assessment of mutated chromosome
17 Add mutated chromosome to next generation
18 end
19 else
20 Rank current generation on fitness value
21 for Size of a generation -1 do
22 Pick chromosome from the current generation
23 for Local search iterations do
24 Randomly pick one of the local search mutations
25 Fitness assessment
26 if Chromosome fitness is improved then
27 Replace picked chromosome with improved chromosome
28 else
29 Keep the picked chromosome for next local search iteration
30 end
31 end
32 end
33 end
34 end
6.5 Hyperparameter tuning
The algorithm has several parameters that can be tuned in order to improve the performance of the algorithm. The individual parameters influence the result of the algorithm. So to find the influence of the parameter and to find an indication of its optimal value a factorial design analysis can be done. With the implantation of the local search algorithms come extra algorithm parameters. Table 6.1 shows the parameters and the values that are being assessed in the factorial design analysis.
