Computational costs - Evaluation criteria

Proposed algorithms

CHAPTER 8. EXPERIMENTAL RESULTS

8.2 Evaluation criteria

8.2.3 Computational costs

Computational costs can be expressed in total number iterations needed to need the final result.

It is true that computation times can fluctuate between iterations, but such small differences can safely be neglected since in each iteration the FEA computations are the bottleneck.

CHAPTER 8. EXPERIMENTAL RESULTS

0.0001 0.001 0.01 0.1 1

Von Mises strain (uStrain)

average Von Mises strain (uStrain)

iteration number

stage 0 stage 1 stage 2 price

(d) Example strain/price versus iteration Figure 8.3: Example results

Convergence speed is a more detailed variant of the computational cost presentation. By plot-ting strain or fixture costs per iteration, one can see how consistent optimization algorithms are:

e.g. the differences between every two iterations are consistent, or the main part of optimization is done in the first few iterations. An example of such plot is shown in Figure8.3d.

In order to compare convergence speed, a well-defined metric is needed. Per benchmark run, the first n iterations are taken of which the last one has a score that is less than 105% of the score of the best found fixture in that benchmark run. Therefore, benchmark runs should include sufficient iterations showing a lack of optimization progress to be a trustworthy answer. In the example shown in Figure8.3d, the 37th iteration. The 25th iteration is the first iteration with a score less than 105% of the 37th iteration, hence the convergence speed is defined to be equal to 25 iterations.

In case of benchmarking individual step algorithms, this convergence speed number suffices. In case of benchmarking a controller algorithm, it is also important to know which step algorithms are used to reach this convergence point. This is done by a plot indicating which step algorithms were executed in these first n iterations. These algorithms are denoted to be dominant step algorithms.

8.3 Initialization strategy

This section describes the results of applying the strategy described in Section6.1to the benchmark set described in Section8.1.

Figure 8.4 compares the reference fixture with the fixture generated by the initialization strategy. The difference is clear: the generated fixture has practically no strain. The number of support probes has been reduced, while the number of push fingers has been increased. Since push fingers are much cheaper than support probes, the total fixture costs are reduced.

Applying the initialization strategy to the P1SMD board gives less good results, as shown in Figure 8.5. Because of distance constraints specifying a minimum distance between push fingers and DUT components, only two push fingers are placed on the panel, while five push fingers are placed on the surrounding PCB. The resulting distance between test probes and push fingers cause large deformation that is only be limited via the breakaway tabs. Hence, strain is concentrated around these breakaway tabs.

Applying the initialization strategy to the THCOBO board gives mixed results, as shown in

(a) Reference: fixture (b) Reference: strain

CHAPTER 8. EXPERIMENTAL RESULTS

(a) Reference: fixture (b) Reference: strain

Figure8.6. At the right side of the board, more push fingers are placed, resulting in lower strain.

At the left side however, only a few contacts are placed, resulting in a sharp bending near the breakaway tab at the centre left part of the board.

Applying the initialization strategy to the TOTSB board gives worse results, as shown in Figure 8.7. Per panel, only a few contacts are placed, causing the individual panels to rotate around the breakaway tabs, increasing strain at those locations. The top and bottom panel still have some support, but especially the centre panel has a big twist with respect to the surrounding PCB.

Applying the initialization strategy to the LAN2RF board gives good results, as shown in Figure8.8. Contacts are placed at the almost perfect positions, reducing strain by approximately 50%.

In short, the initialization strategy gives mixed results. On two boards, strain is reduced, on one board strain is approximately the same, and on two boards the DUT strain is much worse.

Hence, it is up to the various optimization step algorithms to further improve these results.

(a) Reference: fixture (b) Reference: strain

CHAPTER 8. EXPERIMENTAL RESULTS

(a) Reference: fixture (b) Reference: strain

8.4 Slope based step

This section describes the results of applying the algorithm described in Section 6.3 and imple-mented according to Section 7.3 to the benchmark set described in Section 8.1 As described in Section7.3, this algorithm has precisely one variable, aggressiveness. Therefore, the algorithm is tested using on various benchmark boards as well as with different aggressiveness values.

Applying the algorithm to the FTS board using an aggressiveness of 50.0 gives divergent results at first. However, after ten iterations, the system starts to converge. From iteration 40, the slope step algorithm does no longer improve. This is shown in Figure8.9.

Using an aggressiveness of 40.0, the algorithm consistently gives convergent results. Figure8.10 shows that the algorithm converges after approximately 160 iterations.

Figure 8.11provides more insight in the converging process. By design, the DUT is almost symmetric. This symmetry causes the algorithm to not further improve from iteration 25 onward, but the small design irregularity causes the top centre push finger to move a bit to the left, allowing a reordering of the reference 3 × 3 push finger configuration to the craggy 4 × 2 + 1 configuration.

Another thing to note is that the push fingers and support probes keep matching each other.

Testing the algorithm on the P1SMD board using an aggressiveness of 500.0 gives convergent results. New generated fixtures are sometimes worse, sometimes better than the fixture used as a seed. The system does not converge. This is shown in Figure8.12.

The lack of convergence can be explained by taking a look at the generated fixture designs, shown in Figure 8.13. The different push fingers and support probes on the panel move slowly towards a local optimum position, while the different push fingers and support probes on the empty PCB surrounding the panel overshoot each iteration, switching positions. This can best be seen at the top left of the board. At iteration 0, the push finger and support probe are aligned.

At iteration 1, the push finger moves down, the support probe moves up. At iteration 2, these two contacts switch, which is then repeated at iteration 3.

Using an aggressiveness of 250.0, the algorithm gives convergent results. Figure 8.14 shows that the algorithm converges until iteration 15, switches to worse design, continues to converge, then stabilizes around iteration 75.

Figure 8.15 compares the generated fixtures with the resulting strain maps. The reference fixture has five push fingers on the panel and five push fingers on the surrounding PCB. Nine support probes support the panel, while five support probes support the surrounding PCB. Twenty four test probes exert forces on the panel, causing strain to concentrate on the panel push fingers.

The slope based step algorithm causes two of the outer push fingers to move onto the panel, while the different support probes move towards the closest by push finger. After convergence, some push fingers have multiple support probes close by.

Based on these benchmarks, the slope step algorithm seems to be a very efficient algorithm.

The algorithm is able to converge to a local optimum in only a few iterations. Such local optimum

average Von Mises strain (uStrain)

iteration number

stage 0 stage 1 stage 2

Figure 8.9: Slope step; FTS; aggressiveness 50.0; strain versus iteration

CHAPTER 8. EXPERIMENTAL RESULTS

0 2000 4000 6000 8000 10000 12000

1 2 4 8 16 32 64 128 256

average Von Mises strain (uStrain)

iteration number

stage 0 stage 1 stage 2

Figure 8.10: Slope step; FTS; aggressiveness 40.0; strain versus iteration

is then often improved to an even better local optimum in about one hundred iterations.

The algorithm has also some drawbacks. First, the aggressiveness needs to be controlled. If this value is too low, convergence will be slow. If this value is too high, the algorithm overshoots, giving diverging results. Hence, this algorithm needs an optimization controller that is able to choose a fitting aggressiveness value. Section9.1 describes a potential solution to this problem.

Second, the algorithm can only move contacts, it cannot add or remove them. In the FTS example, adding one extra push finger at the right side might have improved the optimization process. In the P1SMD example, multiple support probes converging towards the same push finger could have been replaced by a single support probe.

(a) Iteration 0: fixture (b) Iteration 0: strain

(e) Iteration 4: fixture (f) Iteration 4: strain

(g) Iteration 16: fixture (h) Iteration 16: strain

(i) Iteration 64: fixture (j) Iteration 64: strain

(k) Iteration 256: fixture (l) Iteration 256: strain

Figure 8.11: Slope step; FTS; aggressiveness 40.0; first stage; fixture versus strain map

CHAPTER 8. EXPERIMENTAL RESULTS

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

1 2 4 8 16 32 64

average Von Mises strain (uStrain)

iteration number

stage 0 stage 1 stage 2

Figure 8.12: Slope step; P1SMD; aggressiveness 500.0; strain versus iteration

(a) Iteration 0 (b) Iteration 1 (c) Iteration 2 (d) Iteration 3 Figure 8.13: Slope step; P1SMD; aggressiveness 500.0; generated fixtures

0 500 1000 1500 2000 2500 3000 3500

1 2 4 8 16 32 64 128

averae Von Mises strain (uStrain)

iteration number

stage 0 stage 1 stage 2

Figure 8.14: Slope step; P1SMD; aggressiveness 250.0; strain versus iteration

(a) Iteration 0: fixture (b) Iteration 0: strain

(e) Iteration 4: fixture (f) Iteration 4: strain

(g) Iteration 16: fixture (h) Iteration 16: strain

(i) Iteration 64: fixture (j) Iteration 64: strain

Figure 8.15: Slope step; P1SMD; aggressiveness 250.0; first stage; fixture versus strain map

CHAPTER 8. EXPERIMENTAL RESULTS

8.5 Force based step

This section describes the results of applying the probability and boundary force step algorithms described in Section6.4and implemented according to Section7.4to the benchmark set described in Section 8.1. The quantity version of the algorithm is not benchmarked. Since that version needs an controller algorithm which adjusts the parameters of the step algorithm per iteration, such benchmarks would mainly test the quality of the controller algorithm, instead of the quality of the step algorithm.

8.5.1 Boundary version

The boundary force step algorithm has precisely one adjustable variable, push finger force goal.

Hence, this section will not only describe the algorithm being tested against the different bench-mark DUTs, but also using different force goal values.

Using the FTS board, the algorithm is repeatedly applied, each time using a different force goal. Figure 8.16shows the average strain per iteration. Furthermore, the fixture cost relative to the reference fixture is shown. Using a force goal of 4500mN, the results converge after only one iteration. However, it does not stabilize: each iteration some push fingers are removed and some are duplicated. The same behaviour can be observed when using a force goal of 3000mN and 1500mN: the results converge after a couple of iterations, after which ‘shaky’ behaviour is shown.

Figure 8.17 gives a more in-depth look at the algorithm behaviour when using a force goal of 1500mN. At the first stage, each test probe exerts 1500mN force on the DUT, resulting in the generated fixtures converging towards approximately 30 push fingers, slightly more than test probes and support probes present. However, the algorithm is not capable of placing these different push fingers directly above the different probes. If the different placed push fingers would be shifted to these probe positions, all board strain would be approximately zero, while the number of push fingers would be reduced as well.

Similar behaviour can be observed when benchmarking the P1SMD board. Again, force goals of 4500mN, 3000mN and 1500mN are used. Convergence is shown in Figure8.18. Using a force goal of 4500mN, results converge after a single iteration. Using a force goal of 3000mN, results converge after 12 iterations, but showing a second improvement after the 50th iteration. Using a force goal of 1500mN, results converge after 16 iterations, after which results still differ a bit, but do not show major strain differences.

Figure 8.19shows a more in-depth look on the converging process. During the first iteration, most of the support probes are removed, leaving a minimum number of four support probes. Note that because no force recalculations are done between individual removals, the result is a fixture that would be unstable during the resting stage. In practice, when applying the step algorithm to a complete PCB instead of a single panel, the generated fixture would be stable. However, a controller algorithm should not accept this generated fixture, but for example apply instead several iterations of the quantity force step algorithm.

During the second iteration, the push fingers that no longer have any support probe under them are removed as well. Furthermore, from this iteration onwards, the algorithm tries to add extra push finger near the push fingers on the panel. However, this is not always possible because of DUT component interference. As a result, no extra push fingers are added, or a push finger is added on the surrouding empty PCB, only to be removed again in the next iteration. This behaviour is shown in iterations 16, 32 and 64.

In conclusion: just like with the FTS board, the final result is quite good, but would benefit from small adjustments to the positions of the different contacts.

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

Figure 8.16: Boundary force step; FTS; strain/price versus iteration

CHAPTER 8. EXPERIMENTAL RESULTS

(a) Iteration 0: fixture (b) Iteration 0: strain

(e) Iteration 4: fixture (f) Iteration 4: strain

(g) Iteration 16: fixture (h) Iteration 16: strain

(i) Iteration 64: fixture (j) Iteration 64: strain

Figure 8.17: Boundary force step; FTS; force goal 1500mN; first stage; fixture versus strain map

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

Figure 8.18: Boundary force step; P1SMD; strain/price versus iteration

CHAPTER 8. EXPERIMENTAL RESULTS

(a) Iteration 0: fixture (b) Iteration 0: strain

(e) Iteration 4: fixture (f) Iteration 4: strain

(g) Iteration 16: fixture (h) Iteration 16: strain

(i) Iteration 64: fixture (j) Iteration 64: strain

Figure 8.19: Boundary force step; P1SMD; force goal 1500mN; first stage; fixture versus strain

8.5.2 Probability version

The probability force step algorithm has three adjustable variables: test stage, aggressiveness, and push finger force goal. To limit the total number of results to discuss, only the first test stage is considered in this Subsection. This results in two parameters to test. First, the influence of different aggressiveness parameter values are tested with a fixed force goal. The force goal value chosen is the optimum found in Section 8.5.1. Second, the influence of different force goal parameter values is tested using a fixed aggressiveness parameter. Here, the value used will be the optimum aggressiveness found in the first part.

Figure8.20shows the results of repeatedly applying the probability force step algorithm to the FTS board using a push finger force goal of 1500mN. The influence of the aggressiveness parameter is clearly visible: the less aggression, the slower the convergence. Furthermore, a high aggression gives more diverse results.

Applying the same benchmark to the P1SMD board gives comparable results as shown in Figure8.21. This time, convergence is less visible: strain is hardly reduced over time.

Figure 8.22 shows the influence of using a different push finger force goal. When comparing to Figures 8.20b and 8.21b, it is clearly visible that increasing the force goal from 1500mN to 3000mN results in much worse strain results.

Comparing the boundary force step algorithm with the probability force step algorithm gives some interesting differences. The boundary force step algorithm produces less divergent results.

When simply repeating the algorithm execution multiple times, this is unwelcome behaviour.

However, when used with a proper optimization controller, such divergence can be used as an advantage. Hence, choosing a winner should be left to Section8.8.

CHAPTER 8. EXPERIMENTAL RESULTS

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

Figure 8.20: Probability force step; FTS; force goal 1500mN; strain/price versus iteration

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

Figure 8.21: Probability force step; P1SMD; force goal 1500mN; strain/price versus iteration

CHAPTER 8. EXPERIMENTAL RESULTS

average Von Mises strain (uStrain)

iteration number

average Von Mises strain (uStrain)

iteration number

Figure 8.22: Probability force step; force goal 3000mN; strain/price versus iteration

8.6 Displacement based step

This section describes the results of applying the displacement step algorithm described in Sec-tion6.5and implemented according to Section7.5to the benchmark set described in Section8.1.

As described in Section7.5, this algorithm has one adjustable parameter per test stage, the number of contacts to add.

As a first benchmark, this step algorithm is repeatedly applied to the FTS board. Since this board has no much weight, focusing on the resting stage has hardly any use. Instead, each iteration one contact will be added based on the deformation of the first stage. Figure8.23shows per iteration the average strain per test stage and costs of the generated fixture relative to the reference fixture.

One can see that the generated fixtures converge after approximately 40 iterations. Figure8.25 shows why no improvements are made after this iteration: there is no free space above any probe to place a push finger on. Instead, push fingers are added at the closest by free space, resulting in a push finger explosion. Just like the force step algorithm, contacts which are placed just off the perfect position block further improvement near that position.

Applying the displacement step algorithm to the P1SMD board gives comparable algorithm characteristics. Or to be more precise, even worse results. Figure 8.24 shows the convergence results, Figure 8.26 the corresponding strain maps. The high DUT component density on the panel causes push fingers to be placed on the surrounding PCB, producing another push finger explosion.

To conclude, the displacement based step algorithm is not very useful if used in a stand-alone manner.

average Von Mises strain (uStrain)

iteration number

stage 0 stage 1 stage 2 price

Figure 8.23: Displacement step; FTS; strain/price versus iteration

CHAPTER 8. EXPERIMENTAL RESULTS

50%

100%

150%

200%

250%

300%

350%

0 500 1000 1500 2000 2500 3000 3500

1 2 4 8 16 32 64

price (relative)

average Von Mises strain (uStrain)

iteration number

stage 0 stage 1 stage 2 price

Figure 8.24: Displacement step; P1SMD; strain/price versus iteration

(a) Iteration 0: fixture (b) Iteration 0: strain

(e) Iteration 4: fixture (f) Iteration 4: strain

(g) Iteration 16: fixture (h) Iteration 16: strain

(i) Iteration 64: fixture (j) Iteration 64: strain Figure 8.25: Displacement step; FTS; first stage; fixture versus strain map

CHAPTER 8. EXPERIMENTAL RESULTS

(a) Iteration 0: fixture (b) Iteration 0: strain

(e) Iteration 4: fixture (f) Iteration 4: strain

(g) Iteration 16: fixture (h) Iteration 16: strain

(i) Iteration 64: fixture (j) Iteration 64: strain

Figure 8.26: Displacement step; P1SMD; first stage; fixture versus strain map

8.7 Strain based step

This section describes the results of applying the strain based step algorithm described in Sec-tion6.6and implemented according to Section7.6to the benchmark set described in Section8.1.

As described in Section7.6, this algorithm has one adjustable parameter per test stage, the number of contacts to add.

As a first benchmark, this step algorithm is repeatedly applied to the FTS board. Each iteration one contact will be added based on the deformation of the first stage. Figure8.27shows per iteration the average strain per test stage and costs of the generated fixture relative to the reference fixture.

One can see that the generated fixtures converge after approximately 105 iterations. At the same time, the total fixture costs are 3.5 times higher than the reference fixture. Figure8.29shows the convergence process. Each iteration, push fingers or support probes are added near other push fingers and support probes. Compared to the displacement based step algorithm discussed in the previous section, this algorithm gives worse results on all three evaluation criteria: DUT strain, fixture cost and computation time.

Applying the displacement step algorithm to the P1SMD board gives comparable algorithm characteristics. Figure 8.28 shows the convergence results, Figure8.30 the corresponding strain maps. Again, the fixture price explodes, but this time DUT strain does not improve not that

In document Eindhoven University of Technology MASTER Test fixture optimization Oudman, F.H. (pagina 49-76)