Optimization in Electrical Engineering

1. Optimization in Electrical Engineering Professor David Thiel Centre for Wireless Monitoring and Applications Griffith University, Brisbane Australia

2. My Interest? • Interpretation of geophysical data when there are more knowns than unknowns. • Optimizing antennas for maximum gain and minimal materials. • Using lumped impedance elements to achieve dual frequency operations. • Working with improved performance of switched parasitic antennas

3. Purpose • To make attendees familiar with the process of optimization in electrical engineering. • To introduce the basic elements of an optimisation routine including: • Initial parameters • Forward modelling • Cost function • Guided parameter refinement • Program termination

4. Why Optimisation? It is no longer acceptable to submit a new design to a journal or conference without logical support for the idea. One can gain support for the new design using theory or optimisation.

5. Workshop Strategy • I will not include many equations. Generally these equations are relatively simple, can be found in books, and can be derived using simple geometry. • In some places the equations can’t be avoided and so have been included. • Optimisation is usually conducted in multi-dimensional space and so it quite difficult to visualize. We will use a 2D model so we can visualize!

6. Optimization applications • Standard curve fitting procedures require more data points than the number of unknowns to find. • For example, linear correlation defines the slope and the intercept from a set of data points. (see the Research Methods workshop) • Optimization allows the determination of more unknowns that there are data points.

7. Optimization is curve fitting/pattern recognition • The optimization routine generates improved “guesses” to minimize the cost function – for curve fitting, the cost function is the RMS error. • In optimization, the Cost function is minimized – the RMS error is minimized.

8. Physical structureparameters Change physical parameters Electromagnetic Solver Calculate Cost Function Needs improvement Compare End Optimization Routine Logic

9. Physical Structure • Assume we wish to optimize nparameters in the physical structure. All other parameters remain fixed. • Set the limits on each parameter (maxima and minima) • Set minimum step size (manufacturing tolerances) • Guess the first structure (by random selection?).

10. Electromagnetic Solver • Open boundary (eg Method of Moments) or closed boundary (eg FEM, FDTD) • Set segment/pixel/voxel size • Pre-processing requirements? • Post-processing requirements? (eg time to frequency domain?) • This is often the rate determining step in an optimization routine. Open boundary methods are usually more efficient compared to closed boundary methods.

11. Define the Cost Function • What are the most important parameters in the final design? • For antennas it might be: • Gain • Front to back ratio • Efficiency • Input impedance • Weight • Wind resistance • Environmental impact

12. Define the Cost Function • The parameters in the cost function can be weighted to give the correct emphasis in the final outcome. • The optimization routine is designed to minimize the cost function. • It is the inverse of a “Figure of merit”.

13. Compare • Every updated cost function (often called the next generation or the next iteration) is compared to the last. • If the design has improved (lower cost function value), continue changing the design in the same direction. • If the design is worse (higher cost function value), change the design in a different direction. • If no significant change, then terminate the routine.

14. Change the physical structure • There are many methods of doing this. Some are guided and others are unguided (random) • We will discuss: • Sequential uniform sampling • The Monte Carlo Method • The Simplex Method • The Method of Steepest Descent (the Gradient Method) • The Genetic Algorithm • Simulated Annealing

15. Method 1 Sequential Uniform Sampling (SUS)

16. Sequential uniform sampling • Calculate every possible combination of every parameter (n) through every increment specified. • Chose the design with the lowest value of the cost function. • A plot of the data will give an indication of the robustness of the solution. • The best solution can be improved by using finer increments and smaller range values.

17. Sensitivity Analysis

18. SUS: Comments • The best solution can be improved by using finer increments and smaller range values. • The results give an idea of which design parameters are critical and which are not so critical.

19. SUS: Advantages • The cost function is never trapped in a local minima. • There is no random element. There is no need to repeat the same trial as you will get the same answer.

20. SUS: Problems • This is an unguided optimization process. • This is extremely computationally intensive.

21. Method 2 Monte Carlo (MC) (Random Sampling)

22. Monte Carlo • Randomly select 0.1% to 1% designs (from all possible designs) using randomly selected values for each parameter. • Calculate the cost function for each design. • Select that design with the lowest cost function as the final solution • A plot of the data will give an indication of the robustness of the solution. • The best solution can be improved by using finer increments and smaller range values.

23. Sensitivity Analysis

24. Monte Carlo: Comments • The best solution can be improved by using finer increments and smaller range values.

25. Monte Carlo: Advantages • This is less computationally intensive compared to SUS. • The solution is not trapped in a local minimum.

26. Monte Carlo: Problems • It is more computationally intensive compared to many guided optimisation methods. • It is likely that every run will result in a different solution. You never know if/when you have reached the best solution.

27. Method 3 The Simplex Method

28. Simplex Method • Randomly select the first design and calculate the cost function. • Increment one parameter and recalculate the cost function. • Do this for every parameter in the design. • These cost function values form a “simplex” in n dimensional space.

29. Simplex Method • Calculate the centre of mass (centroid) of the simplex. • Draw a line from the high valued cost function, through the centre of mass. • The new design lies along this line and is the mirrored point through the opposite surface (n-1 dimensions) defined by the other designs – called reflection.

30. Simplex Method • If the cost function is lower than the highest cost function, replace this new point in the simplex. • If the cost function is much lower than the highest cost function, increase the step size and replace this new point in the simplex – called expansion. • Repeat the calculations until no further improvement is gained.

31. Simplex Method • If the cost function is higher than the highest cost function, reduce the step size and replace this new point in the simplex – called contraction. • Repeat the calculations until no further improvement is gained by any of these three operations.

32. expansion reflection centroid contraction Worst Design in simplex 2D Simplex Method • Each circle represents a design. The triangle simplex is formed by one incremental step in the two parameters. The next simplex has the previous two designs and a new design.

33. Simplex Method: Comments • To reduce falling into a local minima, repeat the algorithm a number of times with a new randomly selected design.

34. Simplex Method: Advantages • It is a guided algorithm – the cost function always decreases until the final solution is reached.

35. Simplex Method: Problems • This method is prone to getting trapped in “local” cost function minima. Continuing with the algorithm does not improve the solution. • At a nD saddle point of the cost function surface, the simplex routine may simply repeat every second iteration. • The method is not very efficient in locating the minimum cost function value.

36. Method 4 The Gradient Method (Method of Steepest Descent)

37. Gradient Method • Calculate the simplex. • Calculate the vector gradient of the cost function and use this value to locate the new design point. • Repeat until no improvement is observed. • The “distance” of the step along the line of maximum descent is related to absolute value of the gradient.

38. 2D Gradient Method • The new design is defined along the gradient line and the magnitude of the gradient. Gradient line New point Worst Design in simplex

39. Gradient Method: Comments • To reduce falling into a local minima, repeat the algorithm a number of times with a new randomly selected design.

40. Gradient Method: Advantages • This is more efficient compared to the simplex method as all cost function values in the simplex are used to calculate the direction. This means all parameters are changed at every iteration.

41. Gradient Method: Problems • It can still be trapped in local minima. • It can still be trapped in a saddle point.

42. Method 5 The Genetic Algorithm (GA) (Modelled on Darwin’s theory of natural selection - Survival of the fittest)

43. Genetic Algorithm • Create a binary number “chromosome” to represent one design. • Every parameter is represented by a binary number “gene”. • The chromosome is a string of binary genes.

44. Genetic Algorithm • A population of chromosomes is “created” randomly. • The cost function is calculated for each member of the population. • This population is referred to as “generation” 0. • The population size p should be approximately • where l is the length of the chromosome, k is the average length of the genes in the chromosome, and c= 2 for a binary system.

45. Genetic Algorithm • Generation 1 is formed from generation 0 using genetic processes of “death”, “mutation”, “creation”, “cloning” and “cross-over/mating”. • Every generation should have the same population size p.

46. Death • In the “survival of the fittest”, those chromosomes with a very poor cost function are removed from the population. • This is usually half the population. • The following processes are used to replenish the population.

47. Mutation • Randomly select a bit in one chromosome and flip it. • That is “0” → “1” or “1” → “0” • This is a member of the population of the next generation. • This is limited to a small number of the population (less than 5%).

48. Cloning • Copy one chromosome (usually with a low cost function) into the next generation. • This is often a large portion of the population (up to 50%)

49. Creation • Randomly generate another chromosome and place into the population of the new generation. • This is limited to a small number of the population (less than 5%).

50. Cross-over/Mating • Choose two chromosomes (A and B). • Randomly select a position in the chromosome. • Break both chromosomes at this point. • Join left A with right B and right A with left B to form two new chromosomes in the next generation. • This is often a large portion of the population (up to 50%).