PDF version of this document
One of the main flaws of existing optimisation methods is their inability to find a global minimum (or minima) fairly quickly without some additional knowledge about the function under investigation. Rough assumptions about the behaviour of the curve along each of the axes are otherwise made.
The pace of the optimisation process can be boosted on the expense of overall accuracy and error likelihood. Sometimes these cannot be jeopardised, mammography being an example of choice. It turns out that if no exhaustive search35 is carried out, there is then a danger of convergence at some local minimum. In most applications, any stoppage at a local minimum would be highly undesirable although this may be better than a complete failure at identifying low points. Local minima are a necessary evil for large and complex continuous functions.
In conclusion, there is a trade-off between speed and accuracy although accuracy can be achieved at a lower cost if more knowledge is acquired off-line, before the optimisation task actually begins. Quite expectedly, this also implies that many redundant computations will consume precious resources and time in order to train the optimiser.