This paper proposes a measure-theoretic framework to study iterative stochastic optimizers that provides theoretical tools to explore how the optimization methods may be improved. Within this framework, optimizers form a closed, convex subset of a normed vector space, implying the existence of a distance metric between any two optimizers and a meaningful and computable spectrum of new optimizers between them. It is shown how the formalism applies to evolutionary algorithms in general. The analytic property of continuity is studied in the context of genetic algorithms, revealing the conditions under which approximations such as meta-modeling or surrogate methods may be effective. These results demonstrate the power of the proposed analytic framework, which can be used to propose and analyze new techniques such as controlled convex combinations of optimizers, meta-optimization of algorithm parameters, and more.

PDF

In Proceedings of the 12th International Workshop on Foundations of Genetic Algorithms (FOGA-2013) 2013. ACM Press.

@inproceedings{lockett:foga2013, title={A Measure-Theoretic Analysis of Stochastic Optimization}, author={Alan J. Lockett and Risto Miikkulainen}, booktitle={Proceedings of the 12th International Workshop on Foundations of Genetic Algorithms (FOGA-2013)}, publisher={ACM Press}, url="http://www.cs.utexas.edu/users/ai-lab?lockett:foga2013", year={2013} }