No Free Lunch Theorem and Bayesian probability theory: two sides of the same coin. Some implications for black-box optimization and metaheuristics

2013/11/23 by Loris Serafino, Serafino, Loris
Computer Science · #Evolutionary Algorithms and Applications #cs.LG

paper · pdf · doi:10.48550/arxiv.1311.6041

Multiple changes throughout the paper

arxiv created 2013/12/01 · arxiv updated 2013/12/03

Abstract

Challenging optimization problems, which elude acceptable solution via conventional calculus methods, arise commonly in different areas of industrial design and practice. Hard optimization problems are those who manifest the following behavior: a) high number of independent input variables; b) very complex or irregular multi-modal fitness; c) computational expensive fitness evaluation. This paper will focus on some theoretical issues that have strong implications for practice. I will stress how an interpretation of the No Free Lunch theorem leads naturally to a general Bayesian optimization framework. The choice of a prior over the space of functions is a critical and inevitable step in every black-box optimization.

Citations

• Portfolio Allocation for Bayesian Optimization

Related

• Recent Results on No-Free-Lunch Theorems for Optimization
• What is important about the No Free Lunch theorems?
• Thinking inside the box: A tutorial on grey-box Bayesian optimization
• Understanding an Acquisition Function Family for Bayesian Optimization
• Bayesian Optimization of Expensive Nested Grey-Box Functions