MIT researchers developed a new approach that rethinks how a classic method, known as Bayesian optimization, can be used to solve problems with hundreds of variables. In tests on realistic engineering-style benchmarks, like power-system optimization, the approach found top solutions 10 to 100 times faster than widely used methods.
Their technique leverages a foundation model trained on tabular data that automatically identifies the variables that matter most for improving performance, repeating the process to hone in on better and better solutions. The researchers’ tabular foundation model does not need to be constantly retrained as it works toward a solution, increasing the efficiency of the optimization process.
The technique also delivers greater speedups for more complicated problems, so it could be especially useful in demanding applications like materials development or drug discovery. The research will be presented at the International Conference on Learning Representations.
Their technique leverages a foundation model trained on tabular data that automatically identifies the variables that matter most for improving performance, repeating the process to hone in on better and better solutions. The researchers’ tabular foundation model does not need to be constantly retrained as it works toward a solution, increasing the efficiency of the optimization process.
The technique also delivers greater speedups for more complicated problems, so it could be especially useful in demanding applications like materials development or drug discovery. The research will be presented at the International Conference on Learning Representations.
