Loading...
Please wait, while we are loading the content...
Similar Documents
Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning
| Content Provider | arXiv |
|---|---|
| Author | Wu, Jian Toscano-Palmerin, Saul Frazier, Peter I. Wilson, Andrew Gordon |
| Date of Submission | 2019-03-11 |
| Abstract | Bayesian optimization is popular for optimizing time-consuming black-box objectives. Nonetheless, for hyperparameter tuning in deep neural networks, the time required to evaluate the validation error for even a few hyperparameter settings remains a bottleneck. Multi-fidelity optimization promises relief using cheaper proxies to such objectives --- for example, validation error for a network trained using a subset of the training points or fewer iterations than required for convergence. We propose a highly flexible and practical approach to multi-fidelity Bayesian optimization, focused on efficiently optimizing hyperparameters for iteratively trained supervised learning models. We introduce a new acquisition function, the trace-aware knowledge-gradient, which efficiently leverages both multiple continuous fidelity controls and trace observations --- values of the objective at a sequence of fidelities, available when varying fidelity using training iterations. We provide a provably convergent method for optimizing our acquisition function and show it outperforms state-of-the-art alternatives for hyperparameter tuning of deep neural networks and large-scale kernel learning. |
| Related Links | https://arxiv.org/pdf/1903.04703.pdf |
| arXiv | 1903.04703 |
| Language | English |
| Access Restriction | Open |
| Subject Keyword | Computer Science - Machine Learning Mathematics - Optimization and Control Statistics - Methodology Statistics - Machine Learning Statistics Computer Science Mathematics |
| Content Type | Text |
| Resource Type | Article |
| Subject | Statistics and Probability |
