Gilles Cohen, Patrick Ruch, University Hospital of Geneva; and Mélanie Hilario, University of Geneva
This paper addresses the problem of tuning hyperparameters in support vector machine modeling. A Direct Simplex Search (DSS) method, which seeks to evolve hyperparameter values using an empirical error estimate as steering criterion, is proposed and experimentally evaluated on real-world datasets. DSS is a robust hill climbing scheme, a popular derivative-free optimization method, suitable for low-dimensional optimization problems for which the computation of the derivatives is impossible or difficult. Our experiments show that DSS attains performance levels equivalent to that of GS while dividing computational cost by a minimum factor of 4.