Hyperparameter Search and Nested Cross-Validation › Advantages

Recording the search space, fold rule, budget, and random seeds makes the selection procedure explicit, so it can be inspected and rerun with the same data and code. Instead of trying a few settings by hand and remembering the results, we define a search space, a validation rule, and a budget. Every candidate is evaluated on the same folds, and the selected configuration can be traced back to a specific score. This is valuable when several people collaborate on a model, when a pipeline must be rerun later, or when a regulator or reviewer asks how the final configuration was chosen.

Grid search is simple and exhaustive over a finite grid. When the search space is small and mostly discrete, it is easy to audit and easy to parallelize. Random search is often more efficient when only a few hyperparameters matter, because it spends less effort varying irrelevant dimensions. Nested cross-validation adds an honest estimate of the complete procedure, including the search step. It also exposes instability. If different outer folds select very different hyperparameters, we learn that the selection rule is not stable, which is useful information that a single best score hides.