Scikit‑Optimize (skopt): Features, Use Cases, and Code Examples

Scikit‑Optimize (skopt) is an open‑source Python library designed for black‑box optimization, where the objective function can only be evaluated via calls without gradient or analytical information.

Key features include strong adaptability to various black‑box functions (continuous, discrete, constrained or unconstrained), efficient global optimization algorithms such as random‑forest, Gaussian‑process‑based Bayesian optimization, and built‑in support for hyper‑parameter tuning, interactive monitoring, and visualization. It integrates smoothly with the Scikit‑Learn API.

Typical use cases cover machine‑learning hyper‑parameter tuning (e.g., neural networks, SVMs, random forests), experimental design in physics or chemistry, financial and business decision optimization, and engineering problems such as structural design, circuit layout, or robot path planning.

Below are five code examples demonstrating basic usage, hyper‑parameter optimization, constrained optimization, interactive optimization with callbacks, and result visualization.

from skopt import gp_minimize import numpy as np # define objective def objective(x): return -(np.sin(5 * x) * (1 - np.tanh(x ** 2))) bounds = [(-2, 2)] result = gp_minimize(objective, bounds, n_calls=20) print(f"Best solution: {result.x[0]}, Objective value: {-result.fun}")

from sklearn.ensemble import RandomForestClassifier from skopt import BayesSearchCV # build classifier rf_clf = RandomForestClassifier(random_state=42) # define search space param_space = { 'n_estimators': (10, 500), 'max_depth': (None, 30), 'min_samples_split': (2, 20), 'min_samples_leaf': (1, 20), } # Bayesian optimization bayes_search = BayesSearchCV( rf_clf, param_space, n_iter=50, cv=5, scoring='accuracy', random_state=42, ) bayes_search.fit(X_train, y_train) print(f"Best hyperparameters: {bayes_search.best_params_}") print(f"Best cross-validation score: {bayes_search.best_score_}")

from skopt import forest_minimize from skopt.space import Real # constrained objective def constrained_objective(x): if x < -1 or x > 1: return float("inf") # infeasible return x**2 search_space = [Real(-2, 2)] result = forest_minimize(constrained_objective, search_space) print(f"Best solution under constraints: {result.x[0]}, Objective value: {result.fun}")

from skopt import gp_minimize, dump, load from skopt.callbacks import DeltaYStopper, VerboseCallback # assume objective and bounds are defined optimizer = gp_minimize( objective, bounds, callback=[ VerboseCallback(10), # print every 10 iterations DeltaYStopper(0.01), # stop when improvement <1% ], ) optimizer.run() # save result dump(optimizer, 'optimizer_result.pkl') # later load and continue optimizer = load('optimizer_result.pkl') optimizer.run()

from skopt.plots import plot_convergence, plot_objective # plot convergence curve plot_convergence(result) # plot objective contour (2‑D example) plot_objective(result, dimensions=[(-2, 2), (-2, 2)], n_points=50)

The above examples demonstrate how Scikit‑Optimize can be applied to a variety of black‑box optimization scenarios, from basic function minimization to complex hyper‑parameter tuning, constrained problems, interactive control, and visual analysis of optimization progress.