You have built an SVM model. It runs. It makes predictions. But is it actually good?<\/p>\n\n\n\n
Default parameters are a starting point, not a destination. The difference between default settings and properly tuned hyperparameters can be the difference between a model that barely beats a coin flip and one your team ships to production.<\/p>\n\n\n\n
GridSearchCV bridges that gap by systematically testing every combination of hyperparameters you specify, evaluating each using cross-validation, and returning the configuration that performs best on your data.<\/p>\n\n\n\n
This guide covers what SVM hyperparameters actually do, how GridSearchCV works, how to implement it correctly, and what mistakes to avoid so you do not spend hours tuning parameters the wrong way.<\/p>\n\n\n\n
\n SVM hyperparameter tuning is the process of selecting the optimal values for parameters such as C, gamma, and kernel type that control how a Support Vector Machine learns its decision boundary. These hyperparameters directly affect the model\u2019s accuracy, generalization ability, and performance on unseen data. Techniques like GridSearchCV automate this process by systematically testing different parameter combinations with cross-validation to identify the configuration that produces the best results.\n <\/p>\n\n <\/div>\n\n<\/div>\n\n\n\n
C controls how much the model penalizes training errors. Small C creates a wide margin allowing some misclassifications, prioritizing a generalizable boundary. Large C penalizes every error heavily, fitting training data tightly at the risk of overfitting.<\/p>\n\n\n\n
Typical values to try: 0.001, 0.01, 0.1, 1, 10, 100, 1000<\/p>\n\n\n\n
Gamma applies to RBF, polynomial, and sigmoid kernels. Small gamma means each example influences a wide region, producing smooth boundaries. Large gamma means only nearby points matter, producing complex boundaries that may overfit.<\/p>\n\n\n\n
Typical values to try: 0.0001, 0.001, 0.01, 0.1, 1, 10. Start with “scale” before manual tuning.<\/p>\n\n\n\n
Linear<\/strong> works best for high-dimensional sparse data like text. RBF<\/strong> handles non-linear boundaries and is the default starting point for most problems. Polynomial<\/strong> is useful when feature relationships are known to be polynomial. Sigmoid<\/strong><\/a> rarely outperforms RBF in practice.<\/p>\n\n\n\n \n The RBF (Radial Basis Function) kernel<\/strong> used in Support Vector Machines<\/strong> effectively maps data into an infinite-dimensional feature space<\/strong>, where classes that appear inseparable in the original space may become linearly separable by a simple hyperplane. This is why an RBF SVM can produce highly flexible curved decision boundaries while still relying on linear optimization internally. The behavior of the model is heavily controlled by the C<\/strong> and gamma<\/strong> hyperparameters, which determine the balance between margin size, generalization, and sensitivity to local patterns in the data.\n <\/p>\n<\/div>\n\n\n\n GridSearchCV tests every combination in your parameter grid using k-fold cross-validation. For each combination, it trains on k-1 folds and evaluates on the remaining fold, repeating k times. The average score across all folds determines the best combination.<\/p>\n\n\n\n With 4 C values, 6 gamma values, and 2 kernels, that is 48 combinations. With 5-fold CV, that means 240 model fits total. After finding the best parameters, it refits a fresh model on the entire training set, giving best_estimator_ more data than any individual fold saw during search.<\/p>\n\n\n\n Key outputs:<\/strong> best_params_ (winning combination), best_score_ (its CV score), best_estimator_ (model retrained on full training data), cv_results_ (full results for every combination).<\/p>\n\n\n\n Read More: <\/strong>How Long Does it Take to Learn Machine Learning?<\/strong><\/a><\/p>\n\n\n\n from sklearn.datasets import load_breast_cancer<\/p>\n\n\n\n from sklearn.model_selection import train_test_split, GridSearchCV<\/p>\n\n\n\n from sklearn.preprocessing import StandardScaler<\/p>\n\n\n\n from sklearn.svm import SVC<\/p>\n\n\n\n data = load_breast_cancer()<\/p>\n\n\n\n X, y = data.data, data.target<\/p>\n\n\n\n X_train, X_test, y_train, y_test = train_test_split(<\/p>\n\n\n\n X, y, test_size=0.2, random_state=42, stratify=y<\/p>\n\n\n\n )<\/p>\n\n\n\n scaler = StandardScaler()<\/p>\n\n\n\n X_train_scaled = scaler.fit_transform(X_train)<\/p>\n\n\n\n X_test_scaled = scaler.transform(X_test)<\/p>\n\n\n\n param_grid = {<\/p>\n\n\n\n ‘C’: [0.01, 0.1, 1, 10, 100],<\/p>\n\n\n\n ‘gamma’: [‘scale’, ‘auto’, 0.001, 0.01, 0.1, 1],<\/p>\n\n\n\n ‘kernel’: [‘rbf’, ‘linear’]<\/p>\n\n\n\n }<\/p>\n\n\n\n grid_search = GridSearchCV(<\/p>\n\n\n\n estimator=SVC(random_state=42),<\/p>\n\n\n\n param_grid=param_grid,<\/p>\n\n\n\n cv=5,<\/p>\n\n\n\n scoring=’accuracy’,<\/p>\n\n\n\n n_jobs=-1,<\/p>\n\n\n\n refit=True<\/p>\n\n\n\n )<\/p>\n\n\n\n grid_search.fit(X_train_scaled, y_train)<\/p>\n\n\n\n print(“Best Parameters:”, grid_search.best_params_)<\/p>\n\n\n\n print(“Best CV Score:”, grid_search.best_score_)<\/p>\n\n\n\n print(“Test Accuracy:”, grid_search.best_estimator_.score(<\/p>\n\n\n\n X_test_scaled, y_test))<\/p>\n\n\n\n Always fit the scaler on training data only and transform both sets. Feature scaling is not optional for SVM<\/a>.<\/p>\n\n\n\n import pandas as pd<\/p>\n\n\n\n results = pd.DataFrame(grid_search.cv_results_)<\/p>\n\n\n\n print(results.sort_values(‘rank_test_score’)[[<\/p>\n\n\n\n ‘param_C’, ‘param_gamma’, ‘param_kernel’,<\/p>\n\n\n\n ‘mean_test_score’, ‘std_test_score’<\/p>\n\n\n\n ]].head(10))<\/p>\n\n\n\n Always check std_test_score alongside mean_test_score. A combination with slightly lower mean but much lower standard deviation is often the better real-world choice.<\/p>\n\n\n\n One grid search is rarely enough. Start broad to find the promising region, then narrow in for precision.<\/p>\n\n\n\n # Coarse search<\/p>\n\n\n\n coarse_grid = {<\/p>\n\n\n\n ‘C’: [0.001, 0.01, 0.1, 1, 10, 100, 1000],<\/p>\n\n\n\n ‘gamma’: [0.0001, 0.001, 0.01, 0.1, 1, 10],<\/p>\n\n\n\n ‘kernel’: [‘rbf’, ‘linear’]<\/p>\n\n\n\n }<\/p>\n\n\n\n coarse_search = GridSearchCV(SVC(), coarse_grid, cv=5, n_jobs=-1)<\/p>\n\n\n\n coarse_search.fit(X_train_scaled, y_train)<\/p>\n\n\n\n best_C = coarse_search.best_params_[‘C’]<\/p>\n\n\n\n best_gamma = coarse_search.best_params_[‘gamma’]<\/p>\n\n\n\n # Fine search around best values<\/p>\n\n\n\n fine_grid = {<\/p>\n\n\n\n ‘C’: [best_C\/5, best_C\/2, best_C, best_C*2, best_C*5],<\/p>\n\n\n\n ‘gamma’: [best_gamma\/5, best_gamma\/2, best_gamma,<\/p>\n\n\n\n best_gamma*2, best_gamma*5],<\/p>\n\n\n\n ‘kernel’: [coarse_search.best_params_[‘kernel’]]<\/p>\n\n\n\n }<\/p>\n\n\n\n fine_search = GridSearchCV(SVC(), fine_grid, cv=5, n_jobs=-1)<\/p>\n\n\n\n fine_search.fit(X_train_scaled, y_train)<\/p>\n\n\n\n print(“Fine best:”, fine_search.best_params_)<\/p>\n\n\n\n When your grid exceeds roughly 100 combinations, RandomizedSearchCV samples randomly from parameter distributions instead of testing every point. Research shows it finds near-optimal configurations in around 60 samples that would need hundreds with grid search.<\/p>\n\n\n\n from sklearn.model_selection import RandomizedSearchCV<\/p>\n\n\n\n from scipy.stats import loguniform<\/p>\n\n\n\n param_distributions = {<\/p>\n\n\n\n ‘C’: loguniform(0.001, 1000),<\/p>\n\n\n\n ‘gamma’: loguniform(0.0001, 10),<\/p>\n\n\n\n ‘kernel’: [‘rbf’, ‘linear’, ‘poly’]<\/p>\n\n\n\n }<\/p>\n\n\n\n random_search = RandomizedSearchCV(<\/p>\n\n\n\n SVC(),<\/p>\n\n\n\n param_distributions=param_distributions,<\/p>\n\n\n\n n_iter=60,<\/p>\n\n\n\n cv=5,<\/p>\n\n\n\n scoring=’accuracy’,<\/p>\n\n\n\n n_jobs=-1,<\/p>\n\n\n\n random_state=42<\/p>\n\n\n\n )<\/p>\n\n\n\n random_search.fit(X_train_scaled, y_train)<\/p>\n\n\n\n print(“Best params:”, random_search.best_params_)<\/p>\n\n\n\n print(“Best score:”, random_search.best_score_)<\/p>\n\n\n\nHow GridSearchCV Works<\/strong><\/h2>\n\n\n\n
Full Implementation<\/strong><\/h2>\n\n\n\n
\n
\n
Coarse-to-Fine Tuning<\/strong><\/h2>\n\n\n\n
RandomizedSearchCV: Faster Alternative for Large Grids<\/strong><\/h2>\n\n\n\n