AttributeError: super object has no attribute __sklearn_tags__ in Scikit-learn and XGBoost

Problem Statement

When using RandomizedSearchCV or other Scikit-learn tools with XGBRegressor, you may encounter the error:

AttributeError: 'super' object has no attribute '__sklearn_tags__'

This error occurs during the fit() operation and is triggered by compatibility issues between Scikit-learn and XGBoost versions. The problem specifically arises when:

• Using Scikit-learn's hyperparameter tuning utilities (RandomizedSearchCV, GridSearchCV)
• Working with Python 3.12
• Installing the latest versions of both libraries without version constraints
• Attempting to tune XGBoost estimators

The error stems from API changes in Scikit-learn and requires version-specific resolution.

Compatibility Analysis

The root cause lies in Scikit-learn's API modifications:

• Scikit-learn 1.6.0 introduced breaking changes to the "tags" API
• XGBoost versions <2.1.4 weren't compatible with these changes
• Scikit-learn 1.6.1 downgraded the error to a warning (later to be reinstated as an error in Scikit-learn 1.7+)

Key version interactions:

│ Scikit-learn Version │ XGBoost Version │ Result          │
├──────────────────────┼─────────────────┼─────────────────┤
│ <1.6.0               │ Any             │ ✅ Works        │
│ 1.6.0                │ <2.1.4          │ ❌ Fails        │
│ 1.6.1 to 1.6.x       │ <2.1.4          │ ⚠️ Warning     │
│ ≥1.6.0               │ ≥2.1.4          │ ✅ Works        │
│ ≥1.7                 │ <2.1.4          │ ❌ Fails        │

Recommended Solutions

1. Update XGBoost (Preferred Solution)

Upgrade to XGBoost ≥2.1.4 for seamless compatibility with modern Scikit-learn versions:

pip install --upgrade "xgboost>=2.1.4"

2. Downgrade Scikit-learn

Install compatible Scikit-learn versions if XGBoost update isn't feasible:

pip uninstall -y scikit-learn
pip install "scikit-learn<1.6"

Compatible versions: scikit-learn==1.5.2 or scikit-learn==1.3.1

3. Intermediate Scikit-learn with Warnings

For Scikit-learn 1.6.1 - 1.6.x environments (temporary solution):

pip install "scikit-learn>=1.6.1,<1.7"

Expect DeprecationWarning messages during execution, but code will run.

Verification Steps

Confirm installed versions in Python:

import sklearn, xgboost
print(f"Scikit-learn: {sklearn.__version__}")
print(f"XGBoost: {xgboost.__version__}")

Validate output matches one of these compatible combinations:

Scikit-learn: 1.5.2
XGBoost: any_version

# OR

Scikit-learn: 1.6.1
XGBoost: 2.1.4+

WARNING

Avoid mixing scikit-learn>=1.6.0 with xgboost<2.1.4. This combination is guaranteed to cause failures in production environments.

Complete Working Example

After resolving version conflicts:

from sklearn.model_selection import RandomizedSearchCV
from xgboost import XGBRegressor

# Sample tuning configuration
param_dist = {
    'max_depth': [3, 6, 9],
    'learning_rate': [0.01, 0.1, 0.3],
    'n_estimators': [100, 200]
}

model = RandomizedSearchCV(
    XGBRegressor(),
    param_distributions=param_dist,
    n_iter=10,
    cv=5
)

# Will execute without errors after version fixes
model.fit(X_train, y_train)
print(f"Best parameters: {model.best_params_}")

References

1. Scikit-learn 1.6.1 Release Notes
2. XGBoost 2.1.4 Fix Commit
3. Scikit-learn Issue #30479