What is AdaBoost?
AdaBoost, short for Adaptive Boosting, is a popular ensemble learning algorithm that combines the outputs of multiple weak classifiers to produce a strong classifier. It works by iteratively training weak classifiers on the data, with each classifier focusing on the instances that were misclassified by the previous classifier. The final strong classifier is a weighted combination of the weak classifiers.
How does AdaBoost work?
The main steps of the AdaBoost algorithm are:
- Assign equal weights to each training instance.
- Train a weak classifier on the weighted data.
- Calculate the classifier’s error rate and update its weight in the ensemble.
- Update the training instance weights based on the classifier’s performance, increasing the weights of misclassified instances.
- Repeat steps 2-4 for a specified number of iterations or until a stopping criterion is met.
- Combine the weak classifiers using their weights to form the final strong classifier.
Example of AdaBoost in Python using scikit-learn
from sklearn.ensemble import AdaBoostClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
# Load the iris dataset
iris = load_iris()
X, y = iris.data, iris.target
# Split the dataset into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
# Create and train the AdaBoost classifier
ada_clf = AdaBoostClassifier(n_estimators=50, learning_rate=1, random_state=42)
ada_clf.fit(X_train, y_train)
# Make predictions and evaluate the classifier
y_pred = ada_clf.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))
