Skip to content

Latest commit

 

History

History
42 lines (32 loc) · 1.96 KB

README.md

File metadata and controls

42 lines (32 loc) · 1.96 KB
Raw

Extreme Learning Machine

Introduction

This is the respository that implement the Extreme Learning Machine for Single Hidden-layer Feedfoward Neural Network (SLFN). This learning algorithm is extremely fast if compare to Back-propagation or Gradient-based learning algorithm.

Benchmark

Boston Housing dataset for Regression problem

Algorithm Type Training time (miliseconds) Trainset RMSE Testset RMSE
ELM Neural Network 54.62 4.69 4.82
Ridge Linear Model 2.66 4.72 4.75
SVR Support Vector Machine 26.32 8.25 8.15
K-Nearest Neighbors Nearest Neighbors 1.97 4.98 6.08
Decision Tree Tree-based 7.2 0 (overfit) 5.1
Random Forest Tree-based Ensemble 293.13 1.19 3.8
Perceptron (Back-propagation) Neural Network 237.5 6.94 7.43

MNIST dataset for Classification problem

Algorithm Type Training time (miliseconds) Trainset accuracy (%) Testset accuracy (%)
ELM Neural Network 4754.13 91.94 92.22
Logistic Regression Linear Model 21112.03 93.39 92.55
SVC Support Vector Machine 280275.89 98.99 97.92
K-Nearest Neighbors Nearest Neighbors 5.07 98.19 96.88
Decision Tree Tree-based 17913.2 100.0 (overfit) 87.68
Random Forest Tree-based Ensemble 37244.7 100.0 (overfit) 96.95
Perceptron (Back-propagation) Neural Network 379703.05 99.73 97.85

Above is just a compact comparision table, you guys can see more detail in notebook.

Structure

  • elm.py: This is a file that implement ELMBase class and two class for Classification and Regression: ELMClassifier and ELMRegressor.
  • elm_classification.ipynb: Notebook contain testing code for ELMClassifier for MNIST dataset.
  • elm_regression.ipynb: Notebook contain testing code for ELMRegressor on Boston Housing dataset.
  • utils.py: Utility functions support above 3 files.

References

[1] Guang-Bin Huang, Qin-Yu Zhu, Chee-Kheong Siew, Extreme learning machine: Theory and applications, 2006. https://doi.org/10.1016/j.neucom.2005.12.126.