notMNIST

Letter recognition classifiers trained on notMNIST (alphabet A-J)

• Udacity
  • Intro to Deep Learning

    • Assignment 1: notMNIST

    • Assignment 2: Fully Connected

    • Assignment 3: Regularization

    • Assignment 4: Convulotions

Data

• notMNIST

529119 28 x 28 grayscale images of letters A - J, sorted into directories by letter. There are 52920 images for letters A - I, and 52919 images for letter J. This dataset is more challenging version of the image classification problem than MNIST: classifying letters from images. It is a multiclass classification dataset of glyphs of English letters A - J.

Training

Trained using Zotac GTX 1070 AMP edition 8GB

• Sklearn Logistic Regression

  • 84% accuracy
  • reference

• Full batch: (linear) Logistic Regression with TensorFLow
  • 88% accuracy

• Mini batch (linear)
  • 90% accuracy
  • batch_size=1000, took: 155 seconds

• Mini batch (ReLU)

  • 95% accuracy
  • took: 188 seconds
  

• ReLU with L2-regularization

  • 95.9% accuracy
  • took: 192 seconds

• ReLU with dropout+L2-regularization
  • 95.7% accuracy
  • took: 208 seconds

• CNN with dropout+L2-regularization (similar to LeNet)
  • 95.5% accuracy
  • took: 391.5 seconds

• CNN inception module
  • concatenation
    • 95.1% accuracy
    • took: 554 seconds
    
  • reduce mean
    • 93.6% accuracy
    • took: 528.5 seconds

• RNNs
  • Basic RNN
    • 90.5% accuracy
    • took: 125.7 seconds
    
  • LSTM
    • 94.1% accuracy
    • 474 seconds
    
  • 2 GRUs
    • 94.4% accuracy
    • 1440 seconds
    

Libraries

Tested on Python 3.5

• TensorFlow
• scikit-learn
• NumPy
• IPython
• Matplotlib
• Joblib (optional)