This repo contains tutorials covering understanding and implementing sequence classification models using PyTorch, with Python 3.9. Specifically, we'll train models to predict sentiment from movie reviews.
If you find any mistakes or disagree with any of the explanations, please do not hesitate to submit an issue. I welcome any feedback, positive or negative!
Install the required dependencies with: pip install -r requirements.txt --upgrade.
-
This tutorial covers the workflow of a sequence classification project with PyTorch. We'll cover the basics of sequence classification using a simple, but effective, neural bag-of-words model, and how to use the datasets/torchtext libaries to simplify data loading/preprocessing.
-
Now we have the basic sequence classification workflow covered, this tutorial will focus on improving our results by switching to a recurrent neural network (RNN) model. We'll cover the theory behind RNNs, and look at an implementation of the long short-term memory (LSTM) RNN, one of the most common variants of RNN.
-
3 - Convolutional Neural Networks
Next, we'll cover convolutional neural networks (CNNs) for sentiment analysis. This model will be an implementation of Convolutional Neural Networks for Sentence Classification.
-
4 - Transformers
Finally, we'll show how to use the transformers library to load a pre-trained transformer model, specifically the BERT model from this paper, and use it for sequence classification.
Previous versions of these tutorials used features from the torchtext library which are no longer available. These are stored in the legacy directory.
Here are some things I looked at while making these tutorials. Some of it may be out of date.
- http://anie.me/On-Torchtext/
- http://mlexplained.com/2018/02/08/a-comprehensive-tutorial-to-torchtext/
- https://github.com/spro/practical-pytorch
- https://gist.github.com/Tushar-N/dfca335e370a2bc3bc79876e6270099e
- https://gist.github.com/HarshTrivedi/f4e7293e941b17d19058f6fb90ab0fec
- https://github.com/keras-team/keras/blob/master/examples/imdb_fasttext.py
- https://github.com/Shawn1993/cnn-text-classification-pytorch
