[TENSORFLOW] Very deep convolutional networks for Text Classification

Introduction

Here is my tensorflow implementation of the model described in the paper Very deep convolutional networks for Text Classification paper.

Datasets:

Statistics of datasets I used for experiments. These datasets could be download from link

Dataset	Classes	Train samples	Test samples
AG’s News	4	120 000	7 600
Sogou News	5	450 000	60 000
DBPedia	14	560 000	70 000
Yelp Review Polarity	2	560 000	38 000
Yelp Review Full	5	650 000	50 000
Yahoo! Answers	10	1 400 000	60 000
Amazon Review Full	5	3 000 000	650 000
Amazon Review Polarity	2	3 600 000	400 000

Setting:

I almost keep default setting as described in the paper.

Additionally, in the original model, one epoch is seen as a loop over batch_size x num_batch records (128x5000 or 128x10000 or 128x30000), so it means that there are records used more than once for 1 epoch. In my model, 1 epoch is a complete loop over the whole dataset, where each record is used exactly once.

Training

After putting 2 files train.csv and test.csv from any dataset mentioned above into data folder, there are several options for training:

If you want to train a model with common dataset and default parameters, you could run:

• python train.py

If you want to train a model with your preference parameters, like depth and learning rate, you could run:

• python train.py --depth 29 --lr learning_rate: For example, python train.py --depth 29 --lr 0.001

If you want to train a model with your own dataset, you need to specify the paths to input (train and test files) and output folder:

• python train.py --train_set path/to/training/file --test_set path/to/test/file --saved_path path/to/output/folder

You could find all trained models I have trained in link

Experiments:

I run experiments in 2 machines, one with NVIDIA TITAN X 12gb GPU and the other with NVIDIA quadro 6000 24gb GPU.

Results for test set are presented as follows: A(B):

• A is accuracy reproduced here.
• B is accuracy reported in the paper.

It should be noted that in experiments with depth is 49 layers, there is no accuracy reported in the paper. Therefore here I only show the results obtained from my experiments.

Depth	9	17	29	49
ag_news	87.67(90.17)	88.09(90.61)	88.01(91.33)	84.71
sogu_news	95.67(96.42)	95.89(96.49)	95.73(96.82)	95.35
db_pedia	98.33(98.44)	98.28(98.39)	98.07(98.59)	97.38
yelp_polarity	94.57(94.73)	95.20(94.95)	94.95(95.37)	95.08
yelp_review	62.44(61.96)	63.44(62.59)	62.70(63.00)	62.83
yahoo_answer	69.57(71.76)	70.03(71.75)	70.34(72.84)	69.16
amazon_review	60.34(60.81)	60.98(61.19)	60.67(61.61)	59.80
amazon_polarity	94.30(94.31)	94.60(94.57)	94.53(95.06)	94.10

Below are the training/test loss/accuracy curves for each dataset's experiments (figures for 9, 17, 29-layer model are from left to right) :

• ag_news

• sogou_news

• db_pedia

• yelp_polarity

• yelp_review

• amazon_review

• amazon_polarity

You could find detail log of each experiment containing loss, accuracy and confusion matrix at the end of each epoch in trained_models/logs.txt.