dynet-benchmark

dynet-benchmark (last updated Dec 14, 2017)

This directory contains examples implemented for dynet-benchmark repo. See DyNet technical report for the architectural details of the implemented examples.

• rnnlm-batch: A recurrent neural network language model on PTB corpus.
• bilstm-tagger: A bidirectional LSTM network that predicts a tag for each word. It is trained on WikiNER dataset.
• bilstm-tagger-withchar: Similar to bilstm-tagger, but uses characer-based embeddings for unknown words.
• treenn: A tree-structured LSTM sentiment classifier trained on Stanford Sentiment Treebank dataset.

Results on Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz, Tesla K80

See the dynet-benchmark repo for the source code for each model and each framework.

Model	Metric	Knet	DyNet	Chainer
rnnlm-batch	words/sec	28.5k	18.7k	16k
bilstm-tagger	words/sec	6800	1200	157
bilstm-tagger-withchar	words/sec	1300	900	128
treenn	sents/sec	43	68	10

Example Usage

Simply call each script with -h (or --help) option to see all possible script options. For instance,

$ julia bilstm-tagger.jl -h
usage: bilstm-tagger.jl [--usegpu] [--embed EMBED] [--hidden HIDDEN]
                        [--mlp MLP] [--timeout TIMEOUT]
                        [--epochs EPOCHS] [--minoccur MINOCCUR]
                        [--report REPORT] [--valid VALID]
                        [--seed SEED] [-h]

Bidirectional LSTM Tagger in Knet

optional arguments:
  --usegpu             use GPU or not
  --embed EMBED        word embedding size (type: Int64, default: 128)
  --hidden HIDDEN      LSTM hidden size (type: Int64, default: 50)
  --mlp MLP            MLP size (type: Int64, default: 32)
  --timeout TIMEOUT    max timeout (in seconds) (type: Int64, default:
                       600)
  --epochs EPOCHS      number of training epochs (type: Int64,
                       default: 100)
  --minoccur MINOCCUR  word min occurence limit (type: Int64, default:
                       6)
  --report REPORT      report period in iters (type: Int64, default:
                       500)
  --valid VALID        valid period in iters (type: Int64, default:
                       10000)
  --seed SEED          random seed (type: Int64, default: -1)
  -h, --help           show this help message and exit

Use --usegpu option to run examples on GPU,

$ julia bilstm-tagger.jl --usegpu

To run examples without time limit, pass a non-positive integer to --timeout option,

$ julia bilstm-tagger.jl --usegpu --timeout 0