Support and experiments for running Comportex HTM models on the Numenta Anomaly Benchmark.
For context, read the blog post Attempting the Numenta Anomaly Benchmark.
The results as submitted to the
2016 NAB Competition can be found in the
results directory.
Headline results:
| model | standard | low FP rate | low FN rate |
|---|---|---|---|
| original NuPIC model + anomaly likelihood | 65.3 | 58.6 | 69.4 |
| original NuPIC model, raw bursting score | 52.5 | 41.1 | 58.3 |
| selected Comportex model, delta anomaly score | 64.6 | 58.8 | 69.6 |
Effective time step model results:
| model | standard | low FP rate | low FN rate |
|---|---|---|---|
| effective time steps when 20% columns change | 64.7 | 59.8 | 69.5 |
To reproduce the results submitted to the 2016 NAB Competition:
- install Leiningen.
- clone this git repo. It should be placed alongside the
NABdirectory so that data files can be read from and result files written to../NAB.
git checkout v0.1.0
- In the
NABdirectory:
python scripts/create_new_detector.py --detector comportexDepth1GlobalFrac16Stim18python scripts/create_new_detector.py --detector comportexDepth1GlobalFrac16Stim18NoDeltapython scripts/create_new_detector.py --detector comportexDepth1EfftGlobalFrac16Stim18
- In the
nab-comportexdirectory:
lein repl
- In the repl:
(start-notebook) - Note the port number such as 12345, then go to http://localhost:12345/worksheet.html?filename=worksheeets/method.clj
- Keep pressing Shift+Enter...
The results will be written into ../NAB/results/.
It would also be possible to run the clojure file
worksheets/method.clj in other ways, interactive or not.
Copyright © 2016 Felix Andrews
Distributed under the GNU Affero General Public Licence, Version 3, the same as Comportex.