LibLinear and LibSVM have unmanaged global RNGs #172
Conversation
… it synchronized and resetting the RNG everywhere.
…nstance of LibLinearTrainer.
…ot anomaly detection.
LibSVM has a global RNG which is used when making probailistic predictions. This commit uses Tribuo's standard mechanisms to track RNG state to ensure this is reproducible, at the cost of sequentializing all LibSVM training runs to ensure that the RNG state is consistent with the provenance. Also added some more Javadoc.
Craigacp
added
Oracle employee
…tReproducibility to show the models differ in tracked ways.
There was a problem hiding this comment.
The train methods in the trainers for LibLinear and LibSVM call Linear.resetRandom() svm.rand.setSeed(localRNG.nextLong()), respectively. The latter looks a little suspect at first glance but note that different instances of LibSVMTrainer will have identical localRNG instances. The respective train methods also synchronize on LibLinearTrainer.class and LibSVMTrainer.class. A couple of unit tests demonstrate that subsequent runs of the train method for both produce the same model parameters.
The trade-off between locking on classes Tribuo does not control (not done) and the risk of someone using Liblinear in the same jvm in a non-synchronized way and mucking things up is documented.
SVMAnomalyType.isClassification and .isAnomaly had the boolean return values flipped. oops! Corrected here.
Various and sundry documentation comments are sprinkled in the code
* Fixing a concurrency and reproducibility issue in liblinear by making it synchronized and resetting the RNG everywhere. * Changing the lock so it's on LibLinearTrainer.class rather than the instance of LibLinearTrainer. * Fixing a bug where SVMAnomalyType reported itself as classification not anomaly detection. * Makes LibSVM training reproducible. LibSVM has a global RNG which is used when making probailistic predictions. This commit uses Tribuo's standard mechanisms to track RNG state to ensure this is reproducible, at the cost of sequentializing all LibSVM training runs to ensure that the RNG state is consistent with the provenance. Also added some more Javadoc. * Updating comments after review. Adding an extra bit to TestLibSVM.testReproducibility to show the models differ in tracked ways. * extended unit test to test that two subsequent models are the same Co-authored-by: Philip Ogren <philip.ogren@oracle.com>
Description
This change fixes Tribuo's LibLinear and LibSVM interfaces so they use trackable RNG states.
LibLinear has a protected global RNG with a reset method (to a fixed seed), so now Tribuo sequentialises all LibLinear training runs under a lock on
LibLinearTrainer.classand resets the seed before each call toLinear.train. There's a new reproducibility test in the classification LibLinear module which fails without these changes. This fix may change depending on the resolution of bwaldvogel/liblinear-java#40.LibSVM has a public global RNG so we use Tribuo's standard RNG idiom (a per train call
localRNGsplit from the trainer's instance ofSplittableRandom), LibSVM's global RNG's seed is set to a draw fromlocalRNGand all LibSVM training runs under a lock onLibSVMTrainer.class. There's a new reproducibility test in the classification LibSVM module which fails without these changes. Unfortunately it's quite a chatty test as LibSVM prints a lot during this run.Motivation
We need Tribuo models to be reproducible.