Lazy loading

Lazy Loading of Libraries

Various RAVEN features can use different libraries. Some of these libraries can take noticeable time to load. If they are only used for some inputs, they should be lazily loaded so they are not loaded for inputs that do not use them.

Methods of Lazily loading

Use importerUtils

In the importerUtils, lazy loading methods can be used to do lazy imports. First the module is imported. It will be imported the first time that any attributes of the module are used.

Before (non lazy):

import tensorflow as tf
...
self.availLayer['dense'] = tf.keras.layers.Dense

After (lazy):

import utils.importerUtils
tf =  utils.importerUtils.importModuleLazyRenamed("tf", globals(), "tensorflow")
...
self.availLayer['dense'] = tf.keras.layers.Dense

If renaming the module is not desired then the shorter form can be used:

Before (non-lazy):

import tensorflow

After (lazy):

import utils.importerUtils
tensorflow =  utils.lazyImporterUtils.importModuleLazy("tensorflow", globals())

Load module in function

If the module is only used in one function, then the module can just be loaded that function.

Before (non-lazy):

import sklearn.neighbors

def someFunc():
  nr =  sklearn.neighbors.KNeighborsRegressor(...)

After (lazy):

def someFunc():
  import sklearn.neighbors
  nr =  sklearn.neighbors.KNeighborsRegressor(...)

How to load plugins lazily

PluginFactory does not load the plugin immediately. In order to load the plugin, PluginFactory.finishLoadPlugin needs to be called.

So something like:

#found something.stuff in input file
PluginFactory.finishLoadPlugin("something")
#then can check for something.stuff

Modules that always must be lazily loaded

These are modules that are time consuming to load and only sometimes needed, so they should only be loaded when the input requires them.

• tensorflow
• statsmodels
• scikit-learn (sklearn)

Modules that can be directly loaded

These are modules that practically every RAVEN input will need, so they can just be loaded directly.

• numpy
• pandas
• xarray