Cannot save SparseLinearDRLearner to persistent memory

[kanikavmohan]

I trained a SparseLinearDRLearner in EconML, but am unable to save the model to persistent memory using joblib/pickle - I get the following error message:

"AttributeError: Can't pickle local object 'DRLearner.__init__.<locals>.ModelNuisance'".

Is this a known issue? Are there any work arounds?

[kbattocchi]

Several of our classes (not only DRLearner) use nested local classes, which python's built-in pickle library can't handle, unfortunately (and I believe that this is what joblib uses for dumping files). Perhaps you could try using a different pickling library like dill?

[kanikavmohan]

Thank you for the quick response. I did try dill as well, and got the following error:

_pickle.PicklingError: Can't pickle <class 'econml.drlearner.DRLearner.__init__.<locals>.ModelNuisance'>: it's not found as econml.drlearner.DRLearner.__init__.<locals>.ModelNuisance

Does Microsoft provide other open source software that can save these models?

[kbattocchi]

@kanikavmohan Sorry, serialization is not something we took into account when writing all of our classes, and I'm not sure of any easy workaround.

We'll treat this as a bug and will have it fixed in the next version of the package, which we anticipate releasing by the end of next week. Sorry for the inconvenience.

[yqc924]

@kanikavmohan Sorry, serialization is not something we took into account when writing all of our classes, and I'm not sure of any easy workaround.

We'll treat this as a bug and will have it fixed in the next version of the package, which we anticipate releasing by the end of next week. Sorry for the inconvenience.

Hi, is this problem solved? I encountered a similar problem when I was training a DMLCateEstimator. The error report was: "Error: can't pickle generator objects."

[kbattocchi]

It has been fixed in our repository and is in our latest beta release on pypi (which doesn't yet have a final version). You should be able to install the beta via pip install -U --pre econml. Please let us know if that doesn't resolve the issue.

[jaronowitz]

I have econml 0.8.1. I was able to dill.Dump a DMLOrthoForest but when I go to dill.Load I get

NotFittedError: This DMLOrthoForest instance is not fitted yet.

Is there any other work around? How can I productionize an econml model if it can't be saved?

[hanshengjiang]

I have encountered similar issues. Just wondering if there is any update on dump/load?