inference='auto'

[vasilismsr]

We should add a "inference='auto'" or "inference=True" option to all estimators, which would use some default option for inference. For instance, this could be bootstrap or None (if we want to disallow inference in some estimators that we think bootstrap will be deceiving) or the more tailored methods.

This will allow the user to not require to know which one is the most appropriate method. Also it would allow one to not have to change the fit call whenever they change the estimator class.

[arose13]

Hey @vasilismsr, for even medium sized datasets or complex models like Random Forests or GBMs automatically enabling inference to do something, worse case bootstrap can be prohibitively expensive computational.

I guess what I'm saying is, if the user doesn't know whether a particular inference method paired with their chosen models would be expensive or it, it should not be automated for them because from the user point of view it will look like the process gets stuck on the .fit() call.

Obviously please correct me if I'm wrong.

[arose13]

For example in my package which has this package as a dependency I replace the inference kwarg with bootstrap_inference argument which accepts a Boolean and pick the appropriate method internally

[vasilismsr]

Thanks @arose13 ! Indeed, maybe having bootstrap as a default is a bad idea. Though we have some particular subclasses where inference is at no computational cost and there is an obvious default method. For instance, in LinearDMLCateEstimator setting inference='statsmodels' is the obvious option and has no computational overhead. Similarly, for SparseLinearDMLCateEstimator it is inference='debiasedlasso' and for ForestDMLCateEstimator it is inference='blb'. Similarly for the corresponding DRLearner and DRIV estimators.

Would then a solution of the form: inference='auto', by default means for most classes inference=None and for these classes where it is clear what the prefered method is, we set it to that? Would that break the dependency in your package?