Confidence Intervals for ATE

[bradyneal]

I can estimate CATEs and their confidence intervals easily enough using estimator.effect(X=X, T0=T0, T1=T1) and estimator.effect_interval(X=X, T0=T0, T1=T1). However, what about (unconditional) averages treatment effects (ATEs)? I can estimate the ATE by just taking a mean over the CATEs estimator.effect(X=X, T0=T0, T1=T1).mean(), but I don't believe just taking the mean over the CATE intervals will give me valid confidence intervals for the ATE (feel free to correct me if I'm wrong or if this mean over CATE intervals gives a strictly more conservative interval that what I would get if I were to directly bootstrap ATE intervals). Is there functionality in EconML to get confidence intervals of the ATE?

[kbattocchi]

Most of our estimators support passing None for X (you'll need to use this with both fit and effect), which should give you an estimate of the ATE.

[kbattocchi]

Perhaps @vasilismsr has other thoughts on whether there are more reasonable aggregation techniques given the CATEs, or other ideas, though.

[bradyneal]

Ohh, it's because I'm using econml.metalearners, which don't support the distinction between W and X in their fit methods (#190), huh?

[vsyrgkanis]

Yeap. Best bet for ATE is either: LinearDRLearner with X=None or LinearDMLCateEstimator with X=None. The former might have higher variance but makes fewer assumptions, the latter might be more stable but makes a “no effect heterogeneity” assumption.

[vsyrgkanis]

Also for any estimator, we also offer:
est.effect_inference(X).population_summary()
Which also gives CIs for the ATE on the population. Though these are based on a “conservative” formula so might by slightly larger than the nominal ones.

Albeit this functionality is yet not supported for “inference=bootstrap”, which will be the case for metalearners. It will very soon be added though as a functionality.

[bradyneal]

Gotcha 👍. Any rough timeline for when either of the following will be supported:

1. The metalearners' fit() will support the W argument
2. effect_inference() will not error when inference='bootstrap' is passed to fit()?

[kbattocchi]

We've got an issue filed for 1 (#190), but as far as I'm aware no immediate plans to do it; for 2 there is a PR currently in progress (#236) that should address it and we are planning to complete it and create a new release by the end of next week.

[bradyneal]

Gotcha. Thanks! And thanks for the great Python package.