Implement tests on inference scenarios for models #1082
Conversation
… on subset of segments
…in during meeting new segments
…values has changed
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## inference-v2.1 #1082 +/- ##
=================================================
Coverage ? 86.50%
=================================================
Files ? 164
Lines ? 8887
Branches ? 0
=================================================
Hits ? 7688
Misses ? 1199
Partials ? 0 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
🚀 Deployed on https://deploy-preview-1082--etna-docs.netlify.app |
| ), | ||
| ], | ||
| ) | ||
| def test_forecast_new_segments(self, model, transforms, example_tsds): |
There was a problem hiding this comment.
I guess we can use lags in all the test case to simplify parametrization
| with pytest.raises(KeyError, match="Unknown category"): | ||
| self._test_forecast_new_segments(example_tsds, model, transforms, train_segments=["segment_1"]) | ||
|
|
||
| @to_be_fixed(raises=NotImplementedError, match="Per-segment models can't make predictions on new segments") |
There was a problem hiding this comment.
How we are going to fix it?
There was a problem hiding this comment.
I don't really know how. We can potentially make it possible for naive models, but for other models I don't think we can do smth like that.
There was a problem hiding this comment.
Let's try to make it possible for naive models, if don't come to simple solution may be it is better to skip it
Before submitting (must do checklist)
Proposed Changes
Look #1076. Also added tests on
MLPModel.Closing issues
Closes #1076.