Adds rollup runner and integration tests #341
Conversation
| // TODO: Scenario: The rollup job is finished, but I (the user) want to redo it all again | ||
| /* | ||
| * TODO situations: | ||
| * There is a rollup.metadataID and doc but theres no job in target index? |
There was a problem hiding this comment.
Just a heads up, I believe we do recover from this scenario now, however, it's not exactly as the TODO message expects:
- During
isJobValid, we'll validate source and target index - If we get the condition in this scenario, where rollup.metadataID and doc exist but there is no job in target index, we resolve it by updating the mappings for the target index and add the job in
So we recover from it but neither start over or move to FAILED, I suppose it would recover silently and continue from wherever the metadata has left off.
We can go over this behavior as part of reviewing retry logic later on to discuss if it's what we want or not.
| logger.info("RollupMetadataException being thrown", e) | ||
| throw e | ||
| } catch (e: Exception) { | ||
| // TODO: Should update metadata and disable job here instead of allowing the rollup to keep going |
There was a problem hiding this comment.
This TODO should also be a candidate for inclusion whenever we work on retry logic.
Codecov Report
@@ Coverage Diff @@
## master #341 +/- ##
============================================
+ Coverage 71.06% 75.92% +4.86%
- Complexity 1219 1296 +77
============================================
Files 181 181
Lines 6448 6538 +90
Branches 984 1004 +20
============================================
+ Hits 4582 4964 +382
+ Misses 1377 1040 -337
- Partials 489 534 +45 Continue to review full report at Codecov.
|
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.