Fix calc_dataset_view join with label values #2202
Conversation
lampajr
force-pushed
the
fix_calc_dataset_view
branch
from
274d003 to
751f5a3
Compare
| // ensure the recalculation is removed, with this approach we should guarantee | ||
| // it gets removed even if transaction-level exception occurs, e.g., timeout | ||
| Util.registerTxSynchronization(tm, txStatus -> recalculations.remove(testId, status)); |
There was a problem hiding this comment.
@johnaohara I also had to remove this because the run transformation is executed async making use of mediator.executeBlocking(.. therefore this recalculations.remove was getting called immediately before returning this method but much before the recalculation was completed which is wrong.. and it does not prevent multiple recalculation on the same test to happen concurrently
There was a problem hiding this comment.
I think we should be fairly safe even without that check as we are wrapping the mediator.transform with try-finally that should ensure the finally clause to get called even if transaction-level exceptions are thrown:
mediator.executeBlocking(() -> {
int newDatasets = 0;
try {
newDatasets = mediator.transform(runId, true);
} finally {
synchronized (status) {
status.finished++;
status.datasets += newDatasets;
if (status.finished == status.totalRuns) {
recalculations.remove(testId, status);
}
}
}
});
Signed-off-by: Andrea Lamparelli <a.lamparelli95@gmail.com>
lampajr
force-pushed
the
fix_calc_dataset_view
branch
from
751f5a3 to
1d550ce
Compare
| WITH view_agg AS ( | ||
| SELECT | ||
| vc.view_id, vc.id as vcid, array_agg(DISTINCT label.id) as label_ids, jsonb_object_agg(label.name, lv.value) as value FROM dataset_schemas ds | ||
| JOIN label ON label.schema_id = ds.schema_id | ||
| JOIN viewcomponent vc ON vc.labels ? label.name | ||
| JOIN label_values lv ON lv.label_id = label.id AND lv.dataset_id = ds.dataset_id | ||
| WHERE ds.dataset_id = datasetId | ||
| AND vc.view_id IN (SELECT view.id FROM view JOIN dataset ON view.test_id = dataset.testid WHERE dataset.id = datasetId) | ||
| GROUP BY vc.view_id, vcid | ||
| ) |
There was a problem hiding this comment.
The changes are:
- Added
lv.dataset_id = ds.dataset_idin the JOIN condition on label_values - Added
AND vc.view_id IN (SELECT view.id FROM view JOIN dataset ON view.test_id = dataset.testid WHERE dataset.id = datasetId)to ensure we are not including view_components coming from other views!
|
I discovered that the Fixed it by adding |
Fixes Issue
Fixes #2201
Check List (Check all the applicable boxes)