Fix reindex command and some es utils #35463
Conversation
we will not reindex the sub indices as we can create them on the fly after we have new indices ready
AmitPhulera
changed the title
| @@ -31,6 +32,12 @@ def to_json(doc): | |||
| found_indices = {} | |||
| es_doc_type = None | |||
| for index in iter_index_cnames(): | |||
| if not settings.IS_SAAS_ENVIRONMENT: | |||
There was a problem hiding this comment.
@gherceg I have been running in circles in refactoring transient utils and was not able to make significant progress on it so I realised by the time I will do that it would be good to fix the page for users.
| # Add doc_id field to the documents | ||
| script_parts.append(""" | ||
| if (!ctx._source.containsKey('doc_id')) { | ||
| ctx._source['doc_id'] = ctx._id; |
There was a problem hiding this comment.
Does this work if purge_ids is true?
It's referencing ctx._id, which may have been removed by ctx._source.remove('_id');? I'm not confident about what ctx._source.remove() does, so I could be wrong.
There was a problem hiding this comment.
Yes. It will work if purge_ids is true.
We are removing _id from ctx._source which is different from ctx._id. ctx._id is a read only property.
ctx._source.remove is basically a groovy style syntax to remove an object from a map. So it is just removing the attribute from the object.
Product Description
This PR started out as to fix a small bug that that I saw in the
elastic_sync_multiplexedcommand. But while reindexing India I came across couple other issues which were small enough to get it in the PR itself.So this PR essentially does three things -
batch_sizeparam was mistakenly removed fromelastic_sync_multiplexedduring a refactor, it was added back.Review by commit 🐡
Feature Flag
N/A
Safety Assurance
The changes are already covered by the tests where required and I have tested them locally. The change affects a reindex command so it has a very low blast radius.
Automated test coverage
QA Plan
N/A
Rollback instructions
Labels & Review