[HUDI-8395] Fix metaClient handling when running upgrade or downgrade #12224
Conversation
codope
force-pushed
the
hudi-8395-upgrade-fix
branch
from
815d36a
to
4c73b37
Compare
| @@ -1447,11 +1457,11 @@ protected void tryUpgrade(HoodieTableMetaClient metaClient, Option<String> insta | |||
| tableServiceClient.rollbackFailedWrites(pendingRollbacks, true, true); | |||
| } | |||
|
|
|||
| new UpgradeDowngrade(metaClient, config, context, upgradeDowngradeHelper) | |||
| .run(HoodieTableVersion.current(), instantTime.orElse(null)); | |||
|
|
|||
| metaClient.reloadActiveTimeline(); | |||
There was a problem hiding this comment.
Can we just move the timeline reload out of the tryUpgrade method and do it outside, so that we can eliminate the tricks of this atomic reference get/set.
There was a problem hiding this comment.
Updated now, please check. I have made tryUpgrade return a boolean and if upgrade happened then reload the meta client outside tryUpgrade but still within the transaction.
There was a problem hiding this comment.
Add a new method reloadTableConfig to address the problem.
codope
force-pushed
the
hudi-8395-upgrade-fix
branch
from
4c73b37
to
546cbcd
Compare
codope
force-pushed
the
hudi-8395-upgrade-fix
branch
from
546cbcd
to
d05bd2b
Compare
| UpgradeDowngrade upgradeDowngrade = | ||
| new UpgradeDowngrade(metaClient, config, context, upgradeDowngradeHelper); | ||
|
|
||
| if (upgradeDowngrade.needsUpgradeOrDowngrade(HoodieTableVersion.current())) { | ||
| metaClient = HoodieTableMetaClient.reload(metaClient); |
There was a problem hiding this comment.
It looks like this line is problematic, if we always use the same reference passed in, it should work, that is: use metaClient.reloadActiveTimeline(); here instead of replace the metaClient reference.
There was a problem hiding this comment.
Actually it is not the timeline, but the table config that is not refreshed.
| return fromJsonString(fromUTF8Bytes(convertCommitMetadataToJsonBytes(deserializeCommitMetadata(bytes), org.apache.hudi.avro.model.HoodieCommitMetadata.class)), clazz); | ||
| } catch (Exception e) { | ||
| // fall back to the alternative method (0.x) | ||
| LOG.warn("Primary method failed; trying alternative deserialization method.", e); |
Change Logs
After debugging, I found that upgrade path is invoked but the
metaClientis not reloaded after that. Simply reloading intryUpgradeis not enough because we call that in a lambda. So, this PR fixes the issue as follows:doInitTable()to return the potentially reloadedmetaClient, allowinginitTable()to access the updatedmetaClientreference after an upgrade.HoodieCommitMetadata.fromBytes()method with a fallback mechanism for JSON deserialization given that commit metadata serialization changed from json to avro in 1.0.Tested manually by running the same steps as mentioned in HUDI-8395.
Impact
Refactor metaClient handling during upgrade/downgrade, and implement fallback in JSON deserialization.
Risk level (write none, low medium or high below)
medium
Documentation Update
Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".
ticket number here and follow the instruction to make
changes to the website.
Contributor's checklist