-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add hive.metastore.glue.skip-archive
config option
#23817
base: master
Are you sure you want to change the base?
Conversation
/test-with-secrets sha=5d538527d13412ac48610f78f48d692a49480102 |
The CI workflow run with tests that require additional secrets has been started: https://github.com/trinodb/trino/actions/runs/11384047749 |
5d53852
to
831245a
Compare
...in/trino-hive/src/main/java/io/trino/plugin/hive/metastore/glue/GlueHiveMetastoreConfig.java
Outdated
Show resolved
Hide resolved
...ino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/glue/IcebergGlueCatalogModule.java
Outdated
Show resolved
Hide resolved
...ino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/glue/IcebergGlueCatalogModule.java
Outdated
Show resolved
Hide resolved
...rg/src/test/java/io/trino/plugin/iceberg/catalog/glue/TestIcebergGlueCatalogSkipArchive.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
few minor comments/question otherwise lgtm
...ino-hive/src/main/java/io/trino/plugin/hive/metastore/glue/GlueHiveExecutionInterceptor.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/metastore/glue/GlueMetastoreModule.java
Outdated
Show resolved
Hide resolved
plugin/trino-hive/src/main/java/io/trino/plugin/hive/metastore/glue/GlueMetastoreModule.java
Outdated
Show resolved
Hide resolved
831245a
to
e8f81a1
Compare
...ino-hive/src/main/java/io/trino/plugin/hive/metastore/glue/GlueHiveExecutionInterceptor.java
Show resolved
Hide resolved
...trino-hive/src/main/java/io/trino/plugin/hive/metastore/glue/v1/GlueHiveMetastoreConfig.java
Outdated
Show resolved
Hide resolved
...hive/src/test/java/io/trino/plugin/hive/metastore/glue/TestGlueHiveMetastoreSkipArchive.java
Outdated
Show resolved
Hide resolved
...hive/src/test/java/io/trino/plugin/hive/metastore/glue/TestGlueHiveMetastoreSkipArchive.java
Outdated
Show resolved
Hide resolved
e8f81a1
to
f7a47e3
Compare
When set to true, updating tables in Glue does not create an archived version of the table. Glue tables are updated during INSERT operations, so this option can be used to avoid reaching the limit of table versions, when executing a large number of such operations.
f7a47e3
to
3c3d1dd
Compare
/test-with-secrets sha=3c3d1dd43e6e3cbd87550ce038b5a288b627ac6f |
The CI workflow run with tests that require additional secrets has been started: https://github.com/trinodb/trino/actions/runs/11515725284 |
All green! |
return skipArchive; | ||
} | ||
|
||
@Config("iceberg.glue.skip-archive") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove also from the metastores.md
if the decision is to remove the property.
Are we talking about a "breaking change" if we remove the property?
Description
When set to true, updating tables in Glue does not create an archived
version of the table. Glue tables are updated during INSERT operations,
so this option can be used to avoid reaching the limit of table
versions, when executing many such operations.
In the second commit, I also added it to the legacy v1 Glue Metastore. I don't think it's needed there, it might help when migrating Iceberg from v1 Glue to the latest one, at least to avoid binding issues in Guice modules.
Additional context and related issues
This is modeled after #14336
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
(x) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: