Split migrate-groups workflow into three different stages for reliability
#442
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## main #442 +/- ##
==========================================
- Coverage 82.17% 81.23% -0.94%
==========================================
Files 31 31
Lines 2984 3027 +43
Branches 564 576 +12
==========================================
+ Hits 2452 2459 +7
- Misses 407 443 +36
Partials 125 125
|
Merged
nfx
added a commit
that referenced
this pull request
Oct 12, 2023
# Version changelog ## 0.4.0 * Added exception handling for secret scope not found. ([#418](#418)). * Added a crawler for creating an inventory of Azure Service Principals ([#326](#326)). * Added check if account group already exists during failure recovery ([#446](#446)). * Added checking for index out of range. ([#429](#429)). * Added hyperlink to UCX releases in the main readme ([#408](#408)). * Added integration test to check backup groups get deleted ([#387](#387)). * Added logging of errors during threadpool operations. ([#376](#376)). * Added recovery mode for workspace-local groups from temporary groups ([#435](#435)). * Added support for migrating Legacy Table ACLs from workspace-local to account-level groups ([#412](#412)). * Added detection for installations of unreleased versions ([#399](#399)). * Decoupled `PermissionsManager` from `GroupMigrationToolkit` ([#407](#407)). * Enabled debug logging for every job task run through a file, which is accessible from both workspace UI and Databricks CLI ([#426](#426)). * Ensured that table exists, even when crawlers produce zero records ([#373](#373)). * Extended test suite for HMS->HMS TACL migration ([#439](#439)). * Fixed handling of secret scope responses ([#431](#431)). * Fixed `crawl_permissions` task to respect 'workspace_start_path' config ([#444](#444)). * Fixed broken logic in `parallel` module and applied hardened error handling design for parallel code ([#405](#405)). * Fixed codecov.io reporting ([#403](#403)). * Fixed integration tests for crawlers ([#379](#379)). * Improved README.py and logging messages ([#433](#433)). * Improved cleanup for workspace backup groups by adding more retries on errors ([#375](#375)). * Improved dashboard queries to show unsupported storage types. ([#398](#398)). * Improved documentation for readme notebook ([#257](#257)). * Improved test coverage for installer ([#371](#371)). * Introduced deterministic `env_or_skip` fixture for integration tests ([#396](#396)). * Made HMS & UC fixtures return `CatalogInfo`, `SchemaInfo`, and `TableInfo` ([#409](#409)). * Merge `workspace_access.Crawler` and `workspace_access.Applier` interfaces to `workspace_access.AclSupport` ([#436](#436)). * Moved examples to docs ([#404](#404)). * Properly isolated integration testing for workflows on an existing shared cluster ([#414](#414)). * Removed thread pool for any IAM Group removals and additions ([#394](#394)). * Replace plus char with minus in version tag for GCP dev installation of UCX ([#420](#420)). * Run integration tests on shared clusters for a faster devloop ([#397](#397)). * Show difference between serverless and PRO warehouses during installation ([#385](#385)). * Split `migrate-groups` workflow into three different stages for reliability ([#442](#442)). * Use groups instead of usernames in code owners file ([#389](#389)).
FastLee
pushed a commit
that referenced
this pull request
Oct 25, 2023
FastLee
pushed a commit
that referenced
this pull request
Oct 25, 2023
# Version changelog ## 0.4.0 * Added exception handling for secret scope not found. ([#418](#418)). * Added a crawler for creating an inventory of Azure Service Principals ([#326](#326)). * Added check if account group already exists during failure recovery ([#446](#446)). * Added checking for index out of range. ([#429](#429)). * Added hyperlink to UCX releases in the main readme ([#408](#408)). * Added integration test to check backup groups get deleted ([#387](#387)). * Added logging of errors during threadpool operations. ([#376](#376)). * Added recovery mode for workspace-local groups from temporary groups ([#435](#435)). * Added support for migrating Legacy Table ACLs from workspace-local to account-level groups ([#412](#412)). * Added detection for installations of unreleased versions ([#399](#399)). * Decoupled `PermissionsManager` from `GroupMigrationToolkit` ([#407](#407)). * Enabled debug logging for every job task run through a file, which is accessible from both workspace UI and Databricks CLI ([#426](#426)). * Ensured that table exists, even when crawlers produce zero records ([#373](#373)). * Extended test suite for HMS->HMS TACL migration ([#439](#439)). * Fixed handling of secret scope responses ([#431](#431)). * Fixed `crawl_permissions` task to respect 'workspace_start_path' config ([#444](#444)). * Fixed broken logic in `parallel` module and applied hardened error handling design for parallel code ([#405](#405)). * Fixed codecov.io reporting ([#403](#403)). * Fixed integration tests for crawlers ([#379](#379)). * Improved README.py and logging messages ([#433](#433)). * Improved cleanup for workspace backup groups by adding more retries on errors ([#375](#375)). * Improved dashboard queries to show unsupported storage types. ([#398](#398)). * Improved documentation for readme notebook ([#257](#257)). * Improved test coverage for installer ([#371](#371)). * Introduced deterministic `env_or_skip` fixture for integration tests ([#396](#396)). * Made HMS & UC fixtures return `CatalogInfo`, `SchemaInfo`, and `TableInfo` ([#409](#409)). * Merge `workspace_access.Crawler` and `workspace_access.Applier` interfaces to `workspace_access.AclSupport` ([#436](#436)). * Moved examples to docs ([#404](#404)). * Properly isolated integration testing for workflows on an existing shared cluster ([#414](#414)). * Removed thread pool for any IAM Group removals and additions ([#394](#394)). * Replace plus char with minus in version tag for GCP dev installation of UCX ([#420](#420)). * Run integration tests on shared clusters for a faster devloop ([#397](#397)). * Show difference between serverless and PRO warehouses during installation ([#385](#385)). * Split `migrate-groups` workflow into three different stages for reliability ([#442](#442)). * Use groups instead of usernames in code owners file ([#389](#389)).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.