Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added databricks labs ucx move command #756

Merged
merged 47 commits into from
Jan 9, 2024
Merged

Added databricks labs ucx move command #756

merged 47 commits into from
Jan 9, 2024

Conversation

HariGS-DB
Copy link
Contributor

closes #675

HariGS-DB and others added 30 commits December 23, 2023 18:25
…air run of any failed workflows, like `assessment`, `migrate-groups` etc. (#724)
* Corrected unit test setup as per PR
[740](#740)
* Fixed corrupted state warning
* Added `databricks labs ucx repair-run --step ...` CLI command for
repair run of any failed workflows, like `assessment`, `migrate-groups`
etc. ([#724](#724)).
* Added `databricks labs ucx revert-migrated-table` command
([#729](#729)).
* Allow specifying a group list when group match options are used
([#725](#725)).
* Fixed installation issue when upgrading from an older version of the
tool and improve logs
([#740](#740)).
* Renamed summary panel from Failure Summary to Assessment Summary
([#733](#733)).
* Retry internal error when getting permissions and update legacy table
ACL documentation
([#728](#728)).
* Speedup installer execution
([#727](#727)).
Closes #330

- Added assessment step estimate dbfs root table size.
- This should help deciding whether the table is a good candidate for
cloning.
src/databricks/labs/ucx/cli.py Outdated Show resolved Hide resolved
src/databricks/labs/ucx/hive_metastore/table_migrate.py Outdated Show resolved Hide resolved
src/databricks/labs/ucx/hive_metastore/table_migrate.py Outdated Show resolved Hide resolved
src/databricks/labs/ucx/hive_metastore/table_migrate.py Outdated Show resolved Hide resolved
src/databricks/labs/ucx/hive_metastore/table_migrate.py Outdated Show resolved Hide resolved
tests/integration/hive_metastore/test_table_move.py Outdated Show resolved Hide resolved
tests/integration/hive_metastore/test_table_move.py Outdated Show resolved Hide resolved
tests/integration/hive_metastore/test_table_move.py Outdated Show resolved Hide resolved
tests/integration/hive_metastore/test_table_move.py Outdated Show resolved Hide resolved
Copy link
Contributor

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nfx nfx merged commit ff10e67 into main Jan 9, 2024
7 checks passed
@nfx nfx deleted the feature/uc_to_uc branch January 9, 2024 20:19
nfx added a commit that referenced this pull request Jan 12, 2024
* Added assessment step to estimate the size of DBFS root tables ([#741](#741)).
* Added `TableMapping` functionality to table migrate ([#752](#752)).
* Added `databricks labs ucx move` command to move tables and schemas between catalogs ([#756](#756)).
* Added functionality to determine migration method based on DBFS Root ([#759](#759)).
* Added `get_tables_to_migrate` functionality in the mapping module ([#755](#755)).
* Added retry and rate limit to rename workspace group operation and corrected rate limit for reflecting account groups to workspace ([#751](#751)).
* Adopted `databricks-labs-blueprint` library for common utilities to be reused in the other projects ([#758](#758)).
* Converted `RuntimeBackend` query executions exceptions to SDK exceptions ([#769](#769)).
* Fixed issue with missing users and temp groups after workspace-local groups migration and skip table when crawling table size if it does not exist anymore ([#770](#770)).
* Improved error handling by not failing group rename step if a group was removed from account before reflecting it to workspace ([#762](#762)).
* Improved error message inference from failed workflow runs ([#753](#753)).
* Moved `TablesMigrate` to a separate module ([#747](#747)).
* Reorganized assessment dashboard to increase readability ([#738](#738)).
* Updated databricks-sdk requirement from ~=0.16.0 to ~=0.17.0 ([#773](#773)).
* Verify metastore exists in current workspace ([#735](#735)).
@nfx nfx mentioned this pull request Jan 12, 2024
nfx added a commit that referenced this pull request Jan 12, 2024
* Added assessment step to estimate the size of DBFS root tables
([#741](#741)).
* Added `TableMapping` functionality to table migrate
([#752](#752)).
* Added `databricks labs ucx move` command to move tables and schemas
between catalogs
([#756](#756)).
* Added functionality to determine migration method based on DBFS Root
([#759](#759)).
* Added `get_tables_to_migrate` functionality in the mapping module
([#755](#755)).
* Added retry and rate limit to rename workspace group operation and
corrected rate limit for reflecting account groups to workspace
([#751](#751)).
* Adopted `databricks-labs-blueprint` library for common utilities to be
reused in the other projects
([#758](#758)).
* Converted `RuntimeBackend` query executions exceptions to SDK
exceptions ([#769](#769)).
* Fixed issue with missing users and temp groups after workspace-local
groups migration and skip table when crawling table size if it does not
exist anymore ([#770](#770)).
* Improved error handling by not failing group rename step if a group
was removed from account before reflecting it to workspace
([#762](#762)).
* Improved error message inference from failed workflow runs
([#753](#753)).
* Moved `TablesMigrate` to a separate module
([#747](#747)).
* Reorganized assessment dashboard to increase readability
([#738](#738)).
* Updated databricks-sdk requirement from ~=0.16.0 to ~=0.17.0
([#773](#773)).
* Verify metastore exists in current workspace
([#735](#735)).
FastLee added a commit that referenced this pull request Jan 19, 2024
closes #675

---------

Co-authored-by: Liran Bareket <lbareket@gmail.com>
Co-authored-by: Marcin Wojtyczka <wojtyczka.marcin@gmail.com>
Co-authored-by: prajin-29 <127273819+prajin-29@users.noreply.github.com>
Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com>
FastLee pushed a commit that referenced this pull request Jan 19, 2024
* Added assessment step to estimate the size of DBFS root tables
([#741](#741)).
* Added `TableMapping` functionality to table migrate
([#752](#752)).
* Added `databricks labs ucx move` command to move tables and schemas
between catalogs
([#756](#756)).
* Added functionality to determine migration method based on DBFS Root
([#759](#759)).
* Added `get_tables_to_migrate` functionality in the mapping module
([#755](#755)).
* Added retry and rate limit to rename workspace group operation and
corrected rate limit for reflecting account groups to workspace
([#751](#751)).
* Adopted `databricks-labs-blueprint` library for common utilities to be
reused in the other projects
([#758](#758)).
* Converted `RuntimeBackend` query executions exceptions to SDK
exceptions ([#769](#769)).
* Fixed issue with missing users and temp groups after workspace-local
groups migration and skip table when crawling table size if it does not
exist anymore ([#770](#770)).
* Improved error handling by not failing group rename step if a group
was removed from account before reflecting it to workspace
([#762](#762)).
* Improved error message inference from failed workflow runs
([#753](#753)).
* Moved `TablesMigrate` to a separate module
([#747](#747)).
* Reorganized assessment dashboard to increase readability
([#738](#738)).
* Updated databricks-sdk requirement from ~=0.16.0 to ~=0.17.0
([#773](#773)).
* Verify metastore exists in current workspace
([#735](#735)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add databricks labs ucx move command
5 participants