Skip to content

Commit

Permalink
Update assessment.md - spark.catalog.x guidance needed updating (#1708
Browse files Browse the repository at this point in the history
)

The recent release of DBR 14+ make `spark.catalog.x` functions 'safe' to
run on shared compute so updating guidance.

## Changes
Update assessment readme, updating user guidance for shared compute
fixes to `spark.catalog.*` functions

### Functionality 

- [x] added relevant user documentation
- [ ] added new CLI command
- [ ] modified existing command: `databricks labs ucx ...`
- [ ] added a new workflow
- [ ] modified existing workflow: `...`
- [ ] added a new table
- [ ] modified existing table: `...`

### Tests
<!-- How is this tested? Please see the checklist below and also
describe any other relevant tests -->

- [x] manually tested
- [ ] added unit tests
- [ ] added integration tests
- [ ] verified on staging environment (screenshot attached)
  • Loading branch information
dmoore247 authored May 17, 2024
1 parent ee8fffc commit 9107005
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/assessment.md
Original file line number Diff line number Diff line change
Expand Up @@ -519,13 +519,13 @@ The Databricks ML Runtime is not supported on Shared Compute mode clusters. Reco

### AF301.1 - spark.catalog.x

The `spark.catalog.` pattern was found. Commonly used functions in spark.catalog, such as tableExists, listTables, setDefault catalog are not allowed/whitelisted on shared clusters due to security reasons. `spark.sql("<sql command>)` may be a better alternative.
The `spark.catalog.` pattern was found. Commonly used functions in spark.catalog, such as tableExists, listTables, setDefault catalog are not allowed/whitelisted on shared clusters due to security reasons. `spark.sql("<sql command>)` may be a better alternative. DBR 14.1 and above have made these commands available. Upgrade your DBR version.

[[back to top](#migration-assessment-report)]

### AF301.2 - spark.catalog.x (spark._jsparkSession.catalog)

The `spark._jsparkSession.catalog` pattern was found. Commonly used functions in spark.catalog, such as tableExists, listTables, setDefault catalog are not allowed/whitelisted on shared clusters due to security reasons. `spark.sql("<sql command>)` may be a better alternative.
The `spark._jsparkSession.catalog` pattern was found. Commonly used functions in spark.catalog, such as tableExists, listTables, setDefault catalog are not allowed/whitelisted on shared clusters due to security reasons. `spark.sql("<sql command>)` may be a better alternative. The corresponding `spark.catalog.x` methods may work on DBR 14.1 and above.

[[back to top](#migration-assessment-report)]

Expand Down

0 comments on commit 9107005

Please sign in to comment.