Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly read Git metadata when running inside workspace #1945

Merged
merged 25 commits into from
Dec 5, 2024

Conversation

denik
Copy link
Contributor

@denik denik commented Nov 29, 2024

Changes

Since there is no .git directory in Workspace file system, we need to make an API call to api/2.0/workspace/get-status?return_git_info=true to fetch git the root of the repo, current branch, commit and origin.

Added new function FetchRepositoryInfo that either looks up and parses .git or calls remote API depending on env.

Refactor Repository/View/FileSet to accept repository root rather than calculate it. This helps because:

  • Repository is currently created in multiple places and finding the repository root is becoming relatively expensive (API call needed).
  • Repository/FileSet/View do not have access to current Bundle which is where WorkplaceClient is stored.

Tests

  • Tested manually by running "bundle validate --json" inside web terminal within Databricks env.
  • Added integration tests for the new API.

@denik denik temporarily deployed to test-trigger-is November 29, 2024 10:05 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is November 29, 2024 10:05 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is November 29, 2024 11:03 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is November 29, 2024 11:03 — with GitHub Actions Inactive
@denik denik changed the base branch from main to DECO-24125--move-git-load-to-initialize November 29, 2024 11:51
@denik denik marked this pull request as ready for review November 29, 2024 11:52
libs/git/info.go Outdated Show resolved Hide resolved
libs/git/info.go Outdated Show resolved Hide resolved
libs/git/info.go Outdated Show resolved Hide resolved
libs/git/info.go Outdated Show resolved Hide resolved
libs/git/info.go Outdated Show resolved Hide resolved
libs/git/info.go Outdated Show resolved Hide resolved
libs/git/info.go Show resolved Hide resolved
libs/git/info.go Outdated Show resolved Hide resolved
bundle/bundle.go Outdated Show resolved Hide resolved
@denik denik temporarily deployed to test-trigger-is November 29, 2024 16:44 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is November 29, 2024 16:45 — with GitHub Actions Inactive
@denik denik force-pushed the DECO-24125--move-git-load-to-initialize branch from 17f284e to 84d535a Compare December 1, 2024 20:28
@denik denik force-pushed the DECO-24125--fetch-git-details branch from 8729c19 to 05ff508 Compare December 1, 2024 20:30
@denik denik temporarily deployed to test-trigger-is December 1, 2024 20:30 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 1, 2024 20:30 — with GitHub Actions Inactive
bundle/bundle.go Outdated Show resolved Hide resolved
bundle/config/mutator/load_git_details.go Show resolved Hide resolved
libs/git/fileset.go Outdated Show resolved Hide resolved
libs/git/info.go Show resolved Hide resolved
libs/git/info.go Outdated Show resolved Hide resolved
libs/git/view.go Show resolved Hide resolved
Base automatically changed from DECO-24125--move-git-load-to-initialize to main December 2, 2024 09:54
@denik denik force-pushed the DECO-24125--fetch-git-details branch from 05ff508 to c3b02d9 Compare December 2, 2024 10:29
@denik denik temporarily deployed to test-trigger-is December 2, 2024 10:29 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 10:29 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 10:30 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 10:31 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 10:50 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 10:50 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 14:02 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 14:02 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 2, 2024 14:45 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 11:35 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 11:35 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 13:08 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 13:08 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 13:11 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 13:11 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 13:25 — with GitHub Actions Inactive
@denik denik temporarily deployed to test-trigger-is December 4, 2024 13:26 — with GitHub Actions Inactive
@denik denik requested review from pietern and removed request for ilia-db December 4, 2024 13:27
cmd/sync/sync.go Outdated Show resolved Hide resolved
@denik denik temporarily deployed to test-trigger-is December 5, 2024 08:53 — with GitHub Actions Inactive
Copy link

github-actions bot commented Dec 5, 2024

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

  • PR number: 1945
  • Commit SHA: 43da63bc50fd4f5a9b9da515db83ecd9249cdfe3

Checks will be approved automatically on success.

@denik denik temporarily deployed to test-trigger-is December 5, 2024 08:53 — with GitHub Actions Inactive
@eng-dev-ecosystem-bot
Copy link
Collaborator

Test Details: go/deco-tests/12176323188

@denik denik enabled auto-merge December 5, 2024 08:53
@denik denik added this pull request to the merge queue Dec 5, 2024
Merged via the queue into main with commit 0ad790e Dec 5, 2024
11 checks passed
@denik denik deleted the DECO-24125--fetch-git-details branch December 5, 2024 10:19
denik added a commit that referenced this pull request Dec 5, 2024
The git module now uses alternative implementation that uses os module + strings as path directly (#1945).

The remaining use case is also updated to use git.FindLeafInTree, as it's a better fit.
pietern added a commit that referenced this pull request Dec 5, 2024
**New features for Databricks Asset Bundles:**

This release adds support for managing Unity Catalog volumes as part of your bundle configuration.

Bundles:
 * Add DABs support for Unity Catalog volumes ([#1762](#1762)).
 * Support lookup by name of notification destinations ([#1922](#1922)).
 * Extend "notebook not found" error to warn about missing extension ([#1920](#1920)).
 * Skip sync warning if no sync paths are defined ([#1926](#1926)).
 * Add validation for single node clusters ([#1909](#1909)).
 * Fix segfault in bundle summary command ([#1937](#1937)).
 * Add the `bundle_uuid` helper function for templates ([#1947](#1947)).
 * Add default value for `volume_type` for DABs ([#1952](#1952)).
 * Properly read Git metadata when running inside workspace ([#1945](#1945)).
 * Upgrade TF provider to 1.59.0 ([#1960](#1960)).

Internal:
 * Breakout variable lookup into separate files and tests ([#1921](#1921)).
 * Add golangci-lint v1.62.2 ([#1953](#1953)).

Dependency updates:
 * Bump golang.org/x/term from 0.25.0 to 0.26.0 ([#1907](#1907)).
 * Bump github.com/Masterminds/semver/v3 from 3.3.0 to 3.3.1 ([#1930](#1930)).
 * Bump github.com/stretchr/testify from 1.9.0 to 1.10.0 ([#1932](#1932)).
 * Bump github.com/databricks/databricks-sdk-go from 0.51.0 to 0.52.0 ([#1931](#1931)).
github-merge-queue bot pushed a commit that referenced this pull request Dec 5, 2024
**New features for Databricks Asset Bundles:**

This release adds support for managing Unity Catalog volumes as part of
your bundle configuration.

Bundles:
* Add DABs support for Unity Catalog volumes
([#1762](#1762)).
* Support lookup by name of notification destinations
([#1922](#1922)).
* Extend "notebook not found" error to warn about missing extension
([#1920](#1920)).
* Skip sync warning if no sync paths are defined
([#1926](#1926)).
* Add validation for single node clusters
([#1909](#1909)).
* Fix segfault in bundle summary command
([#1937](#1937)).
* Add the `bundle_uuid` helper function for templates
([#1947](#1947)).
* Add default value for `volume_type` for DABs
([#1952](#1952)).
* Properly read Git metadata when running inside workspace
([#1945](#1945)).
* Upgrade TF provider to 1.59.0
([#1960](#1960)).

Internal:
* Breakout variable lookup into separate files and tests
([#1921](#1921)).
* Add golangci-lint v1.62.2
([#1953](#1953)).

Dependency updates:
* Bump golang.org/x/term from 0.25.0 to 0.26.0
([#1907](#1907)).
* Bump github.com/Masterminds/semver/v3 from 3.3.0 to 3.3.1
([#1930](#1930)).
* Bump github.com/stretchr/testify from 1.9.0 to 1.10.0
([#1932](#1932)).
* Bump github.com/databricks/databricks-sdk-go from 0.51.0 to 0.52.0
([#1931](#1931)).
denik added a commit that referenced this pull request Dec 9, 2024
The git module now uses alternative implementation that uses os module + strings as path directly (#1945).

The remaining use case is also updated to use git.FindLeafInTree, as it's a better fit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants