Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the bundle_uuid helper function for templates #1947

Merged
merged 2 commits into from
Dec 2, 2024
+34 −0
Merged

Add the bundle_uuid helper function for templates #1947

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f7c585a
Add the bundle_uuid helper function
shreyas-goenka Dec 1, 2024
ea0b443
-
shreyas-goenka Dec 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions bundle/config/bundle.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,8 @@ type Bundle struct {

// Databricks CLI version constraints required to run the bundle.
DatabricksCliVersion string `json:"databricks_cli_version,omitempty"`

// A stable generated UUID for the bundle. This is normally serialized by
// Databricks first party template when a user runs bundle init.
Uuid string `json:"uuid,omitempty"`
}
10 changes: 10 additions & 0 deletions libs/template/helpers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,13 @@ var cachedUser *iam.User
var cachedIsServicePrincipal *bool
var cachedCatalog *string

// UUID that is stable for the duration of the template execution. This can be used
// to populate the `bundle.uuid` field in databricks.yml by template authors.
//
// It's automatically logged in our telemetry logs when `databricks bundle init`
// is run and can be used to attribute DBU revenue to bundle templates.
var bundleUuid = uuid.New().String()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To confirm, this line will only run once per bundle initialization? Is that when this class is instantiated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this line will only run once per bundle initialization?

Yeah. This PR also adds a unit test to confirm this behaviour when materializing the template.

Is that when this class is instantiated?

There's no class associated with this variable. According to the Go specification the global variables are initialized before the main thread of the go program executes. Specifically it's during the package initialization.

see: https://arc.net/l/quote/vvjgvjzs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this not a global? Looks like it can be scoped to loadHelpers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make this not a global? Looks like it can be scoped to loadHelpers.

We need it to be accessible outside loadHelpers helpers as well once we populate it in the telemetry payload.

Also, while loadHelpers would work in practice today, if in the future we make multiple loadHelpers function calls to materialize a template we could end up changing the value of the UUID being serialized accidentally

I can switch over to the cachedBundleUuid *string pattern like we do for API calls if that's something you'd prefer.


func loadHelpers(ctx context.Context) template.FuncMap {
w := root.WorkspaceClient(ctx)
return template.FuncMap{
Expand All @@ -57,6 +64,9 @@ func loadHelpers(ctx context.Context) template.FuncMap {
"uuid": func() string {
return uuid.New().String()
},
"bundle_uuid": func() string {
return bundleUuid
},
// A key value pair. This is used with the map function to generate maps
// to use inside a template
"pair": func(k string, v any) pair {
Expand Down
18 changes: 18 additions & 0 deletions libs/template/helpers_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,24 @@ func TestTemplatePrintStringWithoutProcessing(t *testing.T) {
assert.Equal(t, `{{ fail "abc" }}`, cleanContent)
}

func TestTemplateBundleUuidFunction(t *testing.T) {
ctx := context.Background()

ctx = root.SetWorkspaceClient(ctx, nil)
helpers := loadHelpers(ctx)
r, err := newRenderer(ctx, nil, helpers, os.DirFS("."), "./testdata/bundle-uuid/template", "./testdata/bundle-uuid/library")
require.NoError(t, err)

err = r.walk()
assert.NoError(t, err)

assert.Len(t, r.files, 2)
c1 := strings.Trim(string(r.files[0].(*inMemoryFile).content), "\n\r")
assert.Equal(t, strings.Repeat(bundleUuid, 3), c1)
c2 := strings.Trim(string(r.files[1].(*inMemoryFile).content), "\n\r")
assert.Equal(t, strings.Repeat(bundleUuid, 5), c2)
shreyas-goenka marked this conversation as resolved.
Show resolved Hide resolved
}

func TestTemplateRegexpCompileFunction(t *testing.T) {
ctx := context.Background()

Expand Down
Empty file added libs/template/testdata/bundle-uuid/library/.gitkeep
View file
Open in desktop
Empty file.
1 change: 1 addition & 0 deletions libs/template/testdata/bundle-uuid/template/bar.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{bundle_uuid}}{{bundle_uuid}}{{bundle_uuid}}
1 change: 1 addition & 0 deletions libs/template/testdata/bundle-uuid/template/foo.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{bundle_uuid}}{{bundle_uuid}}{{bundle_uuid}}{{bundle_uuid}}{{bundle_uuid}}
Loading