Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only test dimensions that are changed #48

Closed
joshtemple opened this issue Jul 10, 2019 · 1 comment
Closed

Only test dimensions that are changed #48

joshtemple opened this issue Jul 10, 2019 · 1 comment
Assignees
Labels
Enhancement New feature or request Performance Making Spectacles faster or more efficient Validator: SQL

Comments

@joshtemple
Copy link
Collaborator

joshtemple commented Jul 10, 2019

Now that we have the ability to parse LookML in Python with lkml, we have the ability to compare the differences in the parsed tree and only run Spectacles against those modified dimensions.

I'm imagining running git diff --name-only $TARGET_BRANCH to determine which files were changed (it doesn't look like Circle has a target branch environment variable in its context but it is a highly requested issue).

For each changed file, we would parse the LookML on the source and target branches and compare the resulting trees to understand which dimensions were changed.

Last, we would need a way to pass specific dimensions to Spectacles via the model selector (might require changes to requirements in #21). Fonz then would query only the modified dimensions.

This could also interact with caching in #22. You might want to run the unchanged dimensions using the cached queries instead of rebuilding them.

@joshtemple joshtemple added the Enhancement New feature or request label Jul 10, 2019
@DylanBaker
Copy link
Collaborator

DylanBaker commented Jul 24, 2019

This all sounds good to me.

Last, we would need a way to pass specific dimensions to Spectacles via the model selector (might require changes to requirements in #21). Spectacles then would query only the modified dimensions.

I agree it might require some changes there. I can post comments on #21, but my gut feel is simply to extend the model selection with another '.'? i.e. model would be model.*, explore would be model.explore.* and dimension would be model.explore.dimension? It's possible that gets too verbose?

One thing I think we would probably need to do is also run any dimensions that reference the changed dimensions? I think we could also pull that from the parser?

@joshtemple joshtemple added the Performance Making Spectacles faster or more efficient label Jun 23, 2020
@joshtemple joshtemple self-assigned this Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Performance Making Spectacles faster or more efficient Validator: SQL
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants