Daily spellbook sub project #6027
Conversation
daily_spellbook/packages.yml
Outdated
| - package: dbt-labs/codegen | ||
| version: 0.12.1 | ||
| - package: starburstdata/trino_utils | ||
| version: 0.6.0 |
There was a problem hiding this comment.
these are new? Should we also add them to the main packages.yml?
There was a problem hiding this comment.
Yes probably, @jeff-dude added these files but we can update main in a separate PR
There was a problem hiding this comment.
these were added during @couralex6 discovery of subprojects during tokens build. i believe codegen was intended to help the process of moving files around, but not actively used? then trino_utils is just trino-specific dbt_utils. we can just add per subproject as needed. as of now, i'm not sure we actually use any from there
There was a problem hiding this comment.
maybe i'll remove from this new subproject, to ensure it doesn't break anything without
| chainlink: | ||
| +schema: chainlink | ||
| arbitrum: | ||
| +schema: chainlink_arbitrum | ||
| avalanche_c: | ||
| +schema: chainlink_avalanche_c | ||
| base: | ||
| +schema: chainlink_base | ||
| bnb: | ||
| +schema: chainlink_bnb | ||
| ethereum: | ||
| +schema: chainlink_ethereum | ||
| fantom: | ||
| +schema: chainlink_fantom | ||
| gnosis: | ||
| +schema: chainlink_gnosis | ||
| optimism: | ||
| +schema: chainlink_optimism | ||
| polygon: | ||
| +schema: chainlink_polygon |
There was a problem hiding this comment.
just noting that we will want to remove these soon and put in models. just a lot of models to do. we can wait until the team refactors the code into macros (already underway)
There was a problem hiding this comment.
Yeah agree, was surprised to see these in the project file
|
i'm going to push a commit to try to remove those extra packages for now, to prove they aren't needed. we're also in a tough spot here, where CI won't run because the changes to CI are in the same PR. |
actually, looks like the workflow does run, but no models kicked off? |
jeff-dude
added
WIP
|
mentioned in DMs with alan, but we should double check the downstream impacts of moving these. for instance, what else in spellbook reads from any of these as a |
This is because I ran the compile locally and uploaded the manifest. Once merged it will automatically do a full refresh in DBT cloud. |
This PR initiates the a new sub project for models that are going to be run daily.
I also added the necessary CI steps,and (generated and) uploaded a manifest to S3 so that CI works.