Variables in project.yml that use jinja templating are not properly rendering when referenced in packages

[randypitcherii]

Describe the bug

When working with the dbt_artifacts package, I found that a variable I defined with jinja:
``

was not rendering correctly when referenced in the package, but it did render correctly when I referenced for models / sources in my local project.

Steps To Reproduce

1. Add the dbt_artifacts package to a dbt project (I used dbt core v0.20.0)
2. run dbt deps
3. add the following to your vars block in your dbt_project.yml:

vars:
    dbt_artifacts_schema: "{{ target.schema }}"

4. run dbt run-operation create_artifact_resources and observe that the schema dbt attempts to use is the raw {{ target.schema }} string rather than the expected value of target.schema
5. Furthermore, observe that in a local model you may reference this variable using {{var('dbt_artifacts_schema')}} and the proper rendering occurs.

Expected behavior

I expect installed dependencies that reference variables from a local project to receive the properly jinja-rendered values rather than the raw jinja templating string.

System information

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

The output of dbt --version:

installed version: 0.20.0
   latest version: 0.20.0

Up to date!

Plugins:
  - bigquery: 0.20.0
  - snowflake: 0.20.0
  - redshift: 0.20.0
  - postgres: 0.20.0

The operating system you're using:
MacOS Big Sur

The output of python --version:
Python 3.8.6

[jtcohen6]

@randypitcherii Thanks for opening! This is very similar to #3105, and both of them sum up to:

• The unavailability of vars elsewhere in dbt_project.yml is not a good thing, and we should adjust the reading order accordingly. (There's a parallel suggestion in Defining vars, folder-level configs outside dbt_project.yml #2955 (comment).)
• The rendering context available to project-defined vars is not clear—in fact, it doesn't exist! The project vars block is not rendered today. The Jinja string itself is what's stored:

ipdb> config.__dict__['_active_project'].vars.__dict__
{'vars': {'dbt_artifacts_schema': '{{ target.schema }}'}}

Depending on where you call those variables, though, they may or may not go through extra Jinja rendering in those spots. For instance, all the Jinja code in model files is rendered twice. I have to imagine that the var is extra-rendered as an unintended side effect.

At the same time, I appreciate that Jinja rendering is the anticipated behavior here. I think it would be reasonable to offer a limited, clearly documented rendering context for project-defined vars: env vars, target values (from profiles.yml) and perhaps CLI-defined --vars (if that isn't too confusing).

I believe that this issue is fundamentally about that misalignment in expectations around vars rendering. The piece around packages is a complicating but ultimately unrelated detail. I'm going to close this in favor of #3105, which proposes the remedy above. It's not on the pre-v1.0 roadmap, but certainly a change we want to make over time for the benefit of everyone who depends on complex var- and target- based arrangements.

[randypitcherii]

Awesome! Thank you, @jtcohen6!