-
Notifications
You must be signed in to change notification settings - Fork 14
Description
Describe the bug
Using a variable or Jinja syntax in the variable for: snowplow__database
or snowplow__atomic_schema
does not expand when used with BigQuery.
Steps to reproduce
Using bigquery, within the DBT config put:
snowplow_unified:
snowplow__database: "{{ env_var('DBT_DATBASE', 'default') }}"
snowplow__atomic_schema: "{{ env_var('DBT_SCHEMA', 'default') }}"
NOTE: This does work for Snowflake, but not BigQuery.
Expected results
The models to execute successfully using the env_var that is used.
Actual results
I receive the following error:
22:18:45 6 of 11 START sql table model default.dbt_scratch.snowplow_unified_base_events_this_run [RUN]
22:18:46 Unhandled error while executing
400 GET https://bigquery.googleapis.com/bigquery/v2/projects/%7B%7B%20env_var('DBT_DATABASE',%20'default')%20%7D%7D/datasets/%7B%7B%20env_var('DBT_SCHEMA',%20'default')%20%7D%7D/tables/events?prettyPrint=false: Invalid resource name projects/{{ env_var('DBT_DATABASE', 'default') }}; Project id: {{ env_var('DBT_DATABASE', 'default') }}
22:18:46 6 of 11 ERROR creating sql table model default.dbt_scratch.snowplow_unified_base_events_this_run [ERROR in 1.42s]
Upon investigation, the base macro is returniogn the right sql. It looks like the error is within the get_context field macros when they call source('atomic', 'events`) . Because the underlying source file is not expanding the snowplow variables.
System information
The contents of your packages.yml
file:
packages:
- package: snowplow/snowplow_unified
version: 0.4.5
- package: snowplow/snowplow_utils
version: 0.16.8
- package: snowplow/snowplow_normalize
version: 0.3.5
Which database are you using dbt with?
- postgres
- redshift
- bigquery
- snowflake
- databricks
- other (specify: ____________)
The output of dbt --version
:
dbt Cloud CLI - 0.38.22 (1183c2abdb6003083b0fa91fcd89cd5feb25f9f7 2024-11-20T15:49:01Z)
The operating system you're using:
Mac OS Sequoia
The output of python --version
:
Python 3.8.18