Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fold in conversion metrics from upstream releases #122

Merged
merged 13 commits into from
Nov 20, 2024

Conversation

fivetran-jamie
Copy link
Contributor

@fivetran-jamie fivetran-jamie commented Oct 30, 2024

PR Overview

This PR will address the following Issue/Feature:
Conversion support!

This PR will result in the following new package version:

v1.10.0

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

❗ Special Updates for Ad Reporting ❗

To reduce integration testing time, not all models should be enabled in the run_models.sh vars. Update the variables after dbt run and dbt test to set:

  • this PR's package to true
  • Google Ads and Facebook Ads to true (if not already)
  • All other packages to false

Detailed Validation

Please share any and all of your validation steps:

Added new validation tests, which are all passing using our available datasets
image

Also, regarding backwards compatibility, I added the following configs for each end model, and the package runs successfully, producing a conversions and conversions_c field

vars:
## Ad Reporting
  ad_reporting__ad_passthrough_metrics:
    - name: conversions
  ad_reporting__campaign_passthrough_metrics:
    - name: conversions
  ad_reporting__account_passthrough_metrics:
    - name: conversions
  ad_reporting__ad_group_passthrough_metrics:
    - name: conversions
  ad_reporting__keyword_passthrough_metrics:
    - name: conversions
  ad_reporting__search_passthrough_metrics:
    - name: conversions
  
## Amazon
  amazon_ads__advertised_product_passthrough_metrics:
    - name: purchases_30_d
      alias: conversions
  amazon_ads__campaign_passthrough_metrics:
    - name: purchases_30_d
      alias: conversions
  amazon_ads__ad_group_passthrough_metrics:
    - name: purchases_30_d
      alias: conversions
  amazon_ads__targeting_keyword_passthrough_metrics:
    - name: purchases_30_d
      alias: conversions
  amazon_ads__search_term_ad_keyword_passthrough_metrics:
    - name: purchases_30_d
      alias: conversions

## Snapchat
  snapchat_ads__campaign_hourly_report_passthrough_metrics:
    - name: conversion_purchases_value
    - name: conversion_purchases
      alias: conversions
  snapchat_ads__ad_hourly_passthrough_metrics:
    - name: conversion_purchases_value
    - name: conversion_purchases
      alias: conversions
  snapchat_ads__ad_squad_hourly_passthrough_metrics:
    - name: conversion_purchases
      alias: conversions

## Google
  google_ads__account_stats_passthrough_metrics:
    - name: conversions
  google_ads__ad_stats_passthrough_metrics:
    - name: conversions
  google_ads__ad_group_stats_passthrough_metrics:
    - name: conversions
  google_ads__keyword_stats_passthrough_metrics:
    - name: conversions
  google_ads__campaign_stats_passthrough_metrics:
    - name: conversions

## Apple
  apple_search_ads__ad_group_passthrough_metrics:
    - name: conversions
    - name: lat_off_installs
  apple_search_ads__ad_passthrough_metrics:
    - name: conversions
  apple_search_ads__campaign_passthrough_metrics:
    - name: conversions
  apple_search_ads__keyword_passthrough_metrics:
    - name: conversions
  apple_search_ads__search_term_passthrough_metrics:
    - name: conversions

## Pinterest
  pinterest__campaign_report_passthrough_metrics:
    - name: total_conversions
      alias: conversions
  pinterest__ad_group_report_passthrough_metrics:
    - name: total_conversions
      alias: conversions
  pinterest__advertiser_report_passthrough_metrics:
    - name: total_conversions
      alias: conversions
  pinterest__keyword_report_passthrough_metrics:
    - name: total_conversions
      alias: conversions
  pinterest__pin_promotion_report_passthrough_metrics:
    - name: total_conversions
      alias: conversions

## Microsoft
  microsoft_ads__account_passthrough_metrics: 
    - name: "conversions_qualified"
      alias: "conversions"
  microsoft_ads__ad_passthrough_metrics:
    - name: "conversions_qualified" 
      alias: "conversions"
  microsoft_ads__campaign_passthrough_metrics:
    - name: "conversions_qualified"
      alias: "conversions"
  microsoft_ads__ad_group_passthrough_metrics:
    - name: "conversions_qualified"
      alias: "conversions"
  microsoft_ads__keyword_passthrough_metrics:
    - name: "conversions_qualified"
      alias: "conversions"
  microsoft_ads__search_passthrough_metrics:
    - name: "conversions_qualified"
      alias: "conversions"

## Linkedin
  linkedin_ads__creative_passthrough_metrics:
    - name: 'one_click_leads'
      alias: conversions
      transform_sql: conversions + external_website_conversions
    - name: 'conversion_value_in_local_currency'
  linkedin_ads__campaign_passthrough_metrics:
    - name: 'one_click_leads'
      alias: conversions
      transform_sql: conversions + external_website_conversions
    - name: 'conversion_value_in_local_currency'

## TikTok
  tiktok_ads__ad_hourly_passthrough_metrics:
    - name: conversion
      alias: conversions
  tiktok_ads__campaign_hourly_passthrough_metrics:
    - name: conversion
      alias: conversions
  tiktok_ads__ad_group_hourly_passthrough_metrics:
    - name: conversion
      alias: conversions

## Twitter
  twitter_ads__promoted_tweet_report_passthrough_metrics:
    - name: "conversion_purchases_metric"
      alias: conversions
  twitter_ads__campaign_report_passthrough_metrics:
    - name: "conversion_purchases_metric"
      alias: conversions
  twitter_ads__line_item_report_passthrough_metrics: 
    - name: "conversion_purchases_metric"
      alias: conversions
  twitter_ads__line_item_keywords_report_passthrough_metrics: 
    - name: "conversion_purchases_metric"
      alias: conversions

## Reddit: conversion fields are coming from wholly new table, also its field is already called `conversions`
## Facebook: conversion fields are coming from wholly new table, also its field is already called `conversions`

If you had to summarize this PR in an emoji, which would it be?

🤹‍♂️

@fivetran-jamie fivetran-jamie self-assigned this Oct 30, 2024
@fivetran-jamie fivetran-jamie marked this pull request as ready for review November 4, 2024 23:30
dbt run --target "$db" --full-refresh --vars '{ad_reporting__facebook_ads_enabled: true, ad_reporting__google_ads_enabled: true, ad_reporting__amazon_ads_enabled: false, ad_reporting__apple_search_ads_enabled: false, ad_reporting__linkedin_ads_enabled: true, ad_reporting__microsoft_ads_enabled: false, ad_reporting__pinterest_ads_enabled: false, ad_reporting__reddit_ads_enabled: false, ad_reporting__snapchat_ads_enabled: false, ad_reporting__tiktok_ads_enabled: false, ad_reporting__twitter_ads_enabled: false}'
dbt test --target "$db" --vars '{ad_reporting__facebook_ads_enabled: true, ad_reporting__google_ads_enabled: true, ad_reporting__amazon_ads_enabled: false, ad_reporting__apple_search_ads_enabled: false, ad_reporting__linkedin_ads_enabled: true, ad_reporting__microsoft_ads_enabled: false, ad_reporting__pinterest_ads_enabled: false, ad_reporting__reddit_ads_enabled: false, ad_reporting__snapchat_ads_enabled: false, ad_reporting__tiktok_ads_enabled: false, ad_reporting__twitter_ads_enabled: false}'
dbt run --target "$db" --full-refresh
# --vars '{ad_reporting__facebook_ads_enabled: true, ad_reporting__google_ads_enabled: true, ad_reporting__amazon_ads_enabled: false, ad_reporting__apple_search_ads_enabled: false, ad_reporting__linkedin_ads_enabled: true, ad_reporting__microsoft_ads_enabled: false, ad_reporting__pinterest_ads_enabled: false, ad_reporting__reddit_ads_enabled: false, ad_reporting__snapchat_ads_enabled: false, ad_reporting__tiktok_ads_enabled: false, ad_reporting__twitter_ads_enabled: false}'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will comment back in later

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-jamie great work on this PR! I have a few comments and questions but after testing this on a few different combinations of platforms and passthrough columns, this is looking really good!

Let me know if you have any questions. Thanks!

macros/get_query.sql Show resolved Hide resolved
macros/get_query.sql Outdated Show resolved Hide resolved
packages.yml Outdated Show resolved Hide resolved
integration_tests/dbt_project.yml Show resolved Hide resolved
Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-jamie amazing job on this update and I'm looking forward to making this available to our users. This has been a long awaited update. Great job! One small comment, but this is ready for release review!

packages.yml Outdated Show resolved Hide resolved
fivetran-jamie and others added 2 commits November 18, 2024 09:37
Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>
@fivetran-reneeli fivetran-reneeli self-requested a review November 19, 2024 16:57
Copy link
Contributor

@fivetran-reneeli fivetran-reneeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Amazing job Jamie!!

@fivetran-jamie fivetran-jamie merged commit ab30c27 into main Nov 20, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants