Switch `mobile_search_clients_daily` derived table to pull from baseline ping from Aug 01st 2024 onwards #5985

alekhyamoz · 2024-07-29T21:12:17Z

With the recent KPI DAU changes and the need to align the search DAU with KPI.
We will be changing the SQL to pull the search related metrics from baseline tables instead of metrics.
The planned cut-off is Aug 1st 2024.

As part of this effort, we need to -
Before Aug 1st 2024 (Implemented as part of - #5977)
**[Done]**create a v2 table to pull from the baseline table
[Done] monitor and validate v1 and v2 tables for ~1 week

on Aug 1st 2024 -
3. create a prehistory table mobile_clients_daily_historical_pre202408 with all historical data from metrics table
4. change the view and mobile_aggregates to pull from both history and v2
5. Change other downstream tables to pull from the new view mobile_search_clients_daily
6. cut off populating v1

Checklist for reviewer:

Commits should reference a bug or github issue, if relevant (if a bug is referenced, the pull request should include the bug number in the title).
If the PR comes from a fork, trigger integration CI tests by running the Push to upstream workflow and provide the <username>:<branch> of the fork as parameter. The parameter will also show up
in the logs of the manual-trigger-required-for-fork CI task together with more detailed instructions.
If adding a new field to a query, ensure that the schema and dependent downstream schemas have been updated.
When adding a new derived dataset, ensure that data is not available already (fully or partially) and recommend extending an existing dataset in favor of creating new ones. Data can be available in the bigquery-etl repository, looker-hub or in looker-spoke-default.

For modifications to schemas in restricted namespaces (see CODEOWNERS):

Follow the change control procedure

┆Issue is synchronized with this Jira Task

dataops-ci-bot · 2024-07-29T21:44:14Z

Integration report for "Fix CI errors"

`sql.diff`

Click to expand!

diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_desktop_mobile_search_monthly.py /tmp/workspace/generated-sql/dags/bqetl_desktop_mobile_search_monthly.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_desktop_mobile_search_monthly.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_desktop_mobile_search_monthly.py	2024-07-29 21:43:37.000000000 +0000
@@ -50,10 +50,10 @@
     tags=tags,
 ) as dag:
 
-    wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor(
-        task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         external_dag_id="bqetl_mobile_search",
-        external_task_id="search_derived__mobile_search_clients_daily__v1",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
         execution_delta=datetime.timedelta(days=2, seconds=10800),
         check_existence=True,
         mode="reschedule",
@@ -87,7 +87,7 @@
     )
 
     search_derived__desktop_mobile_search_clients_monthly__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     search_derived__desktop_mobile_search_clients_monthly__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_firefox_ios.py /tmp/workspace/generated-sql/dags/bqetl_firefox_ios.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_firefox_ios.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_firefox_ios.py	2024-07-29 21:43:37.000000000 +0000
@@ -62,10 +62,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor(
-        task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         external_dag_id="bqetl_mobile_search",
-        external_task_id="search_derived__mobile_search_clients_daily__v1",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
         execution_delta=datetime.timedelta(seconds=7200),
         check_existence=True,
         mode="reschedule",
@@ -681,7 +681,7 @@
     )
 
     firefox_ios_derived__attributable_clients__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     firefox_ios_derived__baseline_clients_yearly__v1.set_upstream(
@@ -717,7 +717,7 @@
     )
 
     firefox_ios_derived__clients_activation__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     firefox_ios_derived__firefox_ios_clients__v1.set_upstream(
@@ -793,5 +793,5 @@
     )
 
     firefox_ios_derived__new_profile_activation__v2.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_generated_funnels.py /tmp/workspace/generated-sql/dags/bqetl_generated_funnels.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_generated_funnels.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_generated_funnels.py	2024-07-29 21:43:39.000000000 +0000
@@ -167,21 +167,6 @@
         depends_on_past=False,
     )
 
-    accounts_frontend_derived__login_engagement_funnel__v1 = bigquery_etl_query(
-        task_id="accounts_frontend_derived__login_engagement_funnel__v1",
-        destination_table="login_engagement_funnel_v1",
-        dataset_id="accounts_frontend_derived",
-        project_id="moz-fx-data-shared-prod",
-        owner="ksiegler@mozilla.org",
-        email=[
-            "ascholtz@mozilla.com",
-            "ksiegler@mozilla.org",
-            "telemetry-alerts@mozilla.com",
-        ],
-        date_partition_parameter="submission_date",
-        depends_on_past=False,
-    )
-
     accounts_frontend_derived__login_funnels_by_service__v1 = bigquery_etl_query(
         task_id="accounts_frontend_derived__login_funnels_by_service__v1",
         destination_table="login_funnels_by_service_v1",
@@ -291,6 +276,21 @@
         )
     )
 
+    monitor_frontend_derived__monitor_dashboard_user_journey_funnels__v1 = bigquery_etl_query(
+        task_id="monitor_frontend_derived__monitor_dashboard_user_journey_funnels__v1",
+        destination_table="monitor_dashboard_user_journey_funnels_v1",
+        dataset_id="monitor_frontend_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="ksiegler@mozilla.org",
+        email=[
+            "ascholtz@mozilla.com",
+            "ksiegler@mozilla.org",
+            "telemetry-alerts@mozilla.com",
+        ],
+        date_partition_parameter="submission_date",
+        depends_on_past=False,
+    )
+
     accounts_frontend_derived__email_first_reg_login_funnels__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -299,10 +299,6 @@
         wait_for_copy_deduplicate_all
     )
 
-    accounts_frontend_derived__login_engagement_funnel__v1.set_upstream(
-        wait_for_accounts_frontend_derived__events_stream__v1
-    )
-
     accounts_frontend_derived__login_funnels_by_service__v1.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -344,3 +340,7 @@
     firefox_accounts_derived__registration_funnels_legacy_events__v1.set_upstream(
         wait_for_firefox_accounts_derived__fxa_stdout_events__v1
     )
+
+    monitor_frontend_derived__monitor_dashboard_user_journey_funnels__v1.set_upstream(
+        wait_for_copy_deduplicate_all
+    )
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_kpis_shredder.py /tmp/workspace/generated-sql/dags/bqetl_kpis_shredder.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_kpis_shredder.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_kpis_shredder.py	2024-07-29 21:43:38.000000000 +0000
@@ -86,6 +86,17 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
+        external_dag_id="bqetl_mobile_search",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
+        check_existence=True,
+        mode="reschedule",
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
     wait_for_checks__fail_telemetry_derived__clients_last_seen__v2 = ExternalTaskSensor(
         task_id="wait_for_checks__fail_telemetry_derived__clients_last_seen__v2",
         external_dag_id="bqetl_main_summary",
@@ -217,6 +228,10 @@
         wait_for_search_derived__mobile_search_clients_daily__v1
     )
 
+    fenix_active_users_aggregates_for_deletion_requests.set_upstream(
+        wait_for_search_derived__mobile_search_clients_daily__v2
+    )
+
     firefox_desktop_active_users_aggregates_for_deletion_requests.set_upstream(
         wait_for_checks__fail_telemetry_derived__clients_last_seen__v2
     )
@@ -241,6 +256,10 @@
         wait_for_search_derived__mobile_search_clients_daily__v1
     )
 
+    firefox_ios_active_users_aggregates_for_deletion_requests.set_upstream(
+        wait_for_search_derived__mobile_search_clients_daily__v2
+    )
+
     focus_ios_active_users_aggregates_for_deletion_requests.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -253,6 +272,10 @@
         wait_for_search_derived__mobile_search_clients_daily__v1
     )
 
+    focus_ios_active_users_aggregates_for_deletion_requests.set_upstream(
+        wait_for_search_derived__mobile_search_clients_daily__v2
+    )
+
     klar_ios_active_users_aggregates_for_deletion_requests.set_upstream(
         wait_for_copy_deduplicate_all
     )
@@ -264,3 +287,7 @@
     klar_ios_active_users_aggregates_for_deletion_requests.set_upstream(
         wait_for_search_derived__mobile_search_clients_daily__v1
     )
+
+    klar_ios_active_users_aggregates_for_deletion_requests.set_upstream(
+        wait_for_search_derived__mobile_search_clients_daily__v2
+    )
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_mobile_activation.py /tmp/workspace/generated-sql/dags/bqetl_mobile_activation.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_mobile_activation.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_mobile_activation.py	2024-07-29 21:43:38.000000000 +0000
@@ -182,10 +182,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor(
-        task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         external_dag_id="bqetl_mobile_search",
-        external_task_id="search_derived__mobile_search_clients_daily__v1",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
         execution_delta=datetime.timedelta(days=-1, seconds=79200),
         check_existence=True,
         mode="reschedule",
@@ -354,7 +354,7 @@
     )
 
     fenix_derived__new_profile_activation__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     firefox_ios_derived__new_profile_activation__v1.set_upstream(
@@ -382,5 +382,5 @@
     )
 
     firefox_ios_derived__new_profile_activation__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_mobile_search.py /tmp/workspace/generated-sql/dags/bqetl_mobile_search.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_mobile_search.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_mobile_search.py	2024-07-29 21:43:36.000000000 +0000
@@ -80,12 +80,12 @@
         depends_on_past=False,
     )
 
-    search_derived__mobile_search_clients_daily__v1 = bigquery_etl_query(
-        task_id="search_derived__mobile_search_clients_daily__v1",
-        destination_table="mobile_search_clients_daily_v1",
+    search_derived__mobile_search_clients_daily__v2 = bigquery_etl_query(
+        task_id="search_derived__mobile_search_clients_daily__v2",
+        destination_table="mobile_search_clients_daily_v2",
         dataset_id="search_derived",
         project_id="moz-fx-data-shared-prod",
-        owner="akomar@mozilla.com",
+        owner="cmorales@mozilla.com",
         email=[
             "akomar@mozilla.com",
             "akommasani@mozilla.com",
@@ -98,81 +98,64 @@
     )
 
     with TaskGroup(
-        "search_derived__mobile_search_clients_daily__v1_external",
-    ) as search_derived__mobile_search_clients_daily__v1_external:
+        "search_derived__mobile_search_clients_daily__v2_external",
+    ) as search_derived__mobile_search_clients_daily__v2_external:
         ExternalTaskMarker(
-            task_id="bqetl_kpis_shredder__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_kpis_shredder__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_kpis_shredder",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         )
 
         ExternalTaskMarker(
-            task_id="bqetl_org_mozilla_firefox_derived__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_org_mozilla_firefox_derived",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         )
 
         ExternalTaskMarker(
-            task_id="bqetl_mobile_activation__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_mobile_activation__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_mobile_activation",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
             execution_date="{{ (execution_date - macros.timedelta(seconds=7200)).isoformat() }}",
         )
 
         ExternalTaskMarker(
-            task_id="bqetl_firefox_ios__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_firefox_ios__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_firefox_ios",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
             execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=79200)).isoformat() }}",
         )
 
         ExternalTaskMarker(
-            task_id="bqetl_review_checker__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_review_checker__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_review_checker",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
             execution_date="{{ (execution_date - macros.timedelta(seconds=7200)).isoformat() }}",
         )
 
         ExternalTaskMarker(
-            task_id="bqetl_desktop_mobile_search_monthly__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_desktop_mobile_search_monthly__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_desktop_mobile_search_monthly",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
             execution_date="{{ (execution_date - macros.timedelta(days=-3, seconds=75600)).isoformat() }}",
         )
 
         ExternalTaskMarker(
-            task_id="bqetl_search_dashboard__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_search_dashboard__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_search_dashboard",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
             execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=77400)).isoformat() }}",
         )
 
         ExternalTaskMarker(
-            task_id="bqetl_unified__wait_for_search_derived__mobile_search_clients_daily__v1",
+            task_id="bqetl_unified__wait_for_search_derived__mobile_search_clients_daily__v2",
             external_dag_id="bqetl_unified",
-            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+            external_task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
             execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=82800)).isoformat() }}",
         )
 
-        search_derived__mobile_search_clients_daily__v1_external.set_upstream(
-            search_derived__mobile_search_clients_daily__v1
-        )
-
-    search_derived__mobile_search_clients_daily__v2 = bigquery_etl_query(
-        task_id="search_derived__mobile_search_clients_daily__v2",
-        destination_table="mobile_search_clients_daily_v2",
-        dataset_id="search_derived",
-        project_id="moz-fx-data-shared-prod",
-        owner="cmorales@mozilla.com",
-        email=[
-            "akomar@mozilla.com",
-            "akommasani@mozilla.com",
-            "anicholson@mozilla.com",
-            "cmorales@mozilla.com",
-            "telemetry-alerts@mozilla.com",
-        ],
-        date_partition_parameter="submission_date",
-        depends_on_past=False,
+        search_derived__mobile_search_clients_daily__v2_external.set_upstream(
+            search_derived__mobile_search_clients_daily__v2
     )
 
     search_derived__mobile_search_clients_last_seen__v1 = bigquery_etl_query(
@@ -192,11 +175,7 @@
     )
 
     search_derived__mobile_search_aggregates__v1.set_upstream(
-        search_derived__mobile_search_clients_daily__v1
-    )
-
-    search_derived__mobile_search_clients_daily__v1.set_upstream(
-        wait_for_copy_deduplicate_all
+        search_derived__mobile_search_clients_daily__v2
     )
 
     search_derived__mobile_search_clients_daily__v2.set_upstream(
@@ -204,5 +183,5 @@
     )
 
     search_derived__mobile_search_clients_last_seen__v1.set_upstream(
-        search_derived__mobile_search_clients_daily__v1
+        search_derived__mobile_search_clients_daily__v2
     )
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_org_mozilla_firefox_derived.py /tmp/workspace/generated-sql/dags/bqetl_org_mozilla_firefox_derived.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_org_mozilla_firefox_derived.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_org_mozilla_firefox_derived.py	2024-07-29 21:43:36.000000000 +0000
@@ -139,10 +139,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor(
-        task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         external_dag_id="bqetl_mobile_search",
-        external_task_id="search_derived__mobile_search_clients_daily__v1",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
         check_existence=True,
         mode="reschedule",
         allowed_states=ALLOWED_STATES,
@@ -532,7 +532,7 @@
     )
 
     fenix_derived__attributable_clients__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     fenix_derived__attributable_clients__v2.set_upstream(wait_for_copy_deduplicate_all)
@@ -542,7 +542,7 @@
     )
 
     fenix_derived__attributable_clients__v2.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     fenix_derived__client_adclicks_history__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_review_checker.py /tmp/workspace/generated-sql/dags/bqetl_review_checker.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_review_checker.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_review_checker.py	2024-07-29 21:43:38.000000000 +0000
@@ -112,10 +112,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor(
-        task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         external_dag_id="bqetl_mobile_search",
-        external_task_id="search_derived__mobile_search_clients_daily__v1",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
         execution_delta=datetime.timedelta(days=-1, seconds=79200),
         check_existence=True,
         mode="reschedule",
@@ -262,7 +262,7 @@
     )
 
     org_mozilla_fenix_review_checker_clients__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     org_mozilla_fenix_review_checker_events__v1.set_upstream(
@@ -274,7 +274,7 @@
     )
 
     org_mozilla_ios_firefox_review_checker_clients__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     org_mozilla_ios_firefox_review_checker_events__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_search_dashboard.py /tmp/workspace/generated-sql/dags/bqetl_search_dashboard.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_search_dashboard.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_search_dashboard.py	2024-07-29 21:43:36.000000000 +0000
@@ -71,10 +71,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor(
-        task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         external_dag_id="bqetl_mobile_search",
-        external_task_id="search_derived__mobile_search_clients_daily__v1",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
         execution_delta=datetime.timedelta(seconds=9000),
         check_existence=True,
         mode="reschedule",
@@ -343,7 +343,7 @@
     )
 
     search_derived__mobile_search_aggregates_for_searchreport__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     search_derived__search_revenue_levers_daily__v1.set_upstream(
@@ -407,7 +407,7 @@
     )
 
     search_derived__search_revenue_levers_daily__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     search_derived__search_revenue_levers_daily__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_unified.py /tmp/workspace/generated-sql/dags/bqetl_unified.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_unified.py	2024-07-29 21:25:33.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_unified.py	2024-07-29 21:43:37.000000000 +0000
@@ -128,10 +128,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_search_derived__mobile_search_clients_daily__v1 = ExternalTaskSensor(
-        task_id="wait_for_search_derived__mobile_search_clients_daily__v1",
+    wait_for_search_derived__mobile_search_clients_daily__v2 = ExternalTaskSensor(
+        task_id="wait_for_search_derived__mobile_search_clients_daily__v2",
         external_dag_id="bqetl_mobile_search",
-        external_task_id="search_derived__mobile_search_clients_daily__v1",
+        external_task_id="search_derived__mobile_search_clients_daily__v2",
         execution_delta=datetime.timedelta(seconds=3600),
         check_existence=True,
         mode="reschedule",
@@ -306,7 +306,7 @@
     )
 
     telemetry_derived__unified_metrics__v1.set_upstream(
-        wait_for_search_derived__mobile_search_clients_daily__v1
+        wait_for_search_derived__mobile_search_clients_daily__v2
     )
 
     telemetry_derived__unified_metrics__v1.set_upstream(
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived: login_engagement_funnel_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/search: mobile_search_clients_daily
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search/mobile_search_clients_engines_sources_daily: schema.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search_derived/mobile_search_aggregates_v1: backfill.yaml
Only in /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/search_derived: mobile_search_clients_daily_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/metadata.yaml	2024-07-29 21:19:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
@@ -1,26 +0,0 @@
-friendly_name: Login Engagement Funnel
-description: |-
-  Please provide a description for the query
-owners:
-- ksiegler@mozilla.org
-labels:
-  incremental: true
-  dag: bqetl_generated_funnels
-  owner1: ksiegler
-scheduling:
-  dag_name: bqetl_generated_funnels
-bigquery:
-  time_partitioning:
-    type: day
-    field: submission_date
-    require_partition_filter: false
-    expiration_days: null
-  range_partitioning: null
-  clustering: null
-workgroup_access:
-- role: roles/bigquery.dataViewer
-  members:
-  - workgroup:mozilla-confidential
-references:
-  query.sql:
-  - mozdata.accounts_frontend.events_stream
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/query.sql	2024-07-29 21:19:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_frontend_derived/login_engagement_funnel_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
@@ -1,666 +0,0 @@
--- extract the relevant fields for each funnel step and segment if necessary
-WITH login_engage_login_view AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    IF(
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      ) = 'sync',
-      '5882386c6d801776',
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      )
-    ) AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.view'
-    AND metrics.string.session_flow_id != ''
-),
-login_engage_login_engage AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    prev.service AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  INNER JOIN
-    login_engage_login_view AS prev
-    ON prev.submission_date = DATE(submission_timestamp)
-    AND prev.join_key = metrics.string.session_flow_id
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.engage'
-    AND metrics.string.session_flow_id != ''
-),
-login_submit_login_view AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    IF(
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      ) = 'sync',
-      '5882386c6d801776',
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      )
-    ) AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.view'
-    AND metrics.string.session_flow_id != ''
-),
-login_submit_login_submit AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    prev.service AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  INNER JOIN
-    login_submit_login_view AS prev
-    ON prev.submission_date = DATE(submission_timestamp)
-    AND prev.join_key = metrics.string.session_flow_id
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.submit'
-    AND metrics.string.session_flow_id != ''
-),
-login_submit_success_login_view AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    IF(
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      ) = 'sync',
-      '5882386c6d801776',
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      )
-    ) AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.view'
-    AND metrics.string.session_flow_id != ''
-),
-login_submit_success_login_submit_success AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    prev.service AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  INNER JOIN
-    login_submit_success_login_view AS prev
-    ON prev.submission_date = DATE(submission_timestamp)
-    AND prev.join_key = metrics.string.session_flow_id
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.submit_success'
-    AND metrics.string.session_flow_id != ''
-),
-login_diff_account_engage_login_view AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    IF(
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      ) = 'sync',
-      '5882386c6d801776',
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      )
-    ) AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.view'
-    AND metrics.string.session_flow_id != ''
-),
-login_diff_account_engage_login_diff_account_engage AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    prev.service AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  INNER JOIN
-    login_diff_account_engage_login_view AS prev
-    ON prev.submission_date = DATE(submission_timestamp)
-    AND prev.join_key = metrics.string.session_flow_id
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.diff_account_link_click'
-    AND metrics.string.session_flow_id != ''
-),
-login_forgot_pw_engage_login_view AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    IF(
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      ) = 'sync',
-      '5882386c6d801776',
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      )
-    ) AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.view'
-    AND metrics.string.session_flow_id != ''
-),
-login_forgot_pw_engage_login_forgot_pw_engage AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    prev.service AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  INNER JOIN
-    login_forgot_pw_engage_login_view AS prev
-    ON prev.submission_date = DATE(submission_timestamp)
-    AND prev.join_key = metrics.string.session_flow_id
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.forgot_pwd_submit'
-    AND metrics.string.session_flow_id != ''
-),
-login_google_engage_login_view AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    IF(
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      ) = 'sync',
-      '5882386c6d801776',
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      )
-    ) AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.view'
-    AND metrics.string.session_flow_id != ''
-),
-login_google_engage_login_google_engage AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    prev.service AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  INNER JOIN
-    login_google_engage_login_view AS prev
-    ON prev.submission_date = DATE(submission_timestamp)
-    AND prev.join_key = metrics.string.session_flow_id
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'third_party_auth.google_login_start'
-    AND metrics.string.session_flow_id != ''
-),
-login_apple_engage_login_view AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    IF(
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      ) = 'sync',
-      '5882386c6d801776',
-      COALESCE(
-        NULLIF(metrics.string.relying_party_oauth_client_id, ''),
-        NULLIF(metrics.string.relying_party_service, '')
-      )
-    ) AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'login.view'
-    AND metrics.string.session_flow_id != ''
-),
-login_apple_engage_login_apple_engage AS (
-  SELECT
-    metrics.string.session_flow_id AS join_key,
-    prev.service AS service,
-    DATE(submission_timestamp) AS submission_date,
-    metrics.string.account_user_id_sha256 AS client_id,
-    metrics.string.session_flow_id AS column
-  FROM
-    mozdata.accounts_frontend.events_stream
-  INNER JOIN
-    login_apple_engage_login_view AS prev
-    ON prev.submission_date = DATE(submission_timestamp)
-    AND prev.join_key = metrics.string.session_flow_id
-  WHERE
-    {% if is_init() %}
-      DATE(submission_timestamp) >= DATE("2024-07-25")
-    {% else %}
-      DATE(submission_timestamp) = @submission_date
-    {% endif %}
-    AND event = 'third_party_auth.apple_login_start'
-    AND metrics.string.session_flow_id != ''
-),
--- aggregate each funnel step value
-login_engage_login_view_aggregated AS (
-  SELECT
-    submission_date,
-    "login_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_engage_login_view
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_engage_login_engage_aggregated AS (
-  SELECT
-    submission_date,
-    "login_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_engage_login_engage
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_submit_login_view_aggregated AS (
-  SELECT
-    submission_date,
-    "login_submit" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_submit_login_view
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_submit_login_submit_aggregated AS (
-  SELECT
-    submission_date,
-    "login_submit" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_submit_login_submit
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_submit_success_login_view_aggregated AS (
-  SELECT
-    submission_date,
-    "login_submit_success" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_submit_success_login_view
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_submit_success_login_submit_success_aggregated AS (
-  SELECT
-    submission_date,
-    "login_submit_success" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_submit_success_login_submit_success
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_diff_account_engage_login_view_aggregated AS (
-  SELECT
-    submission_date,
-    "login_diff_account_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_diff_account_engage_login_view
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_diff_account_engage_login_diff_account_engage_aggregated AS (
-  SELECT
-    submission_date,
-    "login_diff_account_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_diff_account_engage_login_diff_account_engage
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_forgot_pw_engage_login_view_aggregated AS (
-  SELECT
-    submission_date,
-    "login_forgot_pw_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_forgot_pw_engage_login_view
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_forgot_pw_engage_login_forgot_pw_engage_aggregated AS (
-  SELECT
-    submission_date,
-    "login_forgot_pw_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_forgot_pw_engage_login_forgot_pw_engage
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_google_engage_login_view_aggregated AS (
-  SELECT
-    submission_date,
-    "login_google_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_google_engage_login_view
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_google_engage_login_google_engage_aggregated AS (
-  SELECT
-    submission_date,
-    "login_google_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_google_engage_login_google_engage
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_apple_engage_login_view_aggregated AS (
-  SELECT
-    submission_date,
-    "login_apple_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_apple_engage_login_view
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
-login_apple_engage_login_apple_engage_aggregated AS (
-  SELECT
-    submission_date,
-    "login_apple_engage" AS funnel,
-    service,
-    COUNT(DISTINCT column) AS aggregated
-  FROM
-    login_apple_engage_login_apple_engage
-  GROUP BY
-    service,
-    submission_date,
-    funnel
-),
--- merge all funnels so results can be written into one table
-merged_funnels AS (
-  SELECT
-    COALESCE(
-      login_engage_login_view_aggregated.service,
-      login_submit_login_view_aggregated.service,
-      login_submit_success_login_view_aggregated.service,
-      login_diff_account_engage_login_view_aggregated.service,
-      login_forgot_pw_engage_login_view_aggregated.service,
-      login_google_engage_login_view_aggregated.service,
-      login_apple_engage_login_view_aggregated.service
-    ) AS service,
-    submission_date,
-    funnel,
-    COALESCE(
-      login_engage_login_view_aggregated.aggregated,
-      login_submit_login_view_aggregated.aggregated,
-      login_submit_success_login_view_aggregated.aggregated,
-      login_diff_account_engage_login_view_aggregated.aggregated,
-      login_forgot_pw_engage_login_view_aggregated.aggregated,
-      login_google_engage_login_view_aggregated.aggregated,
-      login_apple_engage_login_view_aggregated.aggregated
-    ) AS login_view,
-    COALESCE(
-      login_engage_login_engage_aggregated.aggregated,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL
-    ) AS login_engage,
-    COALESCE(
-      NULL,
-      login_submit_login_submit_aggregated.aggregated,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL
-    ) AS login_submit,
-    COALESCE(
-      NULL,
-      NULL,
-      login_submit_success_login_submit_success_aggregated.aggregated,
-      NULL,
-      NULL,
-      NULL,
-      NULL
-    ) AS login_submit_success,
-    COALESCE(
-      NULL,
-      NULL,
-      NULL,
-      login_diff_account_engage_login_diff_account_engage_aggregated.aggregated,
-      NULL,
-      NULL,
-      NULL
-    ) AS login_diff_account_engage,
-    COALESCE(
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      login_forgot_pw_engage_login_forgot_pw_engage_aggregated.aggregated,
-      NULL,
-      NULL
-    ) AS login_forgot_pw_engage,
-    COALESCE(
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      login_google_engage_login_google_engage_aggregated.aggregated,
-      NULL
-    ) AS login_google_engage,
-    COALESCE(
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      login_apple_engage_login_apple_engage_aggregated.aggregated
-    ) AS login_apple_engage,
-  FROM
-    login_engage_login_view_aggregated
-  FULL OUTER JOIN
-    login_engage_login_engage_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_submit_login_view_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_submit_login_submit_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_submit_success_login_view_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_submit_success_login_submit_success_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_diff_account_engage_login_view_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_diff_account_engage_login_diff_account_engage_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_forgot_pw_engage_login_view_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_forgot_pw_engage_login_forgot_pw_engage_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_google_engage_login_view_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_google_engage_login_google_engage_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_apple_engage_login_view_aggregated
-    USING (submission_date, service, funnel)
-  FULL OUTER JOIN
-    login_apple_engage_login_apple_engage_aggregated
-    USING (submission_date, service, funnel)
-)
-SELECT
-  *
-FROM
-  merged_funnels
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-07-29 21:19:19.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2024-07-29 21:32:30.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-07-29 21:19:19.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2024-07-29 21:32:30.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-07-29 21:19:19.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2024-07-29 21:29:21.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-07-29 21:19:19.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2024-07-29 21:29:23.000000000 +0000
@@ -48,6 +48,10 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
+  description: null
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/metadata.yaml	2024-07-29 21:21:34.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/metadata.yaml	2024-07-29 21:38:39.000000000 +0000
@@ -44,6 +44,6 @@
   query.sql:
   - fenix.deletion_request
   - moz-fx-data-shared-prod.fenix.clients_last_seen_joined
-  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1
+  - moz-fx-data-shared-prod.search.mobile_search_clients_daily
   - mozdata.static.csa_gblmkt_languages
 deprecated: true
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/query.sql	2024-07-29 21:19:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/active_users_aggregates_deletion_request_v1/query.sql	2024-07-29 21:39:41.000000000 +0000
@@ -41,7 +41,7 @@
     search_count,
     search_with_ads
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1` search
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily` search
   INNER JOIN
     `moz-fx-data-shared-prod.fenix.deletion_request` AS request
     ON client_info.client_id = client_id
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v1/query.sql	2024-07-29 21:19:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v1/query.sql	2024-07-29 21:20:12.000000000 +0000
@@ -36,7 +36,7 @@
     SUM(search_with_ads) AS searches_with_ads,
     SUM(ad_click) AS ad_clicks
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1`
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily`
   WHERE
     {% if is_init() %}
       submission_date >= DATE("2021-01-01")
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v2/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v2/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v2/query.sql	2024-07-29 21:19:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attributable_clients_v2/query.sql	2024-07-29 21:20:12.000000000 +0000
@@ -37,7 +37,7 @@
     SUM(search_with_ads) AS searches_with_ads,
     SUM(ad_click) AS ad_clicks
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1`
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily`
   WHERE
     {% if is_init() %}
       submission_date >= "2021-08-01"
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/metadata.yaml	2024-07-29 21:21:12.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/metadata.yaml	2024-07-29 21:38:20.000000000 +0000
@@ -31,4 +31,4 @@
   - moz-fx-data-shared-prod.fenix.baseline_clients_first_seen
   - moz-fx-data-shared-prod.fenix.baseline_clients_last_seen
   - moz-fx-data-shared-prod.fenix.first_session
-  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1
+  - moz-fx-data-shared-prod.search.mobile_search_clients_daily
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/query.sql	2024-07-29 21:19:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/new_profile_activation_v1/query.sql	2024-07-29 21:20:12.000000000 +0000
@@ -29,7 +29,7 @@
     client_id,
     SUM(search_count) AS search_count
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1`
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily`
   JOIN
     client_first_seen
     USING (client_id)
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/metadata.yaml	2024-07-29 21:21:26.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/metadata.yaml	2024-07-29 21:38:35.000000000 +0000
@@ -44,6 +44,6 @@
   query.sql:
   - firefox_ios.deletion_request
   - moz-fx-data-shared-prod.firefox_ios.clients_last_seen_joined
-  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1
+  - moz-fx-data-shared-prod.search.mobile_search_clients_daily
   - mozdata.static.csa_gblmkt_languages
 deprecated: true
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/query.sql	2024-07-29 21:19:56.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/active_users_aggregates_deletion_request_v1/query.sql	2024-07-29 21:39:55.000000000 +0000
@@ -41,7 +41,7 @@
     search_count,
     search_with_ads
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1` search
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily` search
   INNER JOIN
     `moz-fx-data-shared-prod.firefox_ios.deletion_request` AS request
     ON client_info.client_id = client_id
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/metadata.yaml	2024-07-29 21:21:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/metadata.yaml	2024-07-29 21:38:17.000000000 +0000
@@ -28,4 +28,4 @@
   query.sql:
   - moz-fx-data-shared-prod.firefox_ios.baseline
   - moz-fx-data-shared-prod.firefox_ios.firefox_ios_clients
-  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1
+  - moz-fx-data-shared-prod.search.mobile_search_clients_daily
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/query.sql	2024-07-29 21:19:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/attributable_clients_v1/query.sql	2024-07-29 21:20:12.000000000 +0000
@@ -37,7 +37,7 @@
     SUM(search_with_ads) AS searches_with_ads,
     SUM(ad_click) AS ad_clicks
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1`
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily`
   WHERE
     {% if is_init() %}
       submission_date >= "2020-05-01"
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/metadata.yaml	2024-07-29 21:21:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/metadata.yaml	2024-07-29 21:38:17.000000000 +0000
@@ -39,4 +39,4 @@
   query.sql:
   - moz-fx-data-shared-prod.firefox_ios.baseline_clients_last_seen
   - moz-fx-data-shared-prod.firefox_ios.firefox_ios_clients
-  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1
+  - moz-fx-data-shared-prod.search.mobile_search_clients_daily
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/query.sql	2024-07-29 21:19:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/clients_activation_v1/query.sql	2024-07-29 21:20:12.000000000 +0000
@@ -38,7 +38,7 @@
     channel,
     SUM(search_count) AS search_count
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1`
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily`
   WHERE
     (submission_date BETWEEN DATE_SUB(@submission_date, INTERVAL 3 DAY) AND @submission_date)
     AND os = 'iOS'
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/metadata.yaml	2024-07-29 21:21:08.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/metadata.yaml	2024-07-29 21:38:17.000000000 +0000
@@ -29,4 +29,4 @@
   query.sql:
   - moz-fx-data-shared-prod.firefox_ios.baseline_clients_first_seen
   - moz-fx-data-shared-prod.firefox_ios.baseline_clients_last_seen
-  - moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1
+  - moz-fx-data-shared-prod.search.mobile_search_clients_daily
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/query.sql	2024-07-29 21:19:20.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/new_profile_activation_v1/query.sql	2024-07-29 21:20:12.000000000 +0000
@@ -29,7 +29,7 @@
     client_id,
     SUM(search_count) AS search_count
   FROM
-    `moz-fx-data-shared-prod.search_derived.mobile_search_clients_daily_v1`
+    `moz-fx-data-shared-prod.search.mobile_search_clients_daily`
   JOIN
     client_first_seen

⚠️ Only part of the diff is displayed.

Link to full diff

alekhyamoz added 2 commits July 29, 2024 17:10

Switch to beginning Aug 01st 2024

20354c0

Fix CI errors

4a031ce

alekhyamoz requested review from scholtzan, curtismorales and skahmann3 July 30, 2024 14:05

alekhyamoz changed the title ~~Switch to mobile_search_clients_daily to pull from metrics from Aug 01st 2024 onwards~~ Switch mobile_search_clients_daily derived table to pull from baseline ping from Aug 01st 2024 onwards Jul 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch `mobile_search_clients_daily` derived table to pull from baseline ping from Aug 01st 2024 onwards #5985

Switch `mobile_search_clients_daily` derived table to pull from baseline ping from Aug 01st 2024 onwards #5985

alekhyamoz commented Jul 29, 2024 •

edited

Loading

dataops-ci-bot commented Jul 29, 2024

Switch mobile_search_clients_daily derived table to pull from baseline ping from Aug 01st 2024 onwards #5985

Are you sure you want to change the base?

Switch mobile_search_clients_daily derived table to pull from baseline ping from Aug 01st 2024 onwards #5985

Conversation

alekhyamoz commented Jul 29, 2024 • edited Loading

dataops-ci-bot commented Jul 29, 2024

Integration report for "Fix CI errors"

sql.diff

Switch `mobile_search_clients_daily` derived table to pull from baseline ping from Aug 01st 2024 onwards #5985

Switch `mobile_search_clients_daily` derived table to pull from baseline ping from Aug 01st 2024 onwards #5985

alekhyamoz commented Jul 29, 2024 •

edited

Loading

`sql.diff`