feat(): update ltv client references to new new profile clients #7728

kik-kik · 2025-07-03T10:54:56Z

feat(): update ltv client references to new new profile clients

This is work towards deprecating firefox_ios_clients and firefox_android_clients

…le_clients

…file_clients

kik-kik · 2025-07-03T10:55:33Z

sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/query.sql

 LEFT JOIN
  `moz-fx-data-shared-prod.fenix.client_adclicks_history`
  USING (sample_id, client_id)
 WHERE
    -- BrowserStack clients are bots, we don't want to accidentally report on them
-  first_reported_isp != "BrowserStack"
+  app_name <> "Fenix BrowserStack"


Should we also be exluding Mozilla Online here @kwindau ?

I think we are only excluding BrowserStack due to bots, not due to DAU definitions (this is more LTV focused than DAU focused). I think it's ok to leave as is, thanks for checking

@kwindau one more question, is there a use case where we may want to look at this data and filter out Mozilla Online users?

kik-kik · 2025-07-03T11:00:24Z

sql/moz-fx-data-shared-prod/firefox_ios_derived/ltv_states_v1/query.sql

-  c.first_reported_isp
+  clients.adjust_network,
+  clients.country AS first_reported_country,
+  CAST(NULL AS STRING) AS first_reported_isp,


isp field no longer available. As far as I recall the only purpose we used it for was to identify bots. This is now done upstream and this filtering can be done via app_name

kik-kik · 2025-07-03T11:01:05Z

sql/moz-fx-data-shared-prod/fenix_derived/client_ltv_v1/schema.yaml

@@ -2,7 +2,7 @@ fields:
  - name: client_id
    mode: NULLABLE
    type: STRING
-    description: "Client ID; uniquely identifies a client. Joinable with fenix.firefox_android_clients."
+    description: "Client ID; uniquely identifies a client."


Removed the last part since it's joinable with anything on client_id.

dataops-ci-bot · 2025-07-03T11:08:16Z

Integration report for "feat: update firefox_ios_references in firefox ios ltv to use new_profile_clients"

`sql.diff`

Click to expand!

diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_firefox_ios.py /tmp/workspace/generated-sql/dags/bqetl_firefox_ios.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_firefox_ios.py	2025-07-03 11:04:05.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_firefox_ios.py	2025-07-03 11:07:19.000000000 +0000
@@ -305,13 +305,6 @@
             execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=82800)).isoformat() }}",
         )
 
-        ExternalTaskMarker(
-            task_id="bqetl_org_mozilla_firefox_derived__wait_for_checks__fail_firefox_ios_derived__firefox_ios_clients__v1",
-            external_dag_id="bqetl_org_mozilla_firefox_derived",
-            external_task_id="wait_for_checks__fail_firefox_ios_derived__firefox_ios_clients__v1",
-            execution_date="{{ (execution_date - macros.timedelta(seconds=7200)).isoformat() }}",
-        )
-
         checks__fail_firefox_ios_derived__firefox_ios_clients__v1_external.set_upstream(
             checks__fail_firefox_ios_derived__firefox_ios_clients__v1
         )
@@ -515,13 +508,6 @@
             execution_date="{{ (execution_date - macros.timedelta(days=-1, seconds=82800)).isoformat() }}",
         )
 
-        ExternalTaskMarker(
-            task_id="bqetl_org_mozilla_firefox_derived__wait_for_firefox_ios_derived__clients_activation__v1",
-            external_dag_id="bqetl_org_mozilla_firefox_derived",
-            external_task_id="wait_for_firefox_ios_derived__clients_activation__v1",
-            execution_date="{{ (execution_date - macros.timedelta(seconds=7200)).isoformat() }}",
-        )
-
         firefox_ios_derived__clients_activation__v1_external.set_upstream(
             firefox_ios_derived__clients_activation__v1
         )
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_glean_usage.py	2025-07-03 11:04:12.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_glean_usage.py	2025-07-03 11:07:29.000000000 +0000
@@ -931,6 +931,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1",
@@ -1098,6 +1104,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1",
@@ -1265,6 +1277,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1",
@@ -1432,6 +1450,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1",
@@ -1599,6 +1623,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1",
@@ -1782,6 +1812,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1",
@@ -1965,6 +2001,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1",
@@ -2134,6 +2176,12 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_mobile_activation__wait_for_bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1",
             external_dag_id="bqetl_mobile_activation",
             external_task_id="wait_for_bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1",
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_mobile_kpi_metrics.py /tmp/workspace/generated-sql/dags/bqetl_mobile_kpi_metrics.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_mobile_kpi_metrics.py	2025-07-03 11:04:10.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_mobile_kpi_metrics.py	2025-07-03 11:07:25.000000000 +0000
@@ -477,6 +477,13 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__fenix_derived__attribution_clients__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__fenix_derived__attribution_clients__v1",
+            execution_date="{{ (execution_date - macros.timedelta(seconds=36000)).isoformat() }}",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_marketing_analysis__wait_for_bigeye__fenix_derived__attribution_clients__v1",
             external_dag_id="bqetl_marketing_analysis",
             external_task_id="wait_for_bigeye__fenix_derived__attribution_clients__v1",
@@ -623,6 +630,13 @@
         )
 
         ExternalTaskMarker(
+            task_id="bqetl_org_mozilla_firefox_derived__wait_for_bigeye__firefox_ios_derived__attribution_clients__v1",
+            external_dag_id="bqetl_org_mozilla_firefox_derived",
+            external_task_id="wait_for_bigeye__firefox_ios_derived__attribution_clients__v1",
+            execution_date="{{ (execution_date - macros.timedelta(seconds=36000)).isoformat() }}",
+        )
+
+        ExternalTaskMarker(
             task_id="bqetl_marketing_analysis__wait_for_bigeye__firefox_ios_derived__attribution_clients__v1",
             external_dag_id="bqetl_marketing_analysis",
             external_task_id="wait_for_bigeye__firefox_ios_derived__attribution_clients__v1",
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_org_mozilla_firefox_derived.py /tmp/workspace/generated-sql/dags/bqetl_org_mozilla_firefox_derived.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_org_mozilla_firefox_derived.py	2025-07-03 11:04:05.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_org_mozilla_firefox_derived.py	2025-07-03 11:07:19.000000000 +0000
@@ -161,11 +161,11 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1 = ExternalTaskSensor(
-        task_id="wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1",
-        external_dag_id="bqetl_firefox_ios",
-        external_task_id="checks__fail_firefox_ios_derived__baseline_clients_yearly__v1",
-        execution_delta=datetime.timedelta(days=-1, seconds=79200),
+    wait_for_bigeye__fenix_derived__attribution_clients__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__fenix_derived__attribution_clients__v1",
+        external_dag_id="bqetl_mobile_kpi_metrics",
+        external_task_id="fenix.bigeye__fenix_derived__attribution_clients__v1",
+        execution_delta=datetime.timedelta(days=-1, seconds=50400),
         check_existence=True,
         mode="reschedule",
         poke_interval=datetime.timedelta(minutes=5),
@@ -174,11 +174,95 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_checks__fail_firefox_ios_derived__client_adclicks_history__v1 = ExternalTaskSensor(
-        task_id="wait_for_checks__fail_firefox_ios_derived__client_adclicks_history__v1",
-        external_dag_id="bqetl_firefox_ios",
-        external_task_id="checks__fail_firefox_ios_derived__client_adclicks_history__v1",
-        execution_delta=datetime.timedelta(days=-1, seconds=79200),
+    wait_for_bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="fenix.bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1",
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="fenix.bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1",
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="fenix.bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1",
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="fenix.bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1",
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="fenix.bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1",
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_bigeye__firefox_ios_derived__attribution_clients__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__firefox_ios_derived__attribution_clients__v1",
+        external_dag_id="bqetl_mobile_kpi_metrics",
+        external_task_id="firefox_ios.bigeye__firefox_ios_derived__attribution_clients__v1",
+        execution_delta=datetime.timedelta(days=-1, seconds=50400),
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="firefox_ios.bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1",
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="firefox_ios.bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1",
         check_existence=True,
         mode="reschedule",
         poke_interval=datetime.timedelta(minutes=5),
@@ -187,10 +271,22 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_checks__fail_firefox_ios_derived__firefox_ios_clients__v1 = ExternalTaskSensor(
-        task_id="wait_for_checks__fail_firefox_ios_derived__firefox_ios_clients__v1",
+    wait_for_bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1 = ExternalTaskSensor(
+        task_id="wait_for_bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1",
+        external_dag_id="bqetl_glean_usage",
+        external_task_id="firefox_ios.bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1",
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1 = ExternalTaskSensor(
+        task_id="wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1",
         external_dag_id="bqetl_firefox_ios",
-        external_task_id="checks__fail_firefox_ios_derived__firefox_ios_clients__v1",
+        external_task_id="checks__fail_firefox_ios_derived__baseline_clients_yearly__v1",
         execution_delta=datetime.timedelta(days=-1, seconds=79200),
         check_existence=True,
         mode="reschedule",
@@ -200,10 +296,10 @@
         pool="DATA_ENG_EXTERNALTASKSENSOR",
     )
 
-    wait_for_firefox_ios_derived__clients_activation__v1 = ExternalTaskSensor(
-        task_id="wait_for_firefox_ios_derived__clients_activation__v1",
+    wait_for_checks__fail_firefox_ios_derived__client_adclicks_history__v1 = ExternalTaskSensor(
+        task_id="wait_for_checks__fail_firefox_ios_derived__client_adclicks_history__v1",
         external_dag_id="bqetl_firefox_ios",
-        external_task_id="firefox_ios_derived__clients_activation__v1",
+        external_task_id="checks__fail_firefox_ios_derived__client_adclicks_history__v1",
         execution_delta=datetime.timedelta(days=-1, seconds=79200),
         check_existence=True,
         mode="reschedule",
@@ -628,11 +724,31 @@
     )
 
     fenix_derived__ltv_states__v1.set_upstream(
-        checks__fail_fenix_derived__client_adclicks_history__v1
+        wait_for_bigeye__fenix_derived__attribution_clients__v1
     )
 
     fenix_derived__ltv_states__v1.set_upstream(
-        wait_for_checks__fail_fenix_derived__firefox_android_clients__v1
+        wait_for_bigeye__org_mozilla_fenix_derived__baseline_clients_last_seen__v1
+    )
+
+    fenix_derived__ltv_states__v1.set_upstream(
+        wait_for_bigeye__org_mozilla_fenix_nightly_derived__baseline_clients_last_seen__v1
+    )
+
+    fenix_derived__ltv_states__v1.set_upstream(
+        wait_for_bigeye__org_mozilla_fennec_aurora_derived__baseline_clients_last_seen__v1
+    )
+
+    fenix_derived__ltv_states__v1.set_upstream(
+        wait_for_bigeye__org_mozilla_firefox_beta_derived__baseline_clients_last_seen__v1
+    )
+
+    fenix_derived__ltv_states__v1.set_upstream(
+        wait_for_bigeye__org_mozilla_firefox_derived__baseline_clients_last_seen__v1
+    )
+
+    fenix_derived__ltv_states__v1.set_upstream(
+        checks__fail_fenix_derived__client_adclicks_history__v1
     )
 
     fenix_derived__ltv_states__v1.set_upstream(fenix_derived__clients_yearly__v1)
@@ -646,19 +762,27 @@
     )
 
     firefox_ios_derived__ltv_states__v1.set_upstream(
-        wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1
+        wait_for_bigeye__firefox_ios_derived__attribution_clients__v1
     )
 
     firefox_ios_derived__ltv_states__v1.set_upstream(
-        wait_for_checks__fail_firefox_ios_derived__client_adclicks_history__v1
+        wait_for_bigeye__org_mozilla_ios_fennec_derived__baseline_clients_last_seen__v1
     )
 
     firefox_ios_derived__ltv_states__v1.set_upstream(
-        wait_for_checks__fail_firefox_ios_derived__firefox_ios_clients__v1
+        wait_for_bigeye__org_mozilla_ios_firefox_derived__baseline_clients_last_seen__v1
     )
 
     firefox_ios_derived__ltv_states__v1.set_upstream(
-        wait_for_firefox_ios_derived__clients_activation__v1
+        wait_for_bigeye__org_mozilla_ios_firefoxbeta_derived__baseline_clients_last_seen__v1
+    )
+
+    firefox_ios_derived__ltv_states__v1.set_upstream(
+        wait_for_checks__fail_firefox_ios_derived__baseline_clients_yearly__v1
+    )
+
+    firefox_ios_derived__ltv_states__v1.set_upstream(
+        wait_for_checks__fail_firefox_ios_derived__client_adclicks_history__v1
     )
 
     org_mozilla_fenix_derived__client_deduplication__v1.set_upstream(
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/client_ltv_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/client_ltv_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/client_ltv_v1/schema.yaml	2025-07-03 11:00:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/client_ltv_v1/schema.yaml	2025-07-03 10:55:54.000000000 +0000
@@ -2,7 +2,7 @@
   - name: client_id
     mode: NULLABLE
     type: STRING
-    description: "Client ID; uniquely identifies a client. Joinable with fenix.firefox_android_clients."
+    description: "Client ID; uniquely identifies a client."
   - name: sample_id
     mode: NULLABLE
     type: INTEGER
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/query.sql	2025-07-03 11:00:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/query.sql	2025-07-03 10:55:54.000000000 +0000
@@ -42,20 +42,20 @@
     WHERE
       key <= clients_yearly.submission_date
   ) AS total_historic_ad_clicks,
-  firefox_android_clients.first_reported_country,
-  firefox_android_clients.first_reported_isp,
-  firefox_android_clients.adjust_network,
-  firefox_android_clients.install_source,
+  clients.country AS first_reported_country,
+  CAST(NULL AS STRING) AS first_reported_isp,
+  clients.adjust_network,
+  clients.install_source,
 FROM
   clients_yearly
 JOIN
-  `moz-fx-data-shared-prod.fenix.firefox_android_clients` AS firefox_android_clients
-  USING (sample_id, client_id)
+  `moz-fx-data-shared-prod.fenix.new_profile_clients` AS clients
+  USING (client_id)
 LEFT JOIN
   `moz-fx-data-shared-prod.fenix.client_adclicks_history`
   USING (sample_id, client_id)
 WHERE
     -- BrowserStack clients are bots, we don't want to accidentally report on them
-  first_reported_isp != "BrowserStack"
+  app_name <> "Fenix BrowserStack"
     -- Remove clients who are new on this day, but have more/less than 1 day of activity
   AND NOT (days_since_first_seen = 0 AND BIT_COUNT(days_seen_bytes) != 1)
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/schema.yaml	2025-07-03 11:00:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/ltv_states_v1/schema.yaml	2025-07-03 10:55:54.000000000 +0000
@@ -2,7 +2,7 @@
   - name: client_id
     mode: NULLABLE
     type: STRING
-    description: "Client ID; uniquely identifies a client. Joinable with fenix.firefox_android_clients."
+    description: "Client ID; uniquely identifies a client."
   - name: sample_id
     mode: NULLABLE
     type: INTEGER
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/ltv_states_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/ltv_states_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/ltv_states_v1/query.sql	2025-07-03 11:00:37.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_ios_derived/ltv_states_v1/query.sql	2025-07-03 10:55:54.000000000 +0000
@@ -60,16 +60,16 @@
   ac.durations,
   ac.ad_clicks,
   ac.total_historic_ad_clicks,
-  c.adjust_network,
-  c.first_reported_country,
-  c.first_reported_isp
+  clients.adjust_network,
+  clients.country AS first_reported_country,
+  CAST(NULL AS STRING) AS first_reported_isp,
 FROM
   ad_clicks ac
 JOIN
-  `moz-fx-data-shared-prod.firefox_ios.firefox_ios_clients` c
-  USING (sample_id, client_id)
+  `moz-fx-data-shared-prod.firefox_ios.new_profile_clients` AS clients
+  USING (client_id)
 WHERE
     -- BrowserStack clients are bots, we don't want to accidentally report on them
-  COALESCE(first_reported_isp, '') != "BrowserStack"
+  app_name <> "Firefox iOS BrowserStack"
     -- Remove clients who are new on this day, but have more/less than 1 day of activity
   AND NOT (days_since_first_seen = 0 AND BIT_COUNT(days_seen_bytes) != 1)

Link to full diff

kik-kik added 2 commits July 3, 2025 12:48

feat: update firefox_android_references in fenix ltv to use new_profi…

0c3bc54

…le_clients

feat: update firefox_ios_references in firefox ios ltv to use new_pro…

53e5a39

…file_clients

kik-kik requested a review from kwindau July 3, 2025 10:54

kik-kik commented Jul 3, 2025

View reviewed changes

kwindau approved these changes Jul 3, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(): update ltv client references to new new profile clients #7728

feat(): update ltv client references to new new profile clients #7728

Uh oh!

kik-kik commented Jul 3, 2025

Uh oh!

kik-kik Jul 3, 2025

Uh oh!

kwindau Jul 3, 2025

Uh oh!

kik-kik Jul 9, 2025

Uh oh!

kik-kik Jul 3, 2025

Uh oh!

kik-kik Jul 3, 2025

Uh oh!

dataops-ci-bot commented Jul 3, 2025

Uh oh!

Uh oh!

feat(): update ltv client references to new new profile clients #7728

Are you sure you want to change the base?

feat(): update ltv client references to new new profile clients #7728

Uh oh!

Conversation

kik-kik commented Jul 3, 2025

feat(): update ltv client references to new new profile clients

Uh oh!

kik-kik Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

kwindau Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

kik-kik Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

kik-kik Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

kik-kik Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

dataops-ci-bot commented Jul 3, 2025

Integration report for "feat: update firefox_ios_references in firefox ios ltv to use new_profile_clients"

sql.diff

Uh oh!

Uh oh!

`sql.diff`