feat(customer_experience): add authorship and engagement columns to kitsune_retrieval_index

[kpham-mozilla]

Description

Adds six new output columns to the Kitsune retrieval index
(customer_experience_derived.kitsune_retrieval_index_v1 and the downstream
customer_experience.kitsune_retrieval_index view) so consumers can see
question/answer authorship and engagement signals.

New columns

Authorship (from creator_username)

• question_creator (STRING) — username of the question author
• answer_creator (STRING) — username of the answer author; NULL when no answer

Engagement (from sumo_syndicate.kitsune_questions_plus)

• page_views (INTEGER) — page views the question has received
• is_solved (BOOLEAN) — whether the question is marked solved
• is_locked (BOOLEAN) — whether the thread is locked (the query filters
  is_locked = FALSE, so this is effectively always FALSE for stored rows)
• num_votes (INTEGER) — "me too" / interest votes on the question

Changes

• kitsune_retrieval_index_v1/query.sql — select the six new columns.
• kitsune_retrieval_index_v1/schema.yaml — declare the new fields.
• kitsune_retrieval_index/schema.yaml (view) — declare the new fields; the view
  is SELECT * EXCEPT(product) so it inherits the columns with no view.sql change.

Notes

• Local bqetl query validate dry-run was not run (expired GCP
  application-default credentials); CI runs the same dry-run + schema check.
• Assumed column types are best-effort; if the CI dry-run reports different types
  for any of the six, the schema.yaml files will need a quick adjustment.

[github-actions]

This PR adds six columns to customer_experience_derived.kitsune_retrieval_index_v1 (and the downstream customer_experience.kitsune_retrieval_index view): question_creator, answer_creator, page_views, is_solved, is_locked, and num_votes, sourced from sumo_syndicate.kitsune_questions_plus and the existing creator_username fields. The query and both schema.yaml files are updated consistently, and the column ordering in the schemas matches the SELECT.

My main concern is that the new source-column references will break the existing test_spam_filter_no_ai SQL test, since its input fixture doesn't define those columns and the schema is autodetected from the fixture. I also flagged that is_locked is constant by construction, and that the engagement columns carry as-of-creation-date snapshot semantics worth documenting. Details inline.

[github-actions]

issue: These new references to q.page_views, q.is_solved, and q.num_votes will break the existing SQL test test_spam_filter_no_ai. Its fixture tests/.../test_spam_filter_no_ai/moz-fx-data-shared-prod.sumo_syndicate.kitsune_questions_plus.yaml has no .schema file, so the source table schema is autodetected from the fixture rows (bigquery_etl/pytest_plugin/sql_test.py:128-129). That fixture only defines question_id, created_date, creator_username, product, ff_version, locale, topic, tier1_topic, tier2_topic, tier3_topic, title, question_content, is_spam, is_locked — it lacks page_views, is_solved, and num_votes, so the query will fail to resolve those columns. Add the three fields to the fixture (and to expect.yaml if you add a non-spam row).

[github-actions]

suggestion: is_locked is constant for every stored row — the CTE's WHERE q.is_locked = FALSE (line 36) guarantees it is always FALSE. Storing a column that carries no information adds width to the table (and the downstream view) for no consumer benefit. Consider dropping is_locked from the SELECT and both schema.yaml files rather than shipping a column whose description already concedes it is "effectively always FALSE".

[github-actions]

suggestion: These engagement fields are point-in-time snapshots, not accumulated totals, and the descriptions don't say so. The query runs per @submission_date with DATE(q.created_date) = @submission_date, so page_views, num_votes, and is_solved are captured on (roughly) the question's creation day and are never refreshed for that partition. A question that later accrues many views or gets solved will still show its day-0 values here. The sibling table sumo_metrics_derived.kitsune_forum_metrics_daily_v1 documents this exact caveat for is_solved ("NOT stable across ETL runs … reflects current solved state"). Consider noting the as-of-creation-date semantics in these descriptions so consumers don't read them as current engagement.

[scholtzan]

Integration report

• View full diff, if available (SQL + DAGs) (requires private-bigquery-etl access)
• Private CI run
• Bigeye monitoring dry-run: ✅ plan clean — view plan (requires private-bigquery-etl access)

[lucia-vargas-a]

We are intentionally avoiding user's data (PII in these public datasets, what would be the use cases?

[kelimuttu]

Hi Lucia,

At the moment, my primary use case for this is to detect duplicate questions. If we have other way to do it, I don't mind not having it.

[lucia-vargas-a]

Hey! what would be a duplicate - E.g. exact same text or repeated question_id (which I would not expect)

[lucia-vargas-a]

...With the definition of duplicates, we deduplicate in the query so that the data is unique and PII free.

[kelimuttu]

Usually, the same user posted similar things in a short period of time. For example:

• https://support.mozilla.org/en-US/questions/1591384
• https://support.mozilla.org/en-US/questions/1591387
• https://support.mozilla.org/en-US/questions/1591388