[BUG] Accepted Values test failing when kpi_category is not one of the 6 main categories

[paulg66]

Describe the bug

If a test is added that does not default to one of the six kpi categories, build fails because of this test. The test doesn't allow "Other" as a KPI category or any customer kpi category that is defined in a test

Steps to reproduce

Add a test with a different kpi_category, example:
tests: - dq_tools.not_null_where_db: severity_level: error kpi_category: TEST

Expected results

The test should not fail

Actual results

The test fails

Screenshots and log output

16:09:21 On test.dq_tools.accepted_values_bi_column_analysis_kpi_category__Validity__Timeliness__Accuracy__Uniqueness__Completeness__Consistency.567ad2dc7e: select
      count(*) as failures,
      count(*) != 0 as should_warn,
      count(*) != 0 as should_error
    from (
   with all_values as (
    select
        kpi_category as value_field,
        count(*) as n_records
    from ANALYTICS_DEV.dbt_pgallagher.bi_column_analysis
    group by kpi_category
)
select *
from all_values
where value_field not in (
    'Validity','Timeliness','Accuracy','Uniqueness','Completeness','Consistency'
)
    ) dbt_internal_test
16:09:21 Opening a new connection, currently in state closed
16:09:22 SQL status: SUCCESS 1 in 0.0 seconds
16:09:22 Timing info for test.dq_tools.accepted_values_bi_column_analysis_kpi_category__Validity__Timeliness__Accuracy__Uniqueness__Completeness__Consistency.567ad2dc7e (execute): 16:09:21.747536 => 16:09:22.268494
16:09:22 On test.dq_tools.accepted_values_bi_column_analysis_kpi_category__Validity__Timeliness__Accuracy__Uniqueness__Completeness__Consistency.567ad2dc7e: Close
16:09:22 105 of 187 FAIL 1 accepted_values_bi_column_analysis_kpi_category__Validity__Timeliness__Accuracy__Uniqueness__Completeness__Consistency  [FAIL 1 in 0.64s]

System information

The contents of your packages.yml file:

  - package: infinitelambda/dq_tools
    version: [">=1.4.0", "<1.5.0"]

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

The output of dbt --version:

1.6

Additional context

All that is needed to add "Other" to the list of excepted values in this test
(https://github.com/infinitelambda/dq-tools/blob/11b696b9f9c066183619563b9ca5a850bf3f50f0/models/03_mart/data-quality-score/bi_column_analysis.yml#L23C6-L23C6)

Are you interested in contributing the fix?

I can update the test, will need someone to deploy it.

[il-dat]

Hi @paulg66 Thanks for raising this issue 🙌
Sorry for not getting back sooner on this, are you willing to open a PR now? Please help to let me know if that's the case (I will go fix it otherwise). I agree to support the "Other" KPI which should be straightforward as mentioned in your additional context section.

I will go release 1.4.3 asap once it's ready.

[paulg66]

HI @il-dat , thanks for getting back to me. I added 'Other' to the accepted values tests in this PR.

[il-dat]

Thanks @paulg66 The CI pipeline is quite broken now, I will fix it later. For this change, I manually trigger the test and looks good.

Since this is the behavioral change, so I will go and tag for a major release 1.5 soon.

Thanks again for the help here 🙌