check notation uniqueness - #691
Open
joergklausen wants to merge 1 commit into
Open
Conversation
…to discover the published WMDR registers, rather than duplicating that mapping in Python. That file currently maps the repository table IDs to the corresponding codes.wmo.int/wmdr/... registers. It then: - checks every local CSV with a notation column for duplicates within that file; - checks every published register using its plain CSV endpoint with status=valid; - checks observed-variable notations jointly across 1-01-01 … 1-01-05; checks observing-method notations jointly across 5-02-01, 5-02-03, 5-02-05; - treats notation comparisons as case-sensitive, trims surrounding whitespace, and normalizes Unicode to NFC; - fails with exit status 1 on duplicates, malformed tables, missing notations, or inability to check the registry; - uses only the Python standard library. The special cross-table checks correspond directly to the repository's current division into five observed-variable tables and three observing-method tables. The workflow runs on every push and every pull_request, with a manual workflow_dispatch option as well. It uses the currently documented actions/checkout@v7 and actions/setup-python@v7 with Python 3.13. No requirements.txt addition is necessary. A successful run is deliberately compact: WMDR notation uniqueness report =============================== Repository CSVs: 65 table(s), ... notation(s) Repository CSVs / within-table: OK - all notations are unique within each table Repository CSVs / cross-table: OK - observed-variable and observing-method notations are unique across their related tables Published registry: checking 65 register(s) at codes.wmo.int ... Published registry: 65 table(s), ... notation(s) Published registry / within-table: OK - all notations are unique within each table Published registry / cross-table: OK - observed-variable and observing-method notations are unique across their related tables RESULT: PASS - all checked notations are unique Duplicates get their notation and locations reported, for example: Repository CSVs / cross-table: FAIL - duplicate notation(s) across related tables [observed variables] '123' occurs in 1-01-01, 1-01-03 Currently published atmosphere and terrestrial observing-method registers both contain inapplicable and unknown. Therefore these are currently legitimate cross-table duplicate findings. Flagging them is preferable initially.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat:The checker uses the existing tables_en/wmdr-tables.csv mapping to discover the published WMDR registers, rather than duplicating that mapping in Python. That file currently maps the repository table IDs to the corresponding codes.wmo.int/wmdr/... registers.
It then:
checks observing-method notations jointly across 5-02-01, 5-02-03, 5-02-05;
The special cross-table checks correspond directly to the repository's current division into five observed-variable tables and three observing-method tables.
The workflow runs on every push and every pull_request, with a manual workflow_dispatch option as well. It uses the currently documented actions/checkout@v7 and actions/setup-python@v7 with Python 3.13.
No requirements.txt addition is necessary.
A successful run is deliberately compact:
WMDR notation uniqueness report
Repository CSVs: 65 table(s), ... notation(s)
Repository CSVs / within-table: OK - all notations are unique within each table
Repository CSVs / cross-table: OK - observed-variable and observing-method notations are unique across their related tables
Published registry: checking 65 register(s) at codes.wmo.int ...
Published registry: 65 table(s), ... notation(s)
Published registry / within-table: OK - all notations are unique within each table
Published registry / cross-table: OK - observed-variable and observing-method notations are unique across their related tables
RESULT: PASS - all checked notations are unique
Duplicates get their notation and locations reported, for example:
Repository CSVs / cross-table: FAIL - duplicate notation(s) across related tables
[observed variables] '123' occurs in 1-01-01, 1-01-03
Currently published atmosphere and terrestrial observing-method registers both contain inapplicable and unknown. Therefore these are currently legitimate cross-table duplicate findings. Flagging them is preferable initially.