Skip to content

check notation uniqueness - #691

Open
joergklausen wants to merge 1 commit into
masterfrom
notation-uniqueness-check
Open

check notation uniqueness#691
joergklausen wants to merge 1 commit into
masterfrom
notation-uniqueness-check

Conversation

@joergklausen

@joergklausen joergklausen commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

feat:The checker uses the existing tables_en/wmdr-tables.csv mapping to discover the published WMDR registers, rather than duplicating that mapping in Python. That file currently maps the repository table IDs to the corresponding codes.wmo.int/wmdr/... registers.

It then:

  • checks every local CSV with a notation column for duplicates within that file;
  • checks every published register using its plain CSV endpoint with status=valid;
  • checks observed-variable notations jointly across 1-01-01 … 1-01-05;
    checks observing-method notations jointly across 5-02-01, 5-02-03, 5-02-05;
  • treats notation comparisons as case-sensitive, trims surrounding whitespace, and normalizes Unicode to NFC;
  • fails with exit status 1 on duplicates, malformed tables, missing notations, or inability to check the registry;
  • uses only the Python standard library.
    The special cross-table checks correspond directly to the repository's current division into five observed-variable tables and three observing-method tables.
    The workflow runs on every push and every pull_request, with a manual workflow_dispatch option as well. It uses the currently documented actions/checkout@v7 and actions/setup-python@v7 with Python 3.13.
    No requirements.txt addition is necessary.
    A successful run is deliberately compact:

WMDR notation uniqueness report

Repository CSVs: 65 table(s), ... notation(s)
Repository CSVs / within-table: OK - all notations are unique within each table
Repository CSVs / cross-table: OK - observed-variable and observing-method notations are unique across their related tables

Published registry: checking 65 register(s) at codes.wmo.int ...
Published registry: 65 table(s), ... notation(s)
Published registry / within-table: OK - all notations are unique within each table
Published registry / cross-table: OK - observed-variable and observing-method notations are unique across their related tables

RESULT: PASS - all checked notations are unique

Duplicates get their notation and locations reported, for example:

Repository CSVs / cross-table: FAIL - duplicate notation(s) across related tables
[observed variables] '123' occurs in 1-01-01, 1-01-03

Currently published atmosphere and terrestrial observing-method registers both contain inapplicable and unknown. Therefore these are currently legitimate cross-table duplicate findings. Flagging them is preferable initially.

…to discover the published WMDR registers, rather than duplicating that mapping in Python. That file currently maps the repository table IDs to the corresponding codes.wmo.int/wmdr/... registers.

It then:
- checks every local CSV with a notation column for duplicates within that file;
- checks every published register using its plain CSV endpoint with status=valid;
- checks observed-variable notations jointly across 1-01-01 … 1-01-05;
checks observing-method notations jointly across 5-02-01, 5-02-03, 5-02-05;
- treats notation comparisons as case-sensitive, trims surrounding whitespace, and normalizes Unicode to NFC;
- fails with exit status 1 on duplicates, malformed tables, missing notations, or inability to check the registry;
- uses only the Python standard library.
The special cross-table checks correspond directly to the repository's current division into five observed-variable tables and three observing-method tables.
The workflow runs on every push and every pull_request, with a manual workflow_dispatch option as well. It uses the currently documented actions/checkout@v7 and actions/setup-python@v7 with Python 3.13.
No requirements.txt addition is necessary.
A successful run is deliberately compact:

WMDR notation uniqueness report
===============================
Repository CSVs: 65 table(s), ... notation(s)
Repository CSVs / within-table: OK - all notations are unique within each table
Repository CSVs / cross-table: OK - observed-variable and observing-method notations are unique across their related tables

Published registry: checking 65 register(s) at codes.wmo.int ...
Published registry: 65 table(s), ... notation(s)
Published registry / within-table: OK - all notations are unique within each table
Published registry / cross-table: OK - observed-variable and observing-method notations are unique across their related tables

RESULT: PASS - all checked notations are unique

Duplicates get their notation and locations reported, for example:

Repository CSVs / cross-table: FAIL - duplicate notation(s) across related tables
  [observed variables] '123' occurs in 1-01-01, 1-01-03

Currently published atmosphere and terrestrial observing-method registers both contain inapplicable and unknown. Therefore these are currently legitimate cross-table duplicate findings. Flagging them is preferable initially.
@amilan17 amilan17 changed the title feat:The checker uses the existing tables_en/wmdr-tables.csv mapping to discover the published WMDR registers, rather than duplicating that mapping in Python. That file currently maps the repository table IDs to the corresponding codes.wmo.int/wmdr/... registers. check notation uniqueness Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant