-
Notifications
You must be signed in to change notification settings - Fork 10
Amendments for INFCOM-4 #268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Changes from 60 commits
Commits
Show all changes
62 commits
Select commit
Hold shift + click to select a range
3e24da1
add a new part for Additional components (#144)
tomkralidis 9363dc5
part III updates (#207)
tomkralidis 81822a4
fix typo for SCGDC
tomkralidis dc35d12
add logging statement provides by sensor centres (#217)
tomkralidis a0e86f9
fix metric name
tomkralidis f146ea8
Enhance documentation for SCGB and SCGC
golfvert 5b1a07c
Add section on primary/backup WIS2 Node
golfvert e37cb9f
Refactor WIS2 Node service improvement section
golfvert 4781365
Remove pre-operational phase note from global services
golfvert 341c822
Remove pre-operational phase message handling note
golfvert ddaa985
Add the processing of primary/backup WIS centre broker
golfvert de79758
Update formatting for primary and secondary WIS nodes
golfvert 0c5a7c0
Update WIS2 Node to WIS Centre in documentation
golfvert 41114bb
Update terminology from 'secondary' to 'backup'
golfvert f137112
Global Replay editorial updates (#232)
tomkralidis ffb3a06
add clarification on use of HTTP status codes for GC Technical consid…
tomkralidis 24e15e8
add section on WCMP2 KPIs (#123) (#247)
tomkralidis aa401aa
Clarify Global Cache behavior when not caching a data instance (#249)
josusky 7860d6b
SCGDC: update topic subscription (#241) (#240)
tomkralidis af59e5c
Add info on metric wmo_wis2_scgc_messages_published_total
golfvert ba8ebb9
update retention period for notifications (#251)
tomkralidis 3fea7c9
Add outage notification guideline for Global Services (#248)
jmaxmarno 3820164
Expand Sensor Centres section with new details (#254)
6a6d74 ff1c7b4
Expand guidance on Global Services metrics and operations
efucile 3b7b0e6
update GDC performance management (#255)
tomkralidis db940d6
Refine subscription and processing steps for Sensor Centre (#253)
golfvert ea89651
Update outage notification procedure for Global Services
golfvert 0080d08
[WIP] ET-W2IT 2026-06 updates (#256)
tomkralidis acc33af
keep consistent ordering of GSs
tomkralidis 8d28b0a
renaming to Other Global Services (#258)
tomkralidis d02bd3e
Document downloading feature assessment for Global Cache (#260)
golfvert 66ffa45
manual rebase from main (#262)
tomkralidis 5016aeb
update front matter
tomkralidis 78c01ba
add sections for WME (#233) (#263)
tomkralidis 40f38bd
Update details of Sensor Centre metrics calculation (SCGB, SCGC) (#265)
6a6d74 f622064
Revise per-component operational criteria for Global Services (#264)
6a6d74 1c2b19a
Update operational time assessment criteria
hhaddouch 1c34493
add kpi appendix with link to GH pages
amilan17 0e6673f
add additional information on monitoring events (#267)
tomkralidis 341a519
Merge pull request #266 from wmo-im/hhaddouch-patch-1
amilan17 f05460c
Merge branch 'main' into INFCOM-4
amilan17 0bb9cf4
only editorial fixes
amilan17 a38e8cb
change kpi appendix from a to b
amilan17 0128aae
Merge pull request #269 from wmo-im/add-kpi-appendix
amilan17 d8ed745
remove one line
amilan17 70dae73
Apply suggestion from @amilan17
amilan17 8edc5c1
Apply suggestion from @amilan17
amilan17 46215aa
Apply suggestions from code review
amilan17 4d6e368
cleanup missing section numbers and appendix/annex names (#273)
amilan17 0590cbf
editorial fixes
amilan17 bd88b8c
editorial changes based on Jeremy's review
amilan17 60ce568
more updates from Jeremy's review
amilan17 a92b252
Merge pull request #274 from wmo-im/infcom-prep2
amilan17 b0146bf
fix spelling
amilan17 741f1f7
add missing tick
amilan17 679a7a9
Add clarification for WME about non-cached instances (#233) (#275)
josusky 0687b96
revisions based on review with Jeremy
amilan17 cc86eef
Apply suggestions from code review
amilan17 98d817f
Apply suggestion from @6a6d74
amilan17 ad36720
Apply suggestions from code review
amilan17 cf57b4f
Apply suggestion from @amilan17
amilan17 d473c32
Apply suggestion from @amilan17
amilan17 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
amilan17 marked this conversation as resolved.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| [appendix] | ||
| :appendix-caption: Annex | ||
|
|
||
| == APPENDIX A: PER-COMPONENT OPERATIONAL CRITERIA | ||
|
|
||
| This appendix contains additional information to section 2.7.2.9. The operational criteria specific for each category of designated Global Services are set out below. Criteria for other types of Global Service will be added in subsequent revisions of this Guide. | ||
|
|
||
| === A.1 Global Broker | ||
|
|
||
| The operational status of a Global Broker is assessed against two KPIs, both measured through Sensor Centre Global Broker (SCGB) instances operating in different parts of the globe, so as to reduce the influence of localized network conditions or of any individual Sensor Centre. Hourly KPI evaluation is based on the best result across all SCGB instances; KPIs are calculated at the Global Monitor, not at the Sensor Centre itself. | ||
|
|
||
| ==== A.1.1 Message completeness KPI | ||
|
|
||
| This KPI measures the percentage of WIS2 Notification Messages (WNM) published by the Global Broker, within a defined cut-off time, compared against a baseline derived from all Global Brokers. | ||
|
|
||
| *Measure (calculated each hour):* | ||
|
|
||
| The percentage of WNM messages published by the Global Broker within the cut-off time of 60 seconds, compared to the baseline, is assessed using a randomized sample of approximately 10 per cent of messages. The number of messages published by each Global Broker within the cutoff time is recorded using the metric `wmo_wis2_scgb_messages_published_total`. The baseline is the highest number of messages published by any Global Broker, as recorded by any SCGB instance, within the previous hour. A message is considered "missed" if a Global Broker has not published that message within 60-seconds of it first being published by any other Global Broker. | ||
|
|
||
| *KPI evaluation (hourly):* | ||
|
|
||
| The message completeness KPI is set to "pass" for the hour if the percentage of messages published is not less than 95 per cent of the baseline; otherwise it is set to "fail" for the hour. | ||
|
|
||
| ==== A.1.2 Connectivity KPI | ||
|
|
||
| This KPI measures the proportion of the hour during which the public MQTT endpoint of the Global Broker are reachable, as determined from the connection status recorded by SCGB instances. | ||
|
|
||
| *Measure (calculated each hour):* | ||
|
|
||
| The proportion of time that the Global Broker’s MQTT end-point is accessible, published by the Sensor Centre Global Broker using metric `wmo_wis2_scgb_connected_flag`. | ||
|
|
||
| *KPI evaluation (hourly):* | ||
|
|
||
| The connectivity KPI is set to "pass" for the hour if the `wmo_wis2_scgb_connected_flag` is set to `1` ("True", connected) for not less than 95 per cent of the time; otherwise it is set to "fail" for the hour. | ||
|
|
||
| ==== A.1.3 Combined hourly and monthly KPI | ||
|
|
||
| * For each hour, the Global Broker is considered operational where *both* the message completeness KPI *and* the connectivity KPI are set to "pass". If either KPI fails for the hour, the Global Broker is considered non-operational for that hour. | ||
| * Each month, the overall KPI is set to "pass" if the combined hourly KPI passes for not less than 98 per cent of hours in the month (i.e. the Global Broker may experience up to 14 "failure" hours in a calendar month and still be considered performing within the service level). | ||
| * Hours coinciding with planned outages, or other _force majeure_ agreed by SC-IMT, are excluded from the monthly KPI calculation. | ||
| * The monthly KPI is used in the assessment of Global Broker performance informing the determination of downtime in accordance with section 2.7.2.6. | ||
|
|
||
| ''' | ||
|
|
||
| === A.2 Global Cache | ||
|
|
||
| The operational status of a Global Cache is assessed against two KPIs, both measured through Sensor Centre Global Cache (SCGC) instances operating in different parts of the globe, so as to reduce the influence of localized network conditions or of any individual Sensor Centre. Hourly KPI evaluation is based on the best result across all SCGC instances; KPIs are calculated at the Global Monitor, not at the Sensor Centre itself. | ||
|
|
||
| ==== A.2.1 Cache completeness KPI | ||
|
|
||
| This KPI measures the percentage of data items published by the Global Cache compared with the total number of data items published by all WIS2 Nodes (on `origin` topics). | ||
|
|
||
| *Measure (calculated each hour):* | ||
|
|
||
| The number of data items published by all origin WIS2 Nodes is recorded using the metric `wmo_wis2_scgc_messages_published_total`. Only those data items expected to be cached are included in this measure. | ||
|
|
||
| The number of data items cached by a Global Cache is recorded using the metric `wmo_wis2_scgc_messages_cached_total`. Data items that are cached more than 600-seconds after publication by the origin WIS2 Node are excluded from this measure. | ||
|
|
||
| Because Global Caches are designed to discard out-of-sequence updates, and counting such discards as "misses" would not be a fair reflection of performance, KPI performance is evaluated only for "canonical" data items (i.e., where the Notification Message specifies the link-relation `canonical`). Both `wmo_wis2_scgc_messages_published_total` and `wmo_wis2_scgc_messages_cached_total` metrics include the label `rel` to distinguish the link-relation. | ||
|
|
||
| The percentage of data items published by a Global Cache is calculated as the increase in metric `wmo_wis2_scgc_messages_cached_total (rel=canonical)` over the previous hour ÷ the increase in metric `wmo_wis2_scgc_messages_published_total (rel=canonical)` over the previous hour. | ||
|
|
||
| *KPI evaluation (hourly):* | ||
|
|
||
| The cache completeness KPI is set to "pass" for the hour if the percentage of data items cached is not less than 95 per cent of the number published by origin; otherwise it is set to "fail" for the hour. | ||
|
|
||
| ==== A.2.2 Successful downloads KPI | ||
|
|
||
| This KPI measures the proportion of cached data items that can be successfully downloaded from the Global Cache, including, where a checksum is available in the message, validation of data integrity. | ||
|
|
||
| *Measure (calculated each hour):* | ||
|
|
||
| This measure is evaluated using a sample set of 300 randomly selected data items each hour. Data items must be cached by all Global Caches to be included in the sample set. The same sample set is used for all Global Caches. A small sample size is used to avoid overloading both Global Caches and Sensor Centres. | ||
|
|
||
| The total number of data items sampled for download validation is recorded using the metric `wmo_wis2_scgc_download_selected_total`. | ||
|
amilan17 marked this conversation as resolved.
|
||
|
|
||
| The total number of data items in the sample set successfully downloaded by each Global Cache is recorded using the metric `wmo_wis2_scgc_download_successful_total`. | ||
|
|
||
| The percentage of data items successfully downloaded is calculated as the increase in metric `wmo_wis2_scgc_download_successful_total` over the previous hour ÷ the increase in metric `wmo_wis2_scgc_download_selected_total` over the previous hour. | ||
|
amilan17 marked this conversation as resolved.
|
||
|
|
||
| *KPI evaluation (hourly):* | ||
|
|
||
| The successful download KPI is set to "pass" for the hour if the percentage of successful downloads is not less than 95 per cent; otherwise it is set to "fail" for the hour. | ||
|
|
||
| Note that if `wmo_wis2_scgc_download_selected_total` is zero for the previous hour (i.e., no data items were cached by any Global Cache - perhaps because one Global Cache is offline), the successful downloads KPI cannot be calculated. In this case, the KPI is automatically set to "pass" for all Global Caches. The Global Cache that is offline will fail the cache completeness KPI, thus the combined KPI (see below) for the offline Global Cache will always be a fail. | ||
|
|
||
| ==== A.2.3 Combined hourly and monthly KPI | ||
|
|
||
| * For each hour, the Global Cache is considered operational where *both* the cache completeness KPI *and* the successful downloads KPI are set to "pass". If either KPI fails for the hour, the Global Cache is considered non-operational for that hour. | ||
| * Each month, the overall KPI is set to "pass" if the combined hourly KPI passes for not less than 98 per cent of hours in the month (i.e. the Global Cache may experience up to 14 "failure" hours in a calendar month and still be considered performing within the service level). | ||
| * Hours coinciding with planned outages, or other _force majeure_ agreed by SC-IMT, are excluded from the monthly KPI calculation. | ||
| * The monthly KPI is used in the assessment of Global Cache performance informing the determination of downtime in accordance with section 2.7.2.6. | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| == APPENDIX B: WMO CORE METADATA PROFILE (VERSION 2), KEY PERFORMANCE INDICATORS | ||
|
|
||
| https://wmo-im.github.io/wcmp2/kpi/wcmp2-kpi-DRAFT.html |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.