PHEP 4: PyHC Package Tiering

pheps/phep-0004.md

@@ -0,0 +1,147 @@
+```
+PHEP: 4
+Title: PyHC Package Tiering
+Author: Julie Barnum <[email protected]> <https://orcid.org/0000-0001-8755-0694>
+Discussions-To: https://github.com/heliophysicsPy/standards/pull/31
+Revision: 1
+Status: Draft
+Type: Process
+Content-Type: text/markdown; charset=UTF-8; variant=myst
+Created: 09-July-2024
+Post-History: 09-July-2024, 26-August-2024, 27-August-2024, 4-September-2024, 11-September-2024, 13-September-2024, 25-April-2025, 05-June-2025, 26-Jun-2025
+```
+
+# Abstract
+<a name="abstract"></a>
+This PHEP establishes a new tiering structure to PyHC projects, which will automatically affect PyHC packages once it goes into effect. Included herein is information on requirements for each of the new three tiers of PyHC projects (Gold, Silver, and Bronze), as well as benefits accrued at each tier. 
+
+# Motivation
+<a name="motivation"></a>
+Currently, PyHC is at a crossroads for how to push forward as a community. There are two main schools of thought — originating from bi-annual meeting discussions, telecon chats, and further sidebar conversation — with regards to what PyHC is and should be:  
+
+- a basic interpretation where PyHC is a collection, and listing, of open-source Python packages with a relevance to Heliophysics and space physics, or
+- a standards-based interpretation where PyHC strives for compliance with our set standards, package interoperability, and standardization around one or more tools.   
+
+There is utility and validity to both approaches. A new PyHC package tiering system is intended to find a "best of both worlds" with the two ideas. Older, out-of-date, unmaintained, or publication-specific code could still have a place for listing and findability, while also allowing nuance between other packages that are more robust, trustworthy, maintained, and work toward the standards-based interpretation of being a PyHC package. 
+
+Further, this tiering system also allows users to get a clearer picture on what each PyHC package has to offer, and the state of the package's condition and development. Creation of a PyHC package tiering system also allows for justification for a myriad of benefits, for example, consideration for funding from a community travel fund, or extra help with improving a standards grouping grade.
+
+# Rationale
+<a name="rationale"></a>
+Decisions for tiering levels, requirements for each tier, and benefits accrued at each tier are based on conversations with the community (bi-annual meetings, telecons, etc.), and are listed here as a starting point for more discussion, likely to be refined in the future. Initially, ideas were presented to the community in a pyramid format. To make the differences between tiers more visible and understandable, it has been transformed into a spreadsheet format. 
+
+This PHEP intentionally violates the PHEP-1 requirement of being formatted in strict CommonMark markdown. CommonMark does not support tables and tabular presentation is the clearest means of communicating the tier structure. It uses [MyST markdown](https://mystmd.org/) for tables only, and the Content-Type indicates that. MyST is commonly used in the scientific Python community and is a strict superset of CommonMark, so is the most compatible means of supporting tabular information.
+
+# PyHC Package Tiering Specifications
+<a name="specification"></a>
+There are three tiers proposed in this PHEP: Bronze, Silver, and Gold. These would replace the current structure of "Core Packages", "Other Packages", and "Un-evaluated Packages". Extensive updates have been made to the following tiering specifications, based on discussions held at the [PyHC Fall 2024 meeting](https://doi.org/10.5281/zenodo.15080483). See the table below for requirements associated with each tier:
+
+|       | **Gold** | **Silver** | **Bronze** | 
+| :-: | :----------: | :-----------: | :------------: | 
+| Self-Evaluation | No | No | Yes | 
+| PyHC-pyOpenSci Review | Yes, defined in PyHC-pyOpenSci Gold-tier PHEP | Yes, defined in PyHC-pyOpenSci Silver-tier PHEP | No |
+| PyHC Standards Compliance | Packages must comply with all applicable standards (as defined in standards-track PHEPs) |
+
+\* = See HSSI metadata schema for details
+
+Descriptions for each heading are as follows:
+   - Self Evaluation: indicates whether a package must complete a self evaluation against PyHC's standards
+   - pyOpenSci Review: indicates whether a package must engage in a PyHC-pyOpenSci review
+   - PyHC Standards Compliance: indicates necessary level of compliance with standards-track PHEPs (set to replace the current PyHC standards document: https://doi.org/10.5281/zenodo.2529131)
+
+The following table shows the benefits that are associated with each tier:  
+
+|       | **Gold** | **Silver** | **Bronze** | 
+| :-: | :----------: | :-----------: | :------------: | 
+| Summer School Inclusion \* | Invited to give a tutorial at the summer school | Invited to give a tutorial at the summer school if space allows\** | Can be brought in by higher-tier package, but not directly invited |
+| PyHC-Chat Bot or PyHC Environment(s) Inclusion \* | Catch-all: higher-tier packages are higher priority for cross-PyHC projects |
+| Standards Compliance Assistance \* | Yes\*** | Yes | Yes |
+| Conference Travel Funding | Yes\* | Yes, if availability after gold-tier packages\*, \** | No |  
+
+\*  = can opt-in or opt-out of this  
+
+\** = conditional upon justification on importance/use of package (e.g. number of users, level of commitment within PyHC activities, or effort made for those items but gold level not yet achieved)  
+
+\*** = conditional upon justification that package is in danger of dropping down to a Silver tier 
+
+Descriptions for each heading are as follows:
+   - Summer School Inclusion: indicates whether a package will be included in summer school teaching materials
+   - PyHC-Chat Bot or PyHC Environment(s) Inclusion: indicates whether a packages will have up-to-date information included within the ChatGPT4-powered PyHC-Chat bot or be prioritized for inclusion in future PyHC environment(s)
+   - Standards Compliance Assistance: indicates whether a package will receive extra help and/or funding (dependent on available PyHC Leadership resources) for conforming to the PyHC standards
+   - Conference Travel Funding: indicates whether developers from a package will be considered for PyHC travel funding assistance to relevant science conferences (e.g. SHINE, CEDAR, or GEM). All tiers are conditional upon availability of funds through PyHC Leadership.
+
+Packages are evaluated (as described in 'Implementation Process' below) against compliance with each requirement, as shown in the PyHC tiering chart. To be accepted for a tier, a package must meet **all** the requirements for said tier. Therefore, should a package fall into different tiers depending on the specific requirement, the package will be accepted at the lowest tier of requirements it meets.  
+
+For example, if a package meets some requirements for the Silver tier, but other requirements only meet the Bronze tier, the package will be considered a Bronze tier package.   
+
+
+# Package Tiering Implementation Process
+
+Following acceptance of this PHEP, the PyHC website will be updated to reflect the new tiers. The process is then as follows:
+- Within six months of PyHC website update (written communication of this provided in email form), packages must submit a PR to [the PyHC website GitHub](https://github.com/heliophysicsPy/heliophysicsPy.github.io) indicating what level they believe their package is
+    - need to include written justification on the PR for chosen tier 
+- Within three months of PR publish, a member of the to-be-established technical steering committee (TSC; including the PyHC Tech Lead and volunteer members of the PyHC not involved with the submitted package) reviews the package and asserts if self-evaluated level matches expectations
+- PyHC Lead reviews decision of TSC member(s) and gives final vote of approval on tier selection
+- PyHC Lead then either:
+    - 1) merges the PR if in agreement with the TSC member(s), or 
+    - 2) meets with the TSC member(s) to discuss dissenting opinion
+        - if consensus reached for previos tier selection, PyHC Lead then merges the PR, or
+        - PyHC Lead provides written justification on the PR if a tier re-grade is deemed necessary  
+
+General comments:  
+
+Packages are allowed to move between tiers. If a package upgrades their status to match that of Silver, instead of Bronze, tier, for example, they can submit a new PR to have their tier regraded.   
+
+The flip side is also true; should a package become defunct or drop in status, they may be downgraded to a lower tier. The TSC shall meet on an annual basis (likely tied to a bi-annual meeting) to review package tier status and determine if any are in danger of a tier downgrade. A package downgrade requires consensus of the technical steering committee. 
+
+Should any package be flagged for downgrade, the follow will occur:  
+- Packages will receive six months' notification before a tier downgrade is to take place through written e-mail communication (from the PyHC Lead)  
+- Packages then have the opportunity to rectify issues within the six-month period, through changes submitted in a PR  
+    - If changes result in continued compliance with current tier (as deemed by the PyHC Tech Lead and PyHC Lead), no tier re-grade happens  
+    - If changes do not result in continued compliance with current tier, a tier re-grade happens; packages may later on apply for a tier re-grade once more  
+
+# Backwards Compatibility
+<a name="backwards-compatibility"></a>
+This PHEP does not propose a direct change to PyHC package code, simply the inclusion or not of packages within the various tiers, thus it introduces no compatibility concerns.
+
+# Security Implications
+<a name="security-implications"></a>
+This PHEP raises no security implications as it does not interact with any executing code.
+
+# How to Teach This
+<a name="how-to-teach-this"></a>
+This PHEP's contents and changes will be presented on, discussed, and hacked at various PyHC bi-weekly telecons and PyHC bi-annual meetings. Additionally, explanations for tiering, the process of obtaining a PyHC tier, etc. will be posted on the new main Projects page, as well as communicated within a blog post under the PyHC Blog page.   
+
+Further, links to pages (and potentially recordings) that explain how to satisfy each requirement for each tier will be made and communicated through a PyHC blog post (link then emailed to the community and presented at either a telecon or bi-annual meeting, whichever comes first).  
+
+Please note that the PyHC project submission process used thus far will be superseded by this PHEP's PyHC tier request submission. Further, the self evaluations relied upon previously will now only grant a package Bronze-tier status; Technical steering committee evaluation is required to fully verify a package's tier. Also, there are now additional requirement for package tiers that exist apart from the main PyHC standards (as detailed in this PHEP). PyHC leadership commits to updating the submission process to match this PHEP's specifications.
+
+# Rejected Ideas
+<a name="rejected-ideas"></a>
+None yet to note.
+
+# Open Issues
+<a name="open-issues"></a>
+None yet to note.
+
+# Footnotes
+<a name="footnotes"></a>
+None yet to note.
+
+# Revisions
+<a name="revisions"></a>
+Revision 1 (pending): Initial draft.
+
+# Copyright
+<a name="copyright"></a>
+This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. It should be cited as:
+```
+@techreport(phep4,
+  author = {Julie I. Barnum},
+  title  = {PHEP Package Tiering},
+    year = {2024--2025},
+    type = {PHEP},
+  number = {4},
+     doi = {10.5281/zenodo.15747682}
+)
+```