variants.csv file rationale

[hforoughmand]

I don't understand the rationale behind the variants.csv file. I would be happy if you could answer my following questions:

1. For the earlier lineages, all the subvariants of concern are listed and no "*" is used to indicate the variants, while that is not the case for more recent variants.

2. Why in some more recent commits lineages are removed from the list? Does the list supposed to only contain the current list of VOCs?

3. How can we contribute in keeping this list up to date? How do you gather all these information? What are the sources?

4. Can we also include the source of the data in the variants.csv file? Then, everyone can check for the validity of the record.

5. Can we also include the designation date for each VOC?

[3dgiordano]

Hi @hforoughmand

I am delighted that there is someone interested in my project. Since no one has shown interest, I have put it aside after the vast majority of organizations that tracked and published name designations and rankings stopped doing so.

variants.csv is a support file, mainly to obtain the label designation from WHO (or if don't have it, the pango id), the type of interest and which organization (generally it was WHO but there were cases where some other agencies defined a type of interest other than WHO).
This data is used when cross-referenced with other data, mainly to help in the visualization of a particular pango identification.

The project itself was about extrapolating case information by country (data from Our World In Data) with variant information by country (data from outbreak.info), along with scraping data from the status of each designation from WHO, CDC, and other sites.
The pango data is used to create pango.csv which allows for more simpler retrieval and processing of the information provided by pango, primarily for visualizations.

The original idea for variants.csv was that it would be generated automatically, but since the sites from which the information was obtained are no longer maintained and are not available, its generation and maintenance should be done manually from now on.
WHO now has pdf reports (https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports) where it provides information on what the current VOI and VUM are.
With this information, it would be possible to update the data in variants.csv.
I could try to update the table with the information I can obtain from November 2023 to date.

Answering your questions

The 1, It is because WHO stopped maintaining the site https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/ from where live information on the designations they made was obtained.
They began to publish some of this information in their infrequent pdf reports, the work of collecting and keeping this data updated became more complex. Since the topic was no longer of interest and there were no requests from people who were interested in me keeping it updated, I stopped doing it.

The 2, I keep in that csv everything that was active at the time and everything that was important at the time.
You will find VOC and PVOC, as well as VOI and PVOI, the difference between them is that those that begin with P are "previous", which in the past were catalogued with that category of interest.
Since I have not updated since November 2023, there are no new designations and the old designations with their initial letter P have not been passed.

The 3, I would have to see if it is possible to obtain some other source that would allow me to easily obtain this data. This way I could keep my update processing more automatic. Otherwise, I would consider disabling what is currently automatic and moving on to performing periodic searches in the WHO reports to see if I can update my data.
All the sources and information it's is obtained from the sources used in https://github.com/3dgiordano/SARS-CoV-2-Variants/blob/main/script/genomics.py
The project is actually broader than just variants.csv, for variants.csv the sources for classifications and names were obtained from WHO, CDC, ECDC and PHE. As different entities stopped using their own designations and started using WHO's, it was the WHO one that maintained the current statuses, until that page was no longer maintained and available.

The 4 and 5, Unfortunately, as I mentioned, most sites have stopped publishing that information, but perhaps you could try to find the reference dates of the designation. Those who keep track of it, for example, are https://outbreak.info/situation-reports
Outbreak.info appears to be trying hard to keep current information up to date, but historical information does not seem to be available.

Unfortunately, this project at the time was because people were interested in knowing which variants were affecting their country and what their designations were by the organizations that monitored them (mainly WHO).
I dedicated my free time to maintaining and updating the data, but the statistics from the end of 2023 and lack of interest made me stop maintaining it.

At the end of 2023 another open source project evolved incorporating several of the things that I had implemented, its data sources, details and algorithms are not the same, but it was more popular and had more support https://covariants.org/
Maybe they can maintain or provide some csv or incorporate the information as outbreak.info does

Do you have a project that needs it? Have you been unable to find anyone else with similar information or data?

I'm not refusing to do anything, I'm just short on time and need to know if it's worth the investment or not. Maybe you need help finding out about other possible alternative data sources for this designation data.