downloads counting system

[yalov]

All_downloads counting

Now downloads in GUI is a sum of all downloads for every released version.

pros:

• it's ready and it works - Add download count column to GUI CKAN#2518
• popular old mods > not popular new mods

cons:

• We can't really compare popularity of 2 mods — age of mods and counts of mods' updates have large influence to all_downloads (but not to popularity)

  modA, modB is used by equal amount of people - n:
  modA is old — has 200 updates since 0.25
  modB in the middle — has 20 updates since 1.3
  modC is new — 2 update since 1.4, but is used by 5*n people — "x5 popular"
  but all_downloads shows roughly: "modA 200/20=10 times more popular than modB, and 200/2/5=20 times more popular than modC" ...

• enormous latency — for a new mod to be qualified as popular is not about quality, but about how long the mod will bouncing around

  When Universal Storage II overcome Universal Storage, or SSPEredux overcome SSPE, in few years?
  I think they are already popular.

Last versions counting

Another way to measure popularity is to count not all_downloads, but last_version_downloads, or, even better, maximum of 2 last versions downloads.

pros:

• has some latency — fixes a yesterday-release fall — but not an enormous latency
• fix all all_downloads cons, mentioned above — it become closer to "popularity"
• Github API has downloads per version

cons:

• SpaceDock API needs updating (relative - [Improvement] Download counters by version (not by day) in the "Stats" tab KSP-SpaceDock/SpaceDock#185)
• most likely other resource also would need API updating — not everything is like a Github

UPD

Last time-period counting

On the NetKAN-bot side save all_downloads values every day in a separate file and calculate downloads for the last week or month or few month as difference of numbers in corresponding files, and maybe allow users to switch that.

One still can spam a new version every day for "popularity", but is fixes enormous latency problem, and do not require any API changes.

[HebaruSan]

Another option:

Downloads per day

If the primary concern is that old mods have an advantage over equally good newer mods, we could determine the date of the first release of a mod and then divide the total downloads by the number of days that have elapsed since that date.

Examples

Mod	Total	First release	Days since	Per day
KER	752195	Nov 30, 2014	1372	548.2
Astrogator	33918	Feb 4, 2017	575	59.0
TychoWeaponry	1001	Mar 1, 2016	915	1.1
StationPartsExpansion	147754	Jun 1, 2016	823	179.5
StationPartsExpansionRedux	42634	Jan 28, 2018	217	196.5
UniversalStorage	312079	Mar 1, 2016	915	341.1
UniversalStorage2	3909	Aug 9, 2018	24	162.9

(Data current as of this writing)

Pros

• Smoother trend lines than methods that ignore some versions
• Doesn't need release-specific data, just original release date
• Easy to understand without explanation
• This data is already in GitHub and Curse APIs
• If we store the release dates separately, we're not stuck with one number; user could switch between this and total downloads as an option

Cons

• Mods with lots of releases might still have an edge (but no mod actually has anywhere near 200 releases)
• SpaceDock's API doesn't have this data either, change required (but the UI does have it)

[yalov]

I like Downloads per day and it looks easier than Last time-period and better than All downloads.

But we still have problem with the first — downloads on the last months/releases more important for popularity than downloads years ago.

modA and modB was released 2 years ago, and they both have 10000 downloads, but for modA most of them 2 years ago on the first few releases, but after some time modB overcome, and most of its downloads in the last releases/months. modB is more popular than modA now.

some mods had "beta-period", when they was released on github, but not on SCAN, or something similar. In the Downloads per day this "beta-period" make mod less popular than equal mod without "beta-period".

I think good idea is to start with Downloads per day (user could switch between this and total downloads), but also to save downloads every day on a different files.

Then, after a month, to change default downloads per day to downloads per day on the last month (same thing as Last time-period) and to leave user possibility to switch to other two counting.

[HebaruSan]

SomeDiscoveries while looking for web hook callers in source code...

While SD doesn't have upload date or a granular download count in its main API pages, both are available elsewhere via specially formatted URLs:

• Each mod has an RSS feed (http://spacedock.info/mod/234.rss), which contains pubDate for each release, which could be used to implement Downloads per day
• https://spacedock.info/mod/234/stats/downloads returns a comma-separated-value format spreadsheet listing some random number of downloads per row plus their timestamp and mod version! This could be used by strategies wishing to emphasize recent downloads over older ones (though these files may be too big to be usable, that one is 800 KB)

[yalov]

Some problems with getting upload date from all rss feeds?
Last time-period counting needs just all_download

[HebaruSan]

When Universal Storage II overcome Universal Storage, or SSPEredux overcome SSPE, in few years?
I think they are already popular.

I remembered reading this and felt like running the numbers. It looks like US2 just isn't nearly as popular as US used to be:

[yalov]

US2 just isn't nearly as popular as US used to be

Why? The chart doesn't show 'used to be' time interval.

In the last 2 years 80k US2 vs 40k US1 — 2x difference; even larger difference for the last year

[HebaruSan]

US2 just isn't nearly as popular as US used to be
Why? The chart doesn't show 'used to be' time interval.

In the last 2 years 80k US2 vs 40k US1 — 2x difference; even larger difference for the last year

300000/80000 = 3.75, 300000/40000 = 7.5.
US1 was first uploaded to SpaceDock on 2016-03-01, so ~2.5 years before the graph. So it outpaced US2 by about 1.5 times.

[yalov]

The game itself could become less popular, that entails new mods been outpaced by the old mods by raw downloads count

[HebaruSan]

Yes, that's one possible explanation.