Add the last commit sha of each addon to the cache

[Reqrefusion]

An addition to #137, this could be very useful for downloading without git. What do you think @chennes .

[chennes]

Can you explain the use-case here? What would you do with this hash if you don't have git?

[Reqrefusion]

Can you explain the use-case here? What would you do with this hash if you don't have git?

This is necessary for the delta patching I mentioned on Discord. A list of changed files can be retrieved through the API, and only the changed files can be downloaded. For example, a workbench that only changes the readme doesn't need to re-download the entire thing. Or, for large workbenches, it doesn't have to be re-downloaded every time a file changes. For Github, this query can be made 50 times an hour without requiring any keys. Downloading individual files doesn't require an API.

[chennes]

We may be better off implementing our own system server-side so we aren't locked into GitHub (and so users aren't subject to their rate limits). We could create an endpoint on addons.freecad.org that takes a date and an addon ID and returns a zipfile containing only the changed files since that date, for example.

[Reqrefusion]

We may be better off implementing our own system server-side so we aren't locked into GitHub (and so users aren't subject to their rate limits). We could create an endpoint on addons.freecad.org that takes a date and an addon ID and returns a zipfile containing only the changed files since that date, for example.

I considered using something server-side for this, but there's a dependency issue with getting things server-side. Actually, doing things server-side is easier for me. Also, while the date is good, the latest sha is much more functional. This is needed even when doing something server-side. It's more functional for git APIs. Because even though we're building a server backend, we still need to interact with these APIs. While the last date requires two API requests, the final sha only requires one.

[chennes]

git log --since="2025-07-18T02:34:56+0200" --name-only -- just a single git call per user request.

[Reqrefusion]

git log --since="2025-07-18T02:34:56+0200" --name-only -- just a single git call per user request.

Sorry, I was evaluating it via the API and forgot that the compare API does date-based comparisons. Perhaps it's because I always prefer to use sha-based to be a bit more secure. Yes, it's possible to get this in one go for the API with the last update date.

API usage with last update date

API usage with last sha

I've created a prototype for the server-side solution. You can try it out here.

[chennes]

Part of my point was that we don't have to use the GitHub API at all - we have local clones of all of the repos server-side, we can simply use git directly. This means that we are not tied to GitHub, and the calls will work regardless of which git host is being used.

[Reqrefusion]

Part of my point was that we don't have to use the GitHub API at all - we have local clones of all of the repos server-side, we can simply use git directly. This means that we are not tied to GitHub, and the calls will work regardless of which git host is being used.

Instead of an intermediary script, the server is also required to host the files. Of course, if we want to do it server-side, why not move everything there? Of course, the GitHub API seems more logical to me, since I don't have a server to copy local clones from. If that server is already set up, you can easily create a system that outputs changed files using Python. I don't think you'll prefer my PHP code.

[chennes]

Addons.freecad.org keeps copies of all of the Addons. You can see the script that generates the cache in this repo: https://github.com/FreeCAD/AddonManager/blob/main/AddonCatalogCacheCreator.py

[Reqrefusion]

This requires installing an API system on the server. How this API is set up is a matter of choice. My preference is to do it directly with PHP, but this doesn't fit the other components. I'm likely trying to do it with Python, which has several options, and one needs to be chosen. I've decided not to do anything until this choice is made.