Fastload Utils: Implement and strictly enforce whitelist

[thyttan]

Thanks! Looks like it's because the messagegui's music page uses layout with a remove handler - which makes fastload think it can remove it and have everything unloaded. That's going to happen for any app that uses remove in a setUI/layout/etc and doesn't actually fully remove itself.

... Perhaps what we need is actually a whitelist of apps that support fastload - because I feel like for pretty much every casual user, it's just a fantastic way to make your Bangle really unreliable.

Originally posted by @gfwilliams in #4077 (comment)

I imagine the whitelist to be hardcoded without being configurable by the user.

Ping @halemmerich as original author.

[gfwilliams]

Yes - I guess either we just make an app set Bangle.fastload=1 or we add something to the app.info file from the metadata?

Another thing I think we should check for is since we're loading the app file and checking for loadWidgets in it, we should also check if the file begins with { or (function() { since without it, it's pretty much guaranteed that the app will leak memory as it defines vars in the global scope.

It wouldn't work well for non-pretokenised code, but if you're using fastload you probably care about speed enough to want pretokenising on anyway

[gfwilliams]

Ok, so I did a bit of searching with a script to see how many apps you could use fastload without leaking memory (checking for loadWidgets, remove/uiRemove, and then manually checking correct scoping). So far I have:

• apps/calendar/calendar.js - maybe, but it doesn't unload the font
• apps/gbdiscon/app.js - fast loads (works without fast load utils)
• apps/gpstrek/app.js seems close but it never sets uiRemove so never fast loads
• apps/grocery/app.js - fast loads
• apps/heatsuite/heatsuite.app.js - fast loads, but uses function so leaks memory
• apps/messagelist/app.js - fast loads I think (but has comments about not using Bangle.load because of debugging)
• apps/multitimer/app.js - fast loads, but uses function so leaks memory
• apps/podadrem/app.js - fast loads,
• apps/qcenter/app.js - fast loads,
• apps/rebbleagenda/app.js - fast loads (works without fast load utils)
• apps/spotrem/app.js - fast loads,
• apps/stopwatch/stopwatch.app.js - fast loads,
• apps/torch/app.js - fast loads (works without fast load utils)

So out of 546 apps (I ignored clocks and launchers which do it already), only 61 will attempt fast loading with fastload utils. Of those, all but 10 of them leak memory, most of them a lot and 3 of those 10 would have worked anyway. All the other apps will reload slightly slower because of fastload utils being installed.

I haven't actually tested the ones I think are ok, so some may still leak memory.

But honestly, this just makes fastload feel like a massive trap for users, and I don't think it should exist Calling something that only improves 1% of apps, slows down the rest and actively causes issues on 10% 'Experimental' seems very charitable.

Having looked into this, I realised that fastload utils just calls Bangle.load() to do the fast loading. So really, our 'whitelist' should be just swapping load for Bangle.load in the few apps that support it without leaking memory so they load quickly for everyone, regardless of whether fastload is installed. And then fastload utils should maybe just be renamed to 'app history', and it can override load and Bangle.load to allow it to do the app history thing, but without trying to force apps to fast load that'll just create problems.

[gfwilliams]

I just added a branch as https://github.com/espruino/BangleApps/tree/fastload_apps_by_default_4099 that ensures fast loading on the apps where it seems very possible and makes sense.

I haven't had a chance to properly test they don't leak memory yet so not merging

[thyttan]

I haven't had a chance to properly test they don't leak memory yet so not merging

If you want you can leave the testing of those to me.

[thyttan]

Having looked into this, I realised that fastload utils just calls Bangle.load() to do the fast loading. So really, our 'whitelist' should be just swapping load for Bangle.load in the few apps that support it without leaking memory so they load quickly for everyone, regardless of whether fastload is installed.

I'm happy with that.

I know way back when fastloading was first implemented you were wary of loops/long chains of fastloads. From POC - Combining BW Clock and Desktop Launcher to spend less time "Loading..."
#5398 on the forum:

gfwilliams said:

The issue here is it's not just clock -> launcher -> app but maybe clock -> launcher -> app -> app -> app -> app -> etc and so any leaks will really add up.

[...]

... and even if we tagged apps that did clean up after themselves, I think it's unlikely that each contribution to an app in the future would be checked to see that it didn't leave memory behind.

halemmerich answered:

Clock <-> Launcher: Should be relatively easy to check for memory leaks during development and is one of the most done transitions. This works well on current firmware with setUI-remove and the eval style of loading the clock.

Launcher -> App: The only point that should not be leaking memory is the launcher, since it needs to remove itself to free up memory for the app. The app should not see any difference to beeing loaded after the normal boot code then.

App -> Launcher/Clock: Technically the same as launcher to app, but a lot more apps to check and validate. Modification of all apps and keeping them clean in regard of memory use is not really feasible. Current state of affairs is a reboot of the system if no remove-method is implemented. This also mitigates smaller leaks by rebooting every few app launches. This waiting time is not really a problem, since closing an app often means stopping use of the watch anyway. At least for me the delay after using an app is a lot less impactful than the waiting for an app to open.

All this is to say, there is still the option to more strictly say that apps other than clocks and launchers should not be fastloaded AWAY FROM. I am not the biggest fan of that stance but see the benefits of more peace of mind for maintainability and lessened complexity. Maybe it's a runaway train that better not keep moving?

What I thought Fastload Utils gave us is a middle ground. Where for absolute majority of users fastloading is done exclusively for Clocks <-> Launchers and potential memory leaks is mitigated due to no long chains of fastloads. Those that installed Fastload Utils would mainly benefit from fastloading Launchers -> Other apps (TO other apps, not AWAY FROM them generally) and in the rare cases other apps implemented a remove handler then AWAY FROM those as well. Now knowing e.g. Layout library adds remove handlers that "trick" Fastload Utils that it's safe to fastload makes this a bit more questionable for me.

What I would be most sad to see go would be the fastloading of Launchers -> Other apps. Maybe we could make that work without Fastload Utils. I don't think it's as straight forward as just changing from load() to Bangle.load() in launchers though (i.e. how to handle widgets).

[thyttan]

And then fastload utils should maybe just be renamed to 'app history', and it can override load and Bangle.load to allow it to do the app history thing, but without trying to force apps to fast load that'll just create problems.

I'm happy to look at making app history its own thing. I've thought about that before.

[gfwilliams]

If you want you can leave the testing of those to me.

That would be great, thanks!

What I would be most sad to see go would be the fastloading of Launchers -> Other apps.

Thanks! Yes, sorry - that makes an awful lot more sense, and I'm totally with you on that - fast loading from the launcher should be pretty safe as they already unload themselves. I'd forgotten/hadn't really thought that through.

Ok, so, personally, I feel like maybe fastload should work only from launcher -> app by default? That's pretty safe, and a good improvement for a lot of people - maybe other apps could be enabled via settings, but to be honest I think it's better to modify the apps themselves (load -> Bangle.load()) as I got carried away with checking above.

[thyttan]

Ok, so, personally, I feel like maybe fastload should work only from launcher -> app by default? That's pretty safe, and a good improvement for a lot of people

I find myself agreeing with that. The question I have then is if we should just make that the default without the use of Fastload Utils? Either by modifying Bangle.load in firmware or adding a library for launchers to use. But that may be a consideration for another time.

[...] but to be honest I think it's better to modify the apps themselves (load -> Bangle.load()) as I got carried away with checking above.

Yes - I think we could start allowing this more explicitly in the fastload docs/tutorial. With a disclaimer that we discourage from it and that if we are made aware of instability/memory leaks we will push updates to just disable it, changing back to load() commenting out the remove-handler and adding a comment in code // fastloading disabled due to instability or such.

[gfwilliams]

It's a good point about just enabling fast launch globally - it would be a great improvement. Right now I believe most launchers use launch.cache.json and we copy/paste the same code in each launcher to handle that. I wonder whether we should pull that out into something in the modules folder, and then we could handle it in the same way for every launcher. The cache could then store whether loadWidgets was in the app or not.

I guess it might make sense to make the app loader put whether an app was fast loadable in the .info file? I seem to remember some apps had an option about whether to load widgets or not, so a simple text search for it wouldn't be ideal - using the app loader gives us some options like checking if loadWidgets is in the root scope or not

I think we could start allowing this more explicitly in the fastload docs/tutorial.

Yes, that sounds good, I wouldn't say I discourage it - I think it's good, it just needs to be enabled only when folks are sure there isn't a memory leak :)

[thyttan]

And then fastload utils should maybe just be renamed to 'app history', and it can override load and Bangle.load to allow it to do the app history thing, but without trying to force apps to fast load that'll just create problems.

I'm happy to look at making app history its own thing. I've thought about that before.

I had a go at breaking out app history to a new app. It seems to work well on its own. But if both it and Fastload Utils are installed they seem to interfere so their behaviors become unreliable - since they both overwrite Bangle.load. I reckon if we do the changes moving fastload responsibilities to firmware or a library that unreliability should go away.

• Here's the branch: https://github.com/thyttan/BangleApps/tree/app-history-from-fastload-utils
• And a diff against espruino/BangleApps: https://github.com/espruino/BangleApps/compare/master...thyttan:BangleApps:app-history-from-fastload-utils?expand=1
• Here it is on my app loader: https://thyttan.github.io/BangleApps/?id=apphist

[thyttan]

I haven't had a chance to properly test they don't leak memory yet so not merging

If you want you can leave the testing of those to me.

That would be great, thanks!

The results:

"calendar" has a small memory leak (2 units per cycle).
The others do not.

// "calendar" has a small memory leak (2 units per cycle).
// The others do not.
// Ran this as test, uploaded to ram in Web IDE:

function run(app) {
  print("\n\nStart memory test for " + app)
    print(process.memory().usage)
    Bangle.loadWidgets();Bangle.drawWidgets();
  print(process.memory().usage)
    for (i=1; i<=10; i++) {
      print(i)
        eval(require("Storage").read(app+".app.js"))
        print(process.memory().usage)
        Bangle.setUI()
        print(process.memory().usage)
    }
}

apps = ["spotrem", "podadrem", "calendar", "stopwatch", "grocery","qcenter"]

function runAll(apps) {
  print("\n\n\n# Test all apps")
    apps.forEach(run)
    print("\nAll tests finished.")
}

setTimeout(runAll, 1000, apps)

The output of the script: (click to expand)

>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v28.37 (c) 2025 G.Williams
>
# Test all apps
Start memory test for spotrem
745
1272
1
2599
1452
2
2599
1452
3
2599
1452
4
2599
1452
5
2599
1452
6
2599
1452
7
2599
1452
8
2599
1452
9
2599
1452
10
2599
1452
Start memory test for podadrem
1452
1452
1
2019
1452
2
2019
1452
3
2019
1452
4
2019
1452
5
2019
1452
6
2019
1452
7
2019
1452
8
2019
1452
9
2019
1452
10
2019
1452
Start memory test for calendar
1452
1452
1
2145
1687
2
2147
1689
3
2149
1691
4
2151
1693
5
2153
1695
6
2155
1697
7
2157
1699
8
2159
1701
9
2161
1703
10
2163
1705
Start memory test for stopwatch
1705
1705
1
2054
1708
2
2054
1708
3
2054
1708
4
2054
1708
5
2054
1708
6
2054
1708
7
2054
1708
8
2054
1708
9
2054
1708
10
2054
1708
Start memory test for grocery
1708
1708
1
1977
1708
2
1977
1708
3
1977
1708
4
1977
1708
5
1977
1708
6
1977
1708
7
1977
1708
8
1977
1708
9
1977
1708
10
1977
1708
Start memory test for qcenter
1708
1708
1
2049
1767
2
2049
1767
3
2049
1767
4
2049
1767
5
2049
1767
6
2049
1767
7
2049
1767
8
2049
1767
9
2049
1767
10
2049
1767
All tests finished.
>

[gfwilliams]

Looks great, thanks! I'll merge that now.

I actually just looked at the memory leak in calendar with EspruinoMemView, and I appear to see a bunch of:

Turns out it's an Espruino bug with a memory leak somewhere in the JIT interpreter (espruino/Espruino#2674) - I'll track this down now, but either way 2 vars isn't going to cause big issues for anyone on current builds.