Proposal: "Install App from Files" option in App Loader

[kidneyhex]

Would there be interest in adding an Install App from Files feature to the App Loader?

This would let you select an app's files directly and install them, without needing to manage branches or rebuild apps.local.json.

I've built a working version here:
Demo: https://kidneyhex.github.io/BangleApps/
Code: https://github.com/kidneyhex/BangleApps/tree/install-app-from-files

(Under "More..." there's a new "Install App from Files" button.)

I'm still testing edge cases, but it's working well so far. Initially I considered folder uploads, but that required a change to espruinotools.js, so for now I'm using "select all" within a folder.

Why

My BangleApps repo is a tangle of poorly named branches and commits, that I keep on my laptop out of shame for their unspeakable horror. And I constantly forget to rebuild apps.local.json each time I change branches.

I'd rather just keep my temp apps in their own private folders/repos with just their own code, unless I plan on sharing them.

I know the App Forge app exists, but this just feels like a more complete and user-friendly option.

[thyttan]

Hi,

Just sharing how I do it.

I have a branch named "app-loader" that I point my gh pages to. Then I merge in branches where I work on different things when I want to load them from my app loader.

[thyttan]

I'll elaborate some on my flow then :)

First off I rarely use the github web interface for changes and mostly rely on git in the terminal.

Basically, I use two types of branches in my repo-fork (thyttan/BangleApps):

1. the branch named "app-loader" (I point my gh pages to this one) and
2. development branches.

A development branch remains focused on one thing, more or less. E.g. I don't work on more than one app per branch - if apps are not interdependent in some way that would motivate it. This means it's easy to do nice pull requests to upstream (espruino/BangleApps). Just select the branch and there's not much more to it.

To test the app out during development I merge the focused development branch into the "app-loader" branch and push that to the corresponding "app-loader" branch at origin (thyttan/BangleApps on github).

So the "app-loader" branch may have changes to many apps, but that is only because I merged several development branches into it. I never use that branch to do pull requests to upstream.

Sometimes an interactive rebase of "app-loader" on upstream master branch is in order to clean things up. Or just a hard reset. Everything is preserved in the corresponding development branches so nothing is lost.

I want to start using localhost for my app loader also as I think that would be faster than waiting on gh pages to build. But I haven't got the hang of that. I'd really appreciate pointers!

And BTW - I'm not really sure what a monorepo is so I'm not a git guru :P

[thyttan]

Also, here's some info on branches written up by github if it would interest you: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-branches

[kidneyhex]

Monorepo just means everything's in one big repo, instead of separate repos for each app/project.

In my mind, after develop merged into app-loader, it closed and I couldn't add more. I must have just been deleting branches after merges without thinking!

gitGraph
   commit id: "start"
   branch app-loader
   checkout app-loader
   branch develop
   checkout develop
   commit id: "stuff"
   commit id: "more stuff"
   checkout app-loader
   merge develop
   commit id: "something else"
   checkout develop
   commit id: "omg...its still open"

Loading

I'll try writing up some documentation about how to run the app loader locally. I'm sure it's a pain point for a lot of people so some alternative tutorials for different ways to do it might be good.

=P I've decided I can't go back to pebble after having widgets and JS for this long, so I'm going to try to get off my butt and help out a bit more.

[kidneyhex]

@thyttan Here's a rough draft of how to run the app loader locally with vscode/codium: https://github.com/kidneyhex/BangleApps/wiki/Getting-Started-for-Windows-Users

[thyttan]

Thanks! I'll use that as reference although I'm a Linux + neovim type 😛

[gfwilliams]

I'd be interested to see what others think about this - you've had to do a lot of code to get that working! Personally I find that having the App Loader able to check versions and remind me only when I need to update an app is really useful.

Out of interest, what's the reasoning behind manually creating the metadata in espruino/BangleApps@master...kidneyhex:BangleApps:install-app-from-files#diff-3b1e99bc6bdd9d53a60e32feb6ac9b5be61e9e77138bf6dd4d714ea05b9356a4R107-R119 - couldn't you just use the file as-is?

I'd be a bit concerned that if we added any fields in the future it's unlikely that'd be updated and then the file upload might break. I'd have thought that not much work would be needed before calling Comms.getAppInfo - literally just going through files that have a url, grabbing the data from a zip and putting it in the content field.

And I constantly forget to rebuild apps.local.json each time I change branches.

Just delete it - as long as you're serving from a server that has directory listings enabled (which is easy enough to do usually), the app loader will automatically load all the apps even without that file. It's what I do here (I have a few apps that aren't even in Git).

[kidneyhex]

Personally I find that having the App Loader able to check versions and remind me only when I need to update an app is really useful.

I just use this for my own "under development" stuff, so I'm the only one updating them.
The App Loader works great for everything else.

No worries if nobody else wants it -- I'll still use it myself, just won't bother polishing it up and submitting it.

just delete it - as long as you're serving from a server that has directory listings enabled

*facepalm* I've been using the "Live Server" plug-in in VS Code, and the app folder scan just silently fails so I never noticed it was trying.

couldn't you just use the file as-is?

*facepalm again* Yeah, that would make more sense. I started with a small hard coded version, then expanded to reading the real file and forgot the file *is* the app info.

[kidneyhex]

1. I've simplified the install from files branch; a lot of code I had copied while messing around was redundant.

2. The apps folder scan is skipping if file paths start with "/", but Live Server's directory listing has them all like "/apps/appfolder". So it doesn't error because it does get a directory listing, but it doesn't find any of the apps.

https://github.com/espruino/EspruinoAppLoaderCore/blob/9953b46ca83ee9f1cb3f8ad9fb4e413ec5f0b918/js/index.js#L128-L131

But with that fixed, yeah... that is a lot easier. 😄

There's a short fix with a rough regex:

      // Skip any that are not /?(apps/)?(appname)/? folders
      if (!href || href.match(/^\/?(?:apps\/)?[^\/]+\/?$/) === null) {
        //console.log("Skipping href: '" + href + "'");
        return;
      }
      href = href.match(/\/?(?:apps\/)?([^\/]+)\/?/)[1];
      let metadataURL = appsURL+href+"/metadata.json"; // Remove an extra "/" in there

Or the "probably over doing it" option:

/**
 * Extract an app name from a /apps/appname path or URL
 * Just don't name an app "apps"... :P
 * Returns the app name string or null if not an app folder.
 */
function extractAppNameFromUrl(href) {
  if (!href) return null;
  
  try {
    // In case server is giving us full paths; strip that off
    const u = new URL(href, window.location.href);
    href = u.pathname;
  } catch (e) {
    // ignore - just use href as-is
  }
  // unlikely, but get rid of query/hash... just in case 
  href = href.split('?')[0].split('#')[0].trim();
  // remove leading/trailing slashes
  href = href.replace(/^\/+|\/+$/g, '');
  if (!href) return null; // was just /, throw it out
  
  const parts = href.split('/').filter(Boolean);
  if (parts.length === 0) return null; // was just /, throw it out
  // get rid of leading 'apps' segment
  if (parts[0].toLowerCase() === 'apps') parts.shift();
  if (parts.length === 0) return null; // was just apps, throw it out
  // first segment is the app name
  return parts[0];
}
. . .
        const appName = extractAppNameFromUrl(href);
        // Skip anything that doesn't look like an app folder
        if (!appName) {
          //console.log("Skipping href: '" + href + "'");
          return;
        }
        let metadataURL = appsURL + appName + "/metadata.json";

Edit: I forgot to exclude the underscore apps in the code above. Will update that later or in a pr when I get a chance.

[gfwilliams]

Thanks for that PR! It's looking good.

Thanks for tidying up the installing apps from files thing too. It's nice and self contained - if you wanted to do a PR I'd be happy to have that in. It's always good to have the option as long as it's not too much of a burden to maintain, and I don't think that will be now :)

[kidneyhex]

I realized that hitting cancel on the file dialog was leaving the operation open and blocking other operations.

So, a couple new options for the fileOpenDialog might come in handy?

• Add an options.onCancel to tell that the user closed the dialog
• Add an options.onComplete so multi-file uploads can tell when all files have finished

I've been using timeouts to guess when the files were done... which probably isn't great. I don't think there was any other way though?

What I'm thinking of for the fileOpenDialog: kidneyhex/EspruinoAppLoaderCore@9d2f8e7

If that sounds ok, I'll test a bit more and then submit both PRs.
Also, would you prefer mixed case onCancel/onComplete or lowercase oncancel/oncomplete?

If not, that's fine too -- between the apps folder scan and learning I was using git wrong, both have made things so much easier.

[gfwilliams]

Thanks - maybe it's better if we actually call the callback with data=undefined on cancellation? That's what we appear to be checking for in the rest of the code that uses it? But onComplete sounds like a plan.

Changing fileOpenDialog is a bit of a pain as it's actually part of https://github.com/espruino/EspruinoTools/blob/master/core/utils.js that's merged together and included. Not sure how you feel but it seems to me like if we're changing it, maybe we should make it handle loading all the files, and returning them in one go in the callback function?

The only place we use multi is in https://github.com/espruino/EspruinoWebIDE/blob/master/js/plugins/storage.js#L177 and that could do with changing too, as right now it just pops up X many windows at once, one over the other - which isn't ideal!