Pulsar should bundle a terminal package

[savetheclocktower]

As someone who doesn’t even use a terminal package within Pulsar, it's taken me a while to come around to this idea, but I’m finally there.

Why?

“How do I install a terminal package” is probably the most common question we get across our support channels. We already don’t do a good job of directing folks to x-terminal-reloaded (the one terminal package that is under active maintenance); and once they know which one to install, they’ve got to navigate this documentation page and they need to have build tools installed because of the native module used by the node-pty dependency. Even if they're developers, they'll likely have to install some new stuff unless they're Node developers.

This is especially tricky to get right on Windows. If this were for an obscure package, it’d be easier to justify making new users jump through these hoops, but it’s for an extremely common use case, and one that they get for free with VS Code.

And even if we didn’t care about the user’s experience: the amount of time we’ve devoted to helping users install x-terminal-reloaded would probably be better spent on bringing it into the core so that it Just Works. It’s ongoing support debt.

For me, the last straw is the (seeming) realization that our new and improved x-terminal-reloaded doesn’t work (in its current form) on Windows because recent versions of node-pty use the builtin Node worker_threads library on Windows — and Node worker threads are not architecturally possible in Electron’s renderer process. This is not a show-stopper (if nothing else, we can probably run a separate worker process that is responsible for using node-pty) but it strongly suggests to me that we’d benefit from bringing a terminal package into the core. Among other things, it would allow us to do sanity checking across platforms via unit or integration testing. @Spiker985 uses Linux, I use macOS, and even though I’ve been working on PulsarNext for more than a year, I didn’t catch this issue until now because that’s how infrequently I run Pulsar on my ancient Windows 10 gaming PC.

Upsides

• A terminal package, if shipped as a builtin, will be pre-built for the user’s platform and processor architecture, freeing them from the need to install a Node toolchain. The user is always free to disable this package and install one they like better, but we should make sure that the builtin package meets the use cases of 80% of users.
• I think we can justify its inclusion as a core package merely from how widely used it would be; but even if it were a borderline case, I’d argue that the advantage described in the previous bullet point is compelling enough to give it some extra oomph.
• As @Spiker985 says in the x-terminal-reloaded README: “Eventually, I'd like to transition this over to the Pulsar repo.” “Eventually” has arrived!

Downsides

• Extra weight. Bigger download, and theoretically more of a performance hog — but if we do it right, it won’t even activate until the user asks it to appear. And the user would be free to disable it, as they are free to disable any other core package.
• Possibly increases the amount of time it takes to build Pulsar binaries, along with other associated CI headaches.
• x-terminal-reloaded has a lot of… baggage?… since it’s a fork of a fork of a fork of a fork of a fork of a package called term. We can use it as the basis for a new core package, but we’ll still probably want to do some streamlining.

Alternatives

This is a workaround for the larger issue: it’s annoyingly hard to install packages with native module dependencies. But I’m not aware of any good way to fix that in the general case:

• Prebuilds fix it, but require that the package’s author (node-pty in this case, but could be any other arbitrary package!) is creating prebuilds as part of their own CI publishing pipeline, and many don’t. (Sometimes another person comes by and “fixes” that, but then that’s just another point of failure. node-pty-prebuilt-multiarch served this function for node-pty for a while, but has been idle for four years.)
• We can’t “vendor” our own build toolchain, obviously. We could streamline the process of obtaining the build tools when they’re not detected (perhaps building this logic into ppm?) but that’s just another thing to support, and suggests a dystopian future where we have to make a PR to ppm every few months to update a download URL or a link to documentation page we don’t host.

I can’t even think of a third thing to round out the list. Both of these options suck more than just building a package called terminal and bundling it with Pulsar. Let’s just do that.

[Spiker985]

Agreed! There are quite a few issues I have with XTR, notably the settings system is clunky, the reliance on node-pty is ahem challenging to say the least and it's not portable due to the previous challenges.

Honestly, I think any internal replacement would be better off to not fork (or have me transfer) XTR due to how unwieldy the codebase is.

It would likely be faster to prototype one from scratch, while using XTR and other terminal providers as guidelines for features instead of as a base for implementation

The interesting thing about moving this into a first party support model: node-pty could be built during the bundling (like the GitHub package's native module) and then it should no longer require rebuilding. Since xterm.js is the frontend of the terminal inside of the editor, but node-pty is required to interface with the actual terminal product ie bash, powershell, etc. You always need some kind of pty in order to interact with your terminal.

VSCode and others build and bundle node-pty within their CIs and provide them as a part of their native installs, which is why it's a non-issue for them.

However, it does look like the kind people on node-pty are working towards making prebuilt node-pty a reality! microsoft/node-pty#46 (comment)

[savetheclocktower]

This is basically ready at pulsar-edit/terminal. I didn't bother to try to make it compatible with Electron 12 (I used Node 20+ dependencies, for instance), so it needs to wait until we land #1367.

I started #1373 just to add some terminal colors to the built-in syntax themes, but I might expand it to encompass everything involved with adding terminal as a core package.

If you want to try it out, make sure you're on the #1376 release candidate — not just ordinary PulsarNext, because one of the dependencies' post-install scripts relies on Node 20! — and run ppm install pulsar-edit/terminal. If anything goes wrong with installation or usage, let me know.