Webpack-free dev server

[dillonkearns]

I'm thinking through some high-level design decisions for the webpack-free dev server.

StaticHttp requests

When running a build, we need to build every static page, and if we run into a StaticHttp error of any kind, fail the build.

What about the dev server? With the new infrastructure I have set up, I can pre-render a single page on-demand. We could render pages extremely quickly with a workflow like this:

1. User starts up the dev server
2. The dev server compiles the elm-pages app, and now it's ready to go (hasn't performed any StaticHttp requests of any kind yet)
3. The user requests a page from their browser
4. The dev server pre-renders that specific page, runs any StaticHttp requests as needed, and serves up just that page

If the only consideration was speed, this would be the fastest way to get the dev server up and running. There are a few other questions, though.

Subsequent speed

If the server isn't doing any other work, should it start eagerly performing StaticHttp requests to make sure they're ready on request? Or pre-rendering pages?

Consistency

If a page is requested, is it acceptable to have requests that were fetched at different times? They could even be fetched 20 minutes apart. Could this lead to any strange cases of data being out of sync? For example, if you load the data for a blog index from a CMS, then leave the dev server running, then change the titles of some blog posts, you could click on a blog post and end up seeing a different title because the individual post was fetched at a different time than the index page.

Repeated Requests

Another way to approach it would be to perform all requests on-demand, so they're always fresh, when in the dev server mode. This could potentially cause rate limiting issues, though, or undesired extra load.

Error feedback

Is it important that the user get feedback about failing StaticHttp requests (because of failing decoders, HTTP requests, validations, etc?). I think this is quite a nice feature, which the current webpack-based dev server has.

A potential approach

My current leaning, which is similar to the current approach, is:

Run all StaticHttp requests on startup. Cache them, and perform any new requests when there is a cache miss (can lead to data fetched at different times, but you can always re-start the dev server if you want fresh data).

Optionally, we could provide a CLI flag for running the dev server without running any StaticHttp requests until they are needed by a requested page. So the StaticHttp cache would be empty until the developer requests a page from their local browser.

Dealing with File StaticHttp data sources

One simple solution to a live reloading experience for the dev server for file-based content could be to invalidate (remove) all cache entries that depended on a particular file path whenever that file path is changed or removed.

It would probably make sense to re-run glob commands whenever the file-system changes as well.

It could be quite tricky to find all the StaticHttp requests that were continuations from one of these glob or file read requests, though.

So at that point, it does start to feel like just invalidating all cache would be a cleaner option for that case.