navigateFallback causing index.html to be returned for the main JS bundle

[jliebrand]

Not sure if this is a bug, or if I'm just not understanding the intentions here...

What I'm seeing:
1- I push my app, load it, and it loads fine, caching the various assets
2- I update my app, so that the main JS bundle updates (from main.xxxx.js to main.yyyyy.js)
3- reload the app results in an error and a white screen

Basically, what appears to be happening is that the service worker will load the old index.html file, which tries to load the main.xxxxx.js. However, the cache appears to have been updated already, so that only the main.yyyy.js is in the cache.

This then means that main.xxxxx.js is an unknown URL, and thus the navigateFallback means the SW returns the index.html for that script. Which, given that it's html, fails the script parser and you're stuck on a white screen.

It would seem this is a race condition between serving the old main.xxxx.js from the cache, and the new main.yyyy.js being put in the cache.

But since I've not ejected, I would have thought the 'out-of-the-box' settings should work here?

[jliebrand]

Ps. I'm not sure if or how this relates to issue #3248 - so I thought I'd keep it separate for now

[seejamescode]

I don’t think this is related to #3248. Are you still letting CRA insert your script tag for the bundle for you? The default service worker config caches the same versions of the HTML and JS files together.

[jliebrand]

Yep, CRA is still in control of that. And indeed, the correct HTML and JS files appear in the cache. But I think the problem is that it's a cache-first strategy. So once the HTML and JS files are cached, the next time you load (assuming the server has changed), it will first serve from the cache but ALSO update the cache. So the old HTML is loaded (and the cache is updated in the background). The old html is parsed and the old JS is attempted to be loaded, but the now the cache is updated and thus that old JS is no longer found..

It's odd to me that the cache would be updating fast enough for this scenario to happen, but it's the only way I can currently explain what I'm seeing.

Another refresh and all is fine...

[seejamescode]

So once the HTML and JS files are cached, the next time you load (assuming the server has changed), it will first serve from the cache but ALSO update the cache.

This part may be a clue to the solution. Can you share the server code you are using to serve these files?

If you are using Node, I just recommend trying: app.use(express.static("./build"));

[jliebrand]

I'm not using a server - the build/ directory is just pushed to an S3 bucket (which is set as static website hosting)

[seejamescode]

I’m stumped then. If you can share the destination and code repo, that would help!

[jliebrand]

sorry, can't share those as the code is private and the app isn't launched yet...

I'm wondering if the browser cache itself is somehow messing things up. Eg not the SW cache, but the browser's own cache. If it has different expiry times for JS than HTML or something, perhaps that's why one gets out of date? Clutching at straws here though - i'll see if i can get more clues

[jliebrand]

One more thought. Regardless of how I got my setup in to this race condition, I just realised the navigateFallback should not be used when a <script> tag is trying to fetch main.xxx.js at all, should it?

The point is for SPA that want to ensure unknown routes are routed to the main HTML, which I understand. But surely <script> tags requesting content-type:application/javascript should never end up receiving the HTML? Surely the fallback should only be used for text/html requests? (or more accurately, maybe the navigateFallbackWhitelist should have a ![.js$] regex?

[gaearon]

cc @jeffposnick

[gaearon]

In the meantime if this is causing issues you can opt out of caching.

[jeffposnick]

It's odd to me that the cache would be updating fast enough for this scenario to happen, but it's the only way I can currently explain what I'm seeing.

I'm at a loss as to why you'd be seeing this behavior, and it sounds to me like something else might be at play here. Are you doing delayed/lazy-loading of these JS resources?

One more thought. Regardless of how I got my setup in to this race condition, I just realised the navigateFallback should not be used when a <script> tag is trying to fetch main.xxx.js at all, should it?

navigateFallback checks to see if the request being made is a navigation, which would never be the case when requesting a subresource loaded via a <script> tag:

https://github.com/GoogleChromeLabs/sw-precache/blob/19596a2b5569d7c7235d6581894278fe31f50c5a/service-worker.tmpl#L149-L158

It sounds like, perhaps, it's your web server (and not the service worker) that's responding to those script requests with the contents of index.html?

[jliebrand]

ooohhhh I think you might be on to something there... it could be my aws cloudfront... I'll dig some more in to that, and close this bug for now. If I do find something new, I'll raise it separately.

Thanks for your time looking in to this!

[marc1161]

@jliebrand I am facing the same issue as you (have S3 and Cloudfront as web server). When I push a new deployment the first visit ends up with a white screen. Then I refresh again and it works fine. Did you solve it?

[jliebrand]

HI @marc1161 - we do see this issue sporadically... it's hard to pin down though, so I haven't raised a new issue for it.

But ultimately, the way CRA packages the html and JS with hash numbers means (I think) that you can always get in to a situation where the client has the old html cached, but not the old JS... so the old HTML is loaded, which requests (from the server) the old JS, which will fail...

In our case (due to S3 cloudfront setup), that failure results in the server sending back an HTML page, which causes the client to barf (white page; console error about failing to parse <).

But regardless of the S3 setup, having HTML and JS bundle update on each build, means you can theoretically always get in to this mismatch situation...

So i'm not sure what the solution is, short of ensuring that the old HTML+JS remain on the server after pushing the new one... and leaving it there until all client caches have been updated...

[jliebrand]

I think the problem stems from the sw-cache getting updated from underneath the running app.. but not 100% sure yet... I've raised this issue which is probably the better place to track this for now:
GoogleChromeLabs/sw-precache#351

[marc1161]

@jliebrand Thanks for your response! I think the problem is what you are describing as well.. So maybe best would be to upload the index.html and the service-worker.js file to S3 with no caching flags so we can use the custom cache settings from Cloudfront to make this work. I am not sure about it though.

[marc1161]

@jliebrand Are you using S3 Origin Cache Headers or are you customizing the TTL in Cloudfront distribution? I think the problem might be solvable from these two options.

[segulino]

@jliebrand @marc1161 Could someone solve this problem? I have the same configuration that yours (CRA + S3 + Cloudfront), and always after a deploy when I enter to app url, this break trying to access to an older main.js.