Return a content-encoding header for resource timing and more

[guohuideng2024]

The major change is to add content-encoding to response header list. This PR also adds description on how content-encoding is determined. (content negotiation)

The purpose is to pass such value to resource timing. Further details are available at
w3c/resource-timing#381.

Note: Per discussion at 12/05/2024 webPerWG call (https://docs.google.com/document/d/1mpFDrAWuV6IgvJ1KiL9sgIlcboC5uArtF8r_oqS1Sco/edit?tab=t.0#heading=h.af6v74wysf4m), we decided to allow arbitrary "content-encoding" value at "fetch". We only filter such value at client side, before passing the value to resource timing.

Related PR to modify resource timing specification:
w3c/resource-timing#411

• At least two implementers are interested (and none opposed):
  Likely, Discussed and agreed at Feb 29, 2024 W3C WebPerf call, Chromium already receiving content-encoding through fetch. But not sure about other browsers.

• Tests are written and can be reviewed and commented upon at:
  [to be updated with a new link] https://chromium-review.googlesource.com/c/chromium/src/+/5958411

• Implementation bugs are filed:
  Chromium: https://issues.chromium.org/issues/327941462
  Gecko: https://bugzilla.mozilla.org/show_bug.cgi?id=1886107
  WebKit: https://bugs.webkit.org/show_bug.cgi?id=271632
  Deno (not for CORS changes): …
  MDN issue is filed:
  New PerformanceResourceTiming.contentEncoding field mdn/content#32823

• The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

Bug: w3c/resource-timing#381

---

Preview | Diff

[annevk]

Thanks for taking the time to pick this up. However, it doesn't seem like this addresses all the issues with #1742? I recommend studying the feedback on that PR.

[guohuideng2024]

Thanks for taking the time to pick this up. However, it doesn't seem like this addresses all the issues with #1742? I recommend studying the feedback on that PR.

Hi Anne! I think I should have put up some background information here.

1. You mentioned in Pass in Content-Encoding to resource-timing #1742 that the spec must define how the value is determined. This PR is trying to do that. The value is a result of the "content negotiation" (determine what encoding should be used) so I tried to add that into the existing text.
   Note that Pass in Content-Encoding to resource-timing #1742 is a change similar to one for a previously added field contentType. But contentEncoding is very different, it's not an extracted MIME type, but a result of "content negotiation". So, this PR should be very different from Pass in Content-Encoding to resource-timing #1742

2. We originally thought that the filtering should happen at the "fetch" stage. But in the last web perf meeting Patrick brought up that the returned contentEncoding can be a proprietary value and that value is needed by service worker. So, the unfiltered value must be kept by the browser and the filtering should happen right before reported to resourceTiming.

https://docs.google.com/document/d/1mpFDrAWuV6IgvJ1KiL9sgIlcboC5uArtF8r_oqS1Sco/edit?tab=t.0#heading=h.af6v74wysf4m

Therefore, in this fetch doc I didn't mention filtering. I mentioned "filtering" in the resourceTiming spec:
w3c/resource-timing#411
And I am going to add more details about the filtering there.

Does this sound right to you? I am new to fetch and I may have missed a lot of things here. Thanks for your patience and guidance.
Guohui

[guohuideng2024]

Updated the patch, I just added the content encoding to the body info struct, and add the clause that updates it.

[guohuideng2024]

very sorry for so many mistakes folks. Thanks for you guys' patence.

[noamr]

very sorry for so many mistakes folks. Thanks for you guys' patence.

No worries, we've all been there! (Or at least I have...)

[guohuideng2024]

I think it should be specified here as that matches how we do MIME types and that reduces the chances of someone inadvertently exposing the information. In other words: the guarantee should come from Fetch, not from the caller.

The "raw" contentEncoding value can be arbitrary proprietary compression the app uses, and it's leaked as a response header.
So it's indeed a new communication channel that's created :(.
Meanwhile I think moving the filtering here guarantees that the only place where the raw contentEncoding is leaked is the fetch response header. I would say something here that contentEncoding is filtered when accessed anywhere else.
If there is any concern pls let me know. Thanks.

Specifically, it needs to be explicitly filtered when assigned to the response body into struct.

Got it, Thanks! I updated the PR accordingly.

[guohuideng2024]

To be clear, the header is not exposed to the website passively embedding the resource, but this getter is. I don't think I understand your suggestion, could you rephrase?

I think the website can get the arbitrary value like this: (I am new to this area so please correct me if I am wrong)

let myCoding = myHeaders.get("Content-Encoding");  //  |myCoding| can be a proprietary compression, i.e., an arbitrary value.

And the reason for that is some use cases involving service workers. See
w3c/resource-timing#381

but the contentEncoding field in resourceTiming is filtered, where only a few pre-determined values are permitted.

[noamr]

To be clear, the header is not exposed to the website passively embedding the resource, but this getter is. I don't think I understand your suggestion, could you rephrase?

I think the website can get the arbitrary value like this: (I am new to this area so please correct me if I am wrong)

let myCoding = myHeaders.get("Content-Encoding");  //  |myCoding| can be a proprietary compression, i.e., an arbitrary value.

You would only get access to myHeaders if this is an actual fetch or via a service worker; Those channels are not always available.

And the reason for that is some use cases involving service workers. See w3c/resource-timing#381

but the contentEncoding field in resourceTiming is filtered, where only a few pre-determined values are permitted.

Yea, so filtering them when assigning to the struct wouldn't change anything observable, but any future user of that struct would get the filtered value.

[guohuideng2024]

Thank you Noam!

@annevk : Would you take one more look? (I also left a response at WebKit/standards-positions#467 )

[annevk]

I noticed you're participating on behalf of Microsoft. That means you cannot sign the contributor's agreement as an individual. Microsoft has already signed up for the Fetch Workstream so you have to join the relevant GitHub organization (MicrosoftWHATWGContributors) and make your membership thereof public.

[annevk]

Now with the filtering in place it's probably slightly more reasonable to leave the existing parsing issue unsolved for now, assuming there's adequate test coverage.

[guohuideng2024]

Thanks @annevk! I am working on MicrosoftWHATWGContributors membership right now.

[guohuideng2024]

update:

1. Since I couldn't register a new reserved "unknown" value for "content coding", per the discussion with Roy and Yoav, I am replacing "unknown" with "". "" cannot be used as a "content coding" value so it won't cause value space problem.
2. I removed the internal reference because the reference has been registered into "specref".

[guohuideng2024]

In the http spec I found the syntax restriction of the "content coding", which gives us a complete list of possibilities.
(see the end of this comment for more details)

@unknown looks good to me but I would like to confirm we don't want to use ?unknown nor [unknown] instead? @annevk

Hopefully nobody objects and I will update this PR with @unknown this Wednesday.

Cheers,
Guohui

+++++++++ details from http spec +++++++++++++++++++++++++++
http spec actually defines the syntax restriction of "content-coding" parameter.
Link to the http spec: https://www.rfc-editor.org/rfc/rfc2616
The "content-coding" is a "token", and the "token" is defined as the following:

token = 1*
separators = "(" | ")" | "<" | ">" | "@"
| "," | ";" | ":" | "" | <">
| "/" | "[" | "]" | "?" | "="
| "{" | "}" | SP | HT
// where:
CTL = <any US-ASCII control character
(octets 0 - 31) and DEL (127)>
SP = <US-ASCII SP, space (32)>
HT = <US-ASCII HT, horizontal-tab (9)>

[guohuideng2024]

I updated the PR with @unknown since there is no objects after asking for potential objects.
The CL to chromium and WPT tests is also updated accordingly.

[annevk]

The change looks good, but wrapping at 100 columns doesn't seem to be done properly. I could address that, but you're welcome to it as well.

Also, #1796 (review) still needs to be resolved to satisfy the IPR bot.

[guohuideng2024]

The change looks good, but wrapping at 100 columns doesn't seem to be done properly. I could address that, but you're welcome to it as well.

Also, #1796 (review) still needs to be resolved to satisfy the IPR bot.

[Edit] I fixed the 100 column width violation. :)

I joined MicrosoftWHATWGContributors organization. but I still fail the agreement check. I think the check failed because I submitted an agreement as an individual in the past. And it looks like I don't have permission to withdraw the individual agreement I submitted.

The error says:

@guohuideng2024 has signed up to participate as an individual, but has not yet been verified. 

And my name is still in the "individuals" list:
https://github.com/whatwg/participant-data/blob/main/individuals.json#L5565

Would you know how I can fix the agreement check failure? Thanks.

[annevk]

Great, this looks good. When do you expect the tests to be ready? We'll want to land those and this change around the same time.

[guohuideng2024]

Great, this looks good. When do you expect the tests to be ready? We'll want to land those and this change around the same time.

I am going to get the changes to the tests landed next week. I will update here as soon as it's done.

[guohuideng2024]

@annevk The tests are updated :)

web-platform-tests/wpt#52807

I greatly appreciate the patience and guidance I received here!