Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore Content-Length from env.request_headers #52

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

pilaf
Copy link

@pilaf pilaf commented Feb 24, 2025

Closes #51

This makes the adapter ignore Content-Length if given in Faraday's env.request_headers, but only if capitalized, since Async::HTTP will set content-length anyway.

I'd like to add tests too, but I wasn't sure how to test what headers end up in the final request without setting up a dummy server to receive them. Any suggestions?

@@ -181,6 +181,12 @@ def perform_request(env)
end

if headers = env.request_headers
# Ignore Content-Length if given, it will be set for us later anyway (lowercased)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good.

I'd suggest extracting this into length and then assigning to BodyReadWrapper, which should have this:

class BodyReadWrapper
  def initialize(body, length = nil, block_size: 4096)
    @body = body
    @length = length
    @block_size = block_size
  end

  # ...

  def length
    @length # if known
  end

You'll need to move the if headers = env.request_headers first to extract a local variable if possible.

Is headers.key?(...) case sensitive?

Testing is a bit tricky, but maybe we can mock the client. If you can get the code updated, I can help write the tests.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ioquatix

Is headers.key?(...) case sensitive?

Yes. I'd change this to check for both Content-Length and content-length if the intent is to use that value for BodyReadWrapper#length, if you agree.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be easier, for a first pass, to construct headers = ::Protocol::HTTP::Headers[headers] first, and then length = headers.delete("content-length").

Comment on lines 193 to 194
body = ::Protocol::HTTP::Body::Buffered.wrap(body).then do |buffered|
::Protocol::HTTP::Body::Buffered.new(buffered.chunks, content_length)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This double initialization is ugly but I had no choice since Protocol::HTTP::Body::Buffered.wrap doesn't take a length argument.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand how you are thinking about the problem.

A buffered body can compute its size accurately and efficiently from the chunks. That's different from a readable body where we can only call #read to stream the data.

Therefore, I wouldn't worry about this code path - just leave it as the original implementation.

Comment on lines 188 to 189
if body.is_a?(::Protocol::HTTP::Body::Readable)
# Good to go
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this case try to rebuild the body with the given length too?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I wouldn’t worry about this case either, it’s expected the body would know its size already if appropriate.

The main issue we are addressing is Faradays opaque “body#read” which may have length but the only way to know that is via the supplied header.

@pilaf
Copy link
Author

pilaf commented Mar 9, 2025

@ioquatix Sorry for the delay, please check the latest commit, I think it mostly addresses your comments.

Again I couldn't write tests for it, but I did test that this code works separately.

@ioquatix
Copy link
Member

ioquatix commented Mar 9, 2025

Thanks, this looks like what I was thinking of, I'll try to add some tests, and if it looks good we should be able to move forward.

@ioquatix
Copy link
Member

This is perfect, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

POST requests with no body include content-length header twice
2 participants