Http 2.0 streaming

[renatillas]

After my last issue we realized gleam_hackney already suports http 2.0 out of the box, great and. thank you Louis! But one caveat I found is that non-finite requests responses, useful for streaming gRPC, are not supported. What do you think of adding support for opening streams, consuming, and closing them?

[lpil]

Ah now I understand, sorry about that.

Is it that you need HTTP streaming (not present yet) and HTTP2 (present already)? Or is there something specific to HTTP2 that needs to be exposed?

[renatillas]

Nah don't worry, it's my bad I forgot to mention streaming. I need stream HTTP yes. That's what's needed for the client I'm building. I've tried the PR attached to this issue locally and it works for the different config options.

[lpil]

Do you need that specific HTTP2 API of Hackney? How come you didn't use the regular streaming API that is HTTP version agnostic?

[renatillas]

Oh I didn't read the hackney docs properly. I'm still learning how to read erlang docs coming from gleam. After trying the regular streaming API I can assure it works for my use case! I would have some questions regarding the implementation:

• Should a type with the shape:

    pub type StreamMessage {
    StreamData(data: BitArray)
    StreamDone
  }

  really be defined in gleam_hackney or would it make sense to put it in the gleam_http package? For me it makes sense to have it in gleam_http, but we would be the first here consuming it and it would require more design thought as maybe other implementations need a

  pub type StreamMessage {
    StreamResponse(status: Int, headers: List(http.Header))
    StreamData(data: BitArray)
    StreamTrailers(headers: List(http.Header))
    StreamDone
  }

  shape. Currently in gleam_hackney after hacking a bit, we would only need the first snippet. But with the old http2 interface I faced myself using the second one.
• For the http2 requests I need to pass some options to the hackney side: I've seen there's a comment in another PR regarding this, Allow setting recv_timeout when sending request #4 (comment), where you suggested (long time ago, maybe your view changed) adding a builder pattern for sending. I think that would be a great fit for the lib. I could open first a PR regarding that, different from the streaming API PR.

[lpil]

OTP streaming data types would need to be in this package as gleam_http is target specific and doesn't concern itself with specific implementations of HTTP at all.

Where did that stream-message design come from? I can't find a data type like that in the Hackney streaming API documentation.

[renatillas]

OTP streaming data types would need to be in this package as gleam_http is target specific and doesn't concern itself with specific implementations of HTTP at all.

Makes sense! Thanks for the clarification.

Where did that stream-message design come from? I can't find a data type like that in the Hackney streaming API documentation.

I replicated this type https://hackney.hexdocs.pm/hackney_h2_stream.html#t:h2_msg/0 that has this shape:

• -type h2_msg() :: {response, non_neg_integer(), list()} | {data, binary()} | {trailers, list()} | done.

This type is returned by this function https://hackney.hexdocs.pm/hackney.html#h2_recv/2.

But using h2_recv would not be needed anymore, that's why I would resort to the simpler

  pub type StreamMessage {
    StreamData(data: BitArray)
    StreamDone
  }

as stream_body/1 returns ok, binary()} | done | {error, term()}

[renatillas]

#16 (comment)

We would use the same API as gleam_httpc for any configuration.

Makes total sense. Would you accept a PR with that feature only or should I put it with the stream PR as well? I'm inclined to doing two PRs but you are smarter than me (no sarcasm here!) so please guide me.

The options I'm looking forward to add due to my use case are:

pub opaque type Configuration {
  Builder(
    // Time in milliseconds for opening a http stream
    connect_timeout: Int,
    // Time for response data to be received.
    receive_timeout: ReceiveTimeout,
    // Wheter to verify the TLS certificate of the server.
    verify_tls: VerifyTls,
    // Wheter to use connection pooling or not
    connection_pooling: Bool,
  )
}

type ReceiveTimeout {
  ReceiveTimeout(Int)
  Infinity
}

pub opaque type VerifyTls {
  TlsVerifyPeer
  TlsVerifyNone
  TlsCaCertificate(file: String)
}

What does setting up the HTTP protocol mean?

I'm facing that not passing [{protocols, http2}] as options to the request to my kurrentdb server returns a Error(Other(TlsAlert(UnknownCa([<bunch of numbers>])))). I found that you can force the protocol on the request here: https://hackney.hexdocs.pm/http2_guide.html#force-http-2-only

Ok I don't know how but after refactoring it works without forcing the http2 protocol. Sorry about that.

[lpil]

One feature per PR please 🙏

I replicated this type https://hackney.hexdocs.pm/hackney_h2_stream.html#t:h2_msg/0 that has this shape:

What about the trailers variant? Where does that come from?

[renatillas]

Perfect, I'll create the PRs once we have this settled down

What about the trailers variant? Where does that come from?

The type h2_msg()

-type h2_msg() :: {response, non_neg_integer(), list()} | {data, binary()} | {trailers, list()} | done.

has the trailers variant, but stream_body/1 returns {ok, binary()} | done | {error, term()} doesn't contain it, so we would only need a Done and Data variants for the streaming, wrapped in a Result to convey the error