Docs debt

[mrlubos]

• People want to know how to use SSE ldelalande+alexhancock/shared-config-client block/goose#1219 (comment)
• ~resolvers feat: add resolvers keyword #2975
• sdk.params feat: add sdk.params option #1927
• sdk.instance

[Pitouli]

Hello, I do not know if I am using the functionality wrong (and in this case it highlight the importance of a doc 😄 ) or if there is a bug, but I tried the following that was feeling the most natural for me (and compatible with the typescript):

        // Call API with FormData (handled automatically by the SDK)
        const result = await ingestPermanentDocumentBatch({
          path: {
            domain_id: domainId,
          },
          body: {
            excel_file: excelFile,
            zip_file: zipFile || null,
          },
          onSseEvent(event) {
            console.log("SSE Event:", event)
          },
          onSseError(error) {
            console.error("SSE Error:", error)
          },
        })

But in this situation I have no request made at all (and no logs)

So I then tried that, following more closely the changelog:

        // Call API with FormData (handled automatically by the SDK)
        const { stream } = await ingestPermanentDocumentBatch({
          path: {
            domain_id: domainId,
          },
          body: {
            excel_file: excelFile,
            zip_file: zipFile || null,
          },
          onSseEvent(event) {
            console.log("SSE Event:", event)
          },
          onSseError(error) {
            console.error("SSE Error:", error)
          },
        })

        for await (const event of stream) {
          console.log("Stream Event:", event)
        }

But I receive the following error from my server:

{
  "detail": [
    {
      "type": "missing",
      "loc": [
        "body",
        "excel_file"
      ],
      "msg": "Field required",
      "input": null,
      "url": "https://errors.pydantic.dev/2.11/v/missing"
    }
  ]
}

And when I look at the request in Chrome console, I have this as "Request payload": [object Object]
Seems like there is a toString() that have been applied somewhere.

Note that if in the openapi I indicate a "JSON" return instead of a stream, then heyapi generates a "normal" function (not SSE), and in that situation the following code works correctly: the request is sent, with the correct payload, and in the chrome console, I see all the stream events (when looking at the request in the network tab, there is an addition tab with the events stream). But of course, they are completely ignored.

        // Call API with FormData (handled automatically by the SDK)
        const result = await ingestPermanentDocumentBatch({
          path: {
            domain_id: domainId,
          },
          body: {
            excel_file: excelFile,
            zip_file: zipFile || null,
          }
        })

Am I using the function in a wrong way? The onSseEvent() callback is unsupported?

For information, both the excelFile and the zipFile are Files (therefore sent as binaries)

[mrlubos]

@Pitouli can you share your spec or ideally a reproducible example? I think I know what might be the issue but would be good to confirm

[Pitouli]

Hello, I can share extract of the spec :)

For a reproducible example, it's a bit complex: I am not the one doing the backend API, so extracting this part and only this part is not something I can do in a simple way :(

But, indeed, I cannot ensure by 100% that the back is perfectly defined as it should. The only thing I know for sure is that when I call it with the "standard heyapi" (and not the SSE version), I see stream events in Chrome Inspector.

/api/v1/documents/{domain_id}/ingest_permanent_document_batch:
    post:
      tags:
      - documents
      summary: Ingest permanent documents and link them to assistants.
      description: Create links between assistants and documents from the given Excel
        file. Uses Server-Sent Events (SSE) to stream progress updates.
      operationId: ingestPermanentDocumentBatch
      parameters:
      - name: domain_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          description: Domain ID
          title: Domain Id
        description: Domain ID
      requestBody:
        required: true
        content:
          multipart/form-data:
            schema:
              $ref: '#/components/schemas/Body_ingestPermanentDocumentBatch'
      responses:
        '200':
          description: SSE stream with progress updates
          content:
            text/event-stream: {}
        '422':
          description: Validation Error
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/HTTPValidationError'

The schemas:

components:
  schemas:
    Body_ingestPermanentDocumentBatch:
      properties:
        excel_file:
          type: string
          format: binary
          title: Excel File
          description: Excel file containing document-assistant links.
        zip_file:
          anyOf:
          - type: string
            format: binary
          - type: 'null'
          title: Zip File
          description: Optional ZIP file containing documents to ingest.
      type: object
      required:
      - excel_file
      title: Body_ingestPermanentDocumentBatch
    HTTPValidationError:
      properties:
        detail:
          items:
            $ref: '#/components/schemas/ValidationError'
          type: array
          title: Detail
      type: object
      title: HTTPValidationError
    ValidationError:
      properties:
        loc:
          items:
            anyOf:
            - type: string
            - type: integer
          type: array
          title: Location
        msg:
          type: string
          title: Message
        type:
          type: string
          title: Error Type
      type: object
      required:
      - loc
      - msg
      - type
      title: ValidationError

[Pitouli]

When using the "standard" library (either by declaring a application/json as returned type before generating sdk, or by manually switching return (options.client ?? _heyApiClient).sse.post<IngestPermanentDocumentBatchResponses to return (options.client ?? _heyApiClient).post<IngestPermanentDocumentBatchResponses in the sdk.gen.ts file), I correctly have a request sent, with stream events.

But it does not trigger the onSseEvent or onSseError callbacks, and it crashes when trying to execute for await (const event of stream) { console.log("Stream Event:", event) } (since stream is undefined)

[Pitouli]

I have perhaps found something.

I added console.logs in the formDataSerializer function of the sdk, and in the CreateSseClient of the serverSentEvent.gen.ts just before it does the fetch.

And I see this in my console:

So it seems the body is at one point serialized as a formData, but this formData is not what it transmitted to the SseClient since we see it is back to an object with a File property.

And from what I read, fetch can handle body as a File, but not if the file is inside an object.

--

On another topic, are you able to split my messages as a new topic? It seems that it is not only a question of documentation in the end, and that I am polluting the initial topic. Sorry for that :(

[Pitouli]

Edit: I am dumb... The issue I am speaking about is already corrected in a newer version than the 0.81.1 I was using... And my package updates were not doing anything because for version <1.0.0, 0.81.1 will only match 0.81.* and not 0.86.11

---

Ok, I think I found a fix! :)

In client.gen.ts we should have opts.serializedBody instead of opts.body in the makeSseFn() function

With this correction, the body is correctly sent, and I have both the console.log("SSE Event:", event) and the console.log("Stream Event:", event) that print :)

event in case of onSseEvent() callback contains { data: DataFromTheServer, event: "name-of-the-event" as string, id: undefined, retry: 3000 }, while event in the case of for await (const event of stream) only contains DataFromTheServer

I wanted to make a pull request, but when I search for const makeSseFn = there are many files that are returned ^^' So I do not know if I am supposed to modify them all, or if they are generated from a master file... So I will let you make the modification (and anyway, since I do not have tested it properly, it would definitely not have been a PR you could have approved directly :p )

https://github.com/search?q=repo%3Ahey-api%2Fopenapi-ts%20const%20makeSseFn%20%3D&type=code

Last but not least: if you confirm a fix is required and that this fix is the right one, how much time do you think is needed before publishing a new version with this fix? The underlying question for me being: should I wait for the fix or should I implement a workaround in the meantime.

Thank you!

Edit: something weird: without the for await (const event of stream) loop, the request is not sent to the server and therefore the server does not send events, so the onSseEvent() are not called. Therefore, I do not understand why those functions exist. For me, I would have expected the following code to work:

        await ingestPermanentDocumentBatch({
          path: {
            domain_id: domainId,
          },
          body: {
            excel_file: excelFile,
            zip_file: zipFile || null,
          },
          onSseEvent(event) {
            console.log("SSE Event:", event)
          },
          onSseError(error) {
            console.error("SSE Error:", error)
          },
        })

       console.log("Server has ended then connection")

[jeremyfiel]

consider defining encoding for the form-data fields and redefining a single form-data files field to take a list of files of defined content types.

openapi: 3.0.4
info:
  title: test
  version: v1
paths:
  /api/v1/documents/{domain_id}/ingest_permanent_document_batch:
    post:
      tags:
      - documents
      summary: Ingest permanent documents and link them to assistants.
      description: Create links between assistants and documents from the given Excel
        file. Uses Server-Sent Events (SSE) to stream progress updates.
      operationId: ingestPermanentDocumentBatch
      parameters:
      - name: domain_id
        in: path
        required: true
        schema:
          type: string
          format: uuid
          description: Domain ID
          title: Domain Id
        description: Domain ID
      requestBody:
        required: true
        content:
          multipart/form-data:
            schema:
              $ref: '#/components/schemas/Body_ingestPermanentDocumentBatch'
            encoding:
              files:
                headers:
                  Content-Disposition:
                    required: true
                    schema:
                      type: string
                    examples:
                      header_value:
                        value: 'content-disposition: form-data; name="files"'
                contentType: 'application/zip,application/vnd.ms-excel,application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
      responses:
        '200':
          description: SSE stream with progress updates
          content:
            text/event-stream: {}
components:
  schemas:
    Body_ingestPermanentDocumentBatch:
      properties:
        files:
          type: array
          items:
            type: string
            format: binary
        # excel_file:
        #   type: string
        #   format: binary
        #   title: Excel File
        #   description: Excel file containing document-assistant links.
        # zip_file:
        #   anyOf:
        #   - type: string
        #     format: binary
        #   - type: 'null'
        #   title: Zip File
        #   description: Optional ZIP file containing documents to ingest.
      type: object
      required:
    #  - excel_file
       - files
      title: Body_ingestPermanentDocumentBatch