Skip to content

feat: add JSON type support - #69

Merged
paurkedal merged 8 commits into
ocaml-community:masterfrom
jchavarri:support-json
Aug 12, 2025
Merged

feat: add JSON type support#69
paurkedal merged 8 commits into
ocaml-community:masterfrom
jchavarri:support-json

Conversation

@jchavarri

Copy link
Copy Markdown
Contributor
  • Add MYSQL_TYPE_JSON constant to FFI bindings
  • Add Json variant to buffer_type and value types
  • Implement json binding and field accessor functions
  • Add comprehensive JSON tests for various data types
  • Support both native JSON and string compatibility modes
  • Tested against TiDB and MySQL 8.0.23

- Add MYSQL_TYPE_JSON constant to FFI bindings
- Add Json variant to buffer_type and value types
- Implement json binding and field accessor functions
- Add comprehensive JSON tests for various data types
- Support both native JSON and string compatibility modes
- Tested against TiDB and MySQL 8.0.23
@jchavarri
jchavarri requested a review from paurkedal as a code owner July 8, 2025 08:56

@paurkedal paurkedal left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it's good to cover the JSON SQL type, though I don't think it needs to be exposed in the interface, see my comment below.

Comment thread lib/field.ml Outdated
| `String of string
| `Bytes of bytes
| `Time of Time.t
| `Json of string

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it not be better to use the `String case for JSON, as well? I didn't write the original code, but the intention seem to me that the value type only contains the cases which have different representation on the OCaml side, e.g. omitting the various integer variants.

Comment thread lib/field.ml Outdated
Comment on lines +129 to +133
let json field =
match value field with
| `Json j -> j
| _ -> err field ~info:"a json value"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the test case you take into account the possibility that the database server returns a string (or bytes, as I've when testing). That would be needed here, as well. But if you agree to the above, this definition can be omitted.

Comment thread lib/field.ml Outdated
Comment on lines +164 to +168
let json_opt field =
match value field with
| `Json j -> Some j
| `Null -> None
| _ -> err field ~info:"a nullable json value"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Comment on lines +395 to +396
| `Json j -> j
| `String s -> s (* TiDB/MySQL might return as string *)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also seeing `Bytes here, using MariaDB client/server version 10.11.13.

@jchavarri

Copy link
Copy Markdown
Contributor Author

@paurkedal Thanks for the review, I think all the comments were tackled. Please let me know if anything else is missing.

@paurkedal

Copy link
Copy Markdown
Collaborator

I added a `Bytes case to Field.sting and Field.string_opt, since as we've seen returned when selecting the data JSON column, but on second though I'm not completely happy about this solution. First of all, it breaks the currently strict interpretation of Field.string by allowing it to also convert from blob types, even if we only intended to add JSON. Wondering why we're receiving bytes instead of a string, I found the answe in MariaDB's explanation of the JSON type, which they implement as an alias for LONGTEXT COLLATE utf8mb4_bin. Further, LONGTEXT is classified together with blobs, so I think `Bytes is the correct type to expect for MariaDB. I'm therefore considering whether we should return JSON as bytes. And for consistency with the existing code, maybe restore `` Json as a `Bind.buffer_type` and adding it `to_blob`.

I need to think about it a bit more. Let me know if you have any thoughts.

This also reintroduce the `JSON case internally, making the code
consistent with other cases.
@paurkedal

Copy link
Copy Markdown
Collaborator

Can you take a look at my latest commit to see if you agree with my implementation? This follows the current pattern for other SQL types, but it means we receive the JSON as bytes instead of string. I think the API documentation could use a table showing the correspondence between SQL and OCaml types, but I can do that after merging.

Shall we preserving the current commit history when I merge?

@paurkedal

Copy link
Copy Markdown
Collaborator

One more thing, can you re-test against TiDB and MySQL 8.0.23?

@jongleb

jongleb commented Aug 11, 2025

Copy link
Copy Markdown
Contributor

You're right about this. After checking the MariaDB docs, JSON is just an alias for LONGTEXT COLLATE utf8mb4_bin, so using bytes makes sense.

Also retested for TiDB and MySQL 8.0.23, together with your last commit, everything works

P.S
I answered instead of the author (he knows and he told me that I will finish it)

@jchavarri

Copy link
Copy Markdown
Contributor Author

Shall we preserving the current commit history when I merge?

I don't think we have a strong opinion. Whichever way seems better to you.

@paurkedal
paurkedal merged commit 1993263 into ocaml-community:master Aug 12, 2025
6 checks passed
@paurkedal

Copy link
Copy Markdown
Collaborator

Great, thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants