Skip to content

[data] new Record encoding#1472

Merged
fwbrasil merged 9 commits intomainfrom
record2-replace
Feb 24, 2026
Merged

[data] new Record encoding#1472
fwbrasil merged 9 commits intomainfrom
record2-replace

Conversation

@fwbrasil
Copy link
Collaborator

@fwbrasil fwbrasil commented Feb 23, 2026

Record has a major drawback: fields can be duplicated in the same record with different types. Ideally "myField" ~ Int & "myField" ~ String would be equivalent at the type level to "myField" ~ (Int | String) but, at the time when I introduced Record, I couldn't find a type-level solution for it so I decided to just support the duplicate fields and save a Tag to disambiguate at runtime.

I've been working on the new kyo-http module and decided to use Record instead of named tuples but the limitation isn't acceptable. There's the additional memory footprint to save the tags and allowing duplicate fields introduces some rough edge cases.

In this new encoding, I'm using a type-level workaround: keep Record invariant but provide subtype relationship via a Conversion. It isn't ideal but works well and produces the expected reduction at the type level. I'm also using Dict to reduce allocations further. Besides the memory footprint reduction and proper field tracking behavior, I simplified most of the APIs and implementations.

* @tparam A
* The field intersection type (e.g., `"name" ~ String & "age" ~ Int`) or a case class type
*/
sealed abstract class Fields[A] extends Serializable:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the replacement for TypeIntersection

type Zipped[T2 <: Tuple] = Fields.ZipValues[AsTuple, T2]

/** Runtime `Field` descriptors (name, tag, nested), lazily materialized. */
lazy val fields: List[Field[?, ?]]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only materializes fields if necessary

/** An opaque map from field name to a type class instance `F[Any]`, summoned inline for each field's value type. Used by operations
* like `Render` that need a type class instance per field.
*/
opaque type SummonAll[A, F[_]] = Map[String, F[Any]]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simpler replacement for Inliner

Copy link
Contributor

@road21 road21 Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I guess Inliner was slightly more powerful, but this api (without having to extend class) is much more user friendly

private inline def summonLoop[T <: Tuple, F[_]]: Map[String, F[Any]] =
inline erasedValue[T] match
case _: EmptyTuple => Map.empty
case _: ((n1 ~ v1) *: (n2 ~ v2) *: (n3 ~ v3) *: (n4 ~ v4) *: rest) =>
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all inline loops are unrolled to avoid stack reaching the inlining limit

* Intersection of `Name ~ Value` field types describing the record's schema
*/
final class Record[+Fields] private (val toMap: Map[Field[?, ?], Any]) extends AnyVal with Dynamic:
final class Record[F](private[kyo] val dict: Dict[String, Any]) extends Dynamic:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this should be opaque type Record[F] = Dict[String, Any] but I couldn't find a way to make it work with Dynamic. I've removed AnyVal since it can actually introduce allocations when used in generic contexts, which is the case for anon functions in Scala 3 due to the lack of specialization.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* instances per field. Call the returned `StageOps` directly to stage without a type class, or chain `.using[TC]` to require a type
* class instance for each field's value type.
*/
inline def stage[A](using f: Fields[A]): StageOps[A, f.AsTuple] = new StageOps(())
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Staging is simplified to regular methods instead of having to extend a class

@fwbrasil
Copy link
Collaborator Author

@road21 you might be interested in this! :)

end hashCode

/** Returns a human-readable string representation in the form `"name ~ Alice & age ~ 30"`. */
def show: String =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: should we call this render?

Also, if dict.mkString accepted a separator for k/v this wouldn't need to be duplicated.

Copy link
Collaborator Author

@fwbrasil fwbrasil Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, we provide a Render evidence, so asText is already available. I'll remove this method

dict.is(other.dict)
case _ => false

override def hashCode(): Int =
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary due to Span? Perhaps we should have Span.hashcode anyways (that doesn't just return obj reference)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. We can't implement hashCode for Span since it's an opaque type pointing to Array

@fwbrasil fwbrasil merged commit 0f2777b into main Feb 24, 2026
5 checks passed
@fwbrasil fwbrasil deleted the record2-replace branch February 24, 2026 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants