conflicting validated input values when piping data through middleware validators with differing schemas

[EamonHeffernan]

Which project does this relate to?

Start

Describe the bug

The output of zod parsing removes any extra keys from the data input and means that any subsequent validators cannot successfully validate the body. This can be worked around by putting .passthrough() at the end of the zod schema. The types for calling the serverFn imply that this is not the case as it has the inputs merged for the data field.

Your Example Website or App

https://codesandbox.io/p/devbox/sleepy-rubin-9qd7w2

Steps to Reproduce the Bug or Issue

Run pnpm dev
Navigate to the home page.

Expected behavior

As a user I would expect that I am able to validate input on the middleware, but then also be able to validate additional inputs on the serverFn, for example if I need to validate that a user can access an organisation, i would put the organisation id in the middleware, validate that they can access it and put the organisation into the context, then validate the serverFn specific inputs in the serverFn. This assumption is backed up by how the types get merged by the middleware and the serverFn.

Screenshots or Videos

No response

Platform

• OS: Linux
• Browser: Firefox
• Version: 1.114.22

Additional context

No response

[SeanCassiere]

This has to do with how each library handles the result of the parse method for data that is not defined in the original schema.

Since TanStack Start (and Router) interface with these validation libraries using Standard Schema, we're only accessing the equivalent parse method set by each validation library (such as zod, valibot, arktype, etc).

Diving deeper into these libraries, this is what I found out about them when piping in data that was in the original schema alongside valid input.

• arktype - Returns the "extra" fields in the same object as the validated data.
• zod - Only returns the fields defined in the schema. Optionally, the schema.passthrough().parse() modifier can be used to return "extra" fields in the same object as the validated data.
• valibot - Only returns the fields defined in the schema. This library does not have an equivalent to passthrough() (like zod does).

This diverging behaviour across the libraries presents a problem as TanStack Start (and Router) do not interface with these libraries directly, rather using the Standard Schema interface to access the parse method and inferring its types.

We should add a callout in the documentation about the user understanding how their validation library of choice behaves with his type of parsing data through multiple schemas.

At the moment, I'm not too sure if there are actual fixes we can apply that would fix the behaviour of these upstream libraries.

[SeanCassiere]

As mentioned above, this makes it a bit tricky to come up with a solution to this. These are the approaches that I've come up with so far,

1 - Perform a shallow merge on the input if its parsed return type is an object in (createServerFn/createMiddleware).validator

We could perform a shallow merge of the input if it is an 'object' at the end of each schema pass in both createServerFn and createMiddleware. Something equivalent to this.

(values) => {
  const parsed = schema['~parse'](values)
  if (isObject(parsed)) {
    const obj = isObject(values) ? {...values} : {}
    obj = {...obj,...parsed}
    return obj
  }
  return parsed
}

This would be the quickest solution, however, it does feel like a hack since this would only be shallow for the top-level keys of the object.

Furthermore, this can have unintended consequences for people intending on this validation library to behave as expected, since folks relying on zod and valibot to strip out unwanted keys, could end up with this extra data being potentially

Also, this would only be restricted if it was an object. This brings up the question as to what happens when it is an Array? or FormData? or etc.?

2 - Perform deep recursive merges on the input if its parsed return type is an object in (createServerFn/createMiddleware).validator

Similar to the above approach, we'd be checking for the type of 'object' and then recursively performing a merge into all the child keys.

This has the same downside as the first option with the addition of being computationally more expensive since it would need to traverse the entire tree.

3 - Add helpers to the individual helper packages we have.

We could abstract the logic used for either 1 or 2 and move them into the helper packages that are currently available (@tanstack/zod-adapter, etc.).

This would essentially move the logic out of createServerFn and createMiddleware, making it more "opt-in", so those that expect the native behaviours of their validation library aren't affected.

This still has the downside of being computationally expensive should a recursive deep merge be used and still poses the question as to what happens when the data type is not an object.

4 - Inversion of control to user and add helper recipes

We could include helper snippets in the documentation on implementing helpers based on 1 and 2, that'd live in the user's code instead of being shipped by TanStack Start (and/or Router).

We'd essentially be using the documentation to show the user a snippet, similar to what was used in 1 with comments for the user to extend it to as they see fit.