feat(fields): implement Param and CallableParam for handling unmapped parameters that can be referenced during build

[leverage-analytics]

This completes the development and testing related to #604.

• Implement new Param and CallableParam field types
• Update handling of **kwargs for base factory process_kwargs and process_kwargs_coverage. A new helper method was used to determine which fields in a factory are Unmapped types (Params or CallableParam)
• Implement tests for new fields as well as changes to factory build
• Update documentation to include overview of feature functionality with examples

[leverage-analytics]

@adhtruong, I have an update ready to update this pull request, but after merging with main, I am encountering issues realted to some typing related to SQLAlchemy factories. Please let me know how you want me to proceed (wait for patch, submit anyways, or something else).

Here's what I'm seeing:

check python ast.........................................................Passed
check for case conflicts.................................................Passed
check for merge conflicts................................................Passed
check toml...............................................................Passed
debug statements (python)................................................Passed
fix end of files.........................................................Passed
mixed line ending........................................................Passed
trim trailing whitespace.................................................Passed
ruff.....................................................................Passed
ruff-format..............................................................Passed
codespell................................................................Passed
prettier.................................................................Passed
mypy.....................................................................Failed
- hook id: mypy
- exit code: 1

tests/sqlalchemy_factory/test_sqlalchemy_factory_v2.py:95: error: Call to untyped function "JSON" in typed context  [no-untyped-call]
tests/sqlalchemy_factory/test_sqlalchemy_factory_v2.py:98: error: Call to untyped function "JSONB" in typed context  [no-untyped-call]
Found 2 errors in 1 file (checked 139 source files)

pyright..................................................................Passed
Sphinx Lint..............................................................Passed

[leverage-analytics]

@adhtruong Interestingly, when I ran pre-commit separately, it raised the mypy errors, but when it ran as part of the git hook, it passed. Not sure what is happening there.

Anyways, here is the updated feature we have been discussing. Thank you for your patience as I reworked it. From our conversation, I did a little thinking, and I decided to alter the API just slightly from what we discussed. Instead of including a default_factory parameter for the constructor of Param, I added a bool flag is_callable. The reason I decided to this is to provide for more flexibility in the use of the Param type.

I needed a way to understand if the user's intention is to call the passed value (for example passing a lambda that they might not want to be called) but also override that value at build time and still have the same action (call it or not). I didn't want the user to have to pass a temporary placeholder value for the param at factory definition time (just to minimize the amount of code needed to use the feature) so we could understand their intention related to calling or not calling the passed value. So I figured asking with the boolean option would be a simpler interface for the user.

Thanks again for your review and feedback!

[adhtruong]

What is the use case of requiring the kwargs from the factory field itself passed to a callable declared at build time? This feels like quite a specific use case that inclined to push to user itself rather as feels niche in the current form. Is this with the intention to have params passed to other params or could the use case describing be supported like that so if more similar to existing code base

This is compared Param itself or callable with args which feels more general with logic pushed to use

[leverage-analytics]

@adhtruong I'm sorry if I didn't explain myself accurately, but the kwargs that get passed to the callable can only be passed during the declaration of the factory, not at build time. I thought there was too much potential for collision between kwarg names and factory field names to attempt to handle passing kwargs at build time.

Essentially, there are two routes:

• declare a param at factory declaration with is_callable set to True: This will require either a callable be passed at factory declaration or at build time (or both, giving priority to whatever is passed at build time). In this case any kwargs passed to the Param constructor in the factory class get passed to the callable at build time.
• declare a param at factory declaration with is_callable set to False. This will require any value to be passed either at factory declaration or at build time (or both, giving priority to whatever is passed at build time). In this case, any kwargs passed in the Param constructor are invalid, and an exception is raised.

I thought it made sense, from the viewpoint of API consistency, that if we allow overriding Param with a non-callable at build time, we should also allow overriding a callable Param at build time, too.

Also, like I mentioned previously, the reason for using an is_callable switch instead of passing a default_factory argument is in the case where the user doesn't want or need to pass a default value for the Param at factory declaration. It's extra work to put in a placeholder entry that will never be used.

[leverage-analytics]

@adhtruong @guacs let me know if I can clarify or respond on this any further or how you want to proceed. Thanks!

[adhtruong]

Hey @leverage-analytics , sorry the late reply on this!

I think the callable use case may be a little confusing as doesn't link the params of kwargs closely to the calllable passed in a build time. I think just having the whole value overriding is fine and keeps overall design simpler. This matches the existing interface of other fields closer e.g. if Use is defined on the factory when a value is passed in the args and kwargs are ignored.

If there is a strong use case for this kind of overriding, then have the original form of separate field types for callable and not makes this difference more clear so usage if consistent at field level.

[leverage-analytics]

@adhtruong thanks for the feedback! As I see it, our options are:

1. drop support for passing kwargs to callable Params (so the user doesn't confuse the two uses of kwargs). I think this makes the most sense. I personally don't have a use for kwargs to be passed to the callable; I was just trying to accommodate use cases I hadn't thought of.
2. revert to the original pull request with separate Param and CallableParam types.
3. drop the functionality related to the calling a callable to get the Param value at build time. I'm not a fan of this, as this would mean, at least for my use case, a lot more repeated code in some cases; users who want to execute a callable for the Param value at build time would be forced to pass it as a build kwarg every time.

Like I said, I'd prefer the first, but I'm open to supporting the project how the maintainers see best.

Thanks!

[adhtruong]

Just to clarify 1. Would this would looks like

class Param:
     def __init__(self, default: T, default_factory: Callable[[], T)-> None: ...

class Factory(DataclassFactory[MyType]):
      a = Param(default_factory=lambda: 1)

assert Factory.build(a=2).a == 2. # would this error?
assert Factory.build(a=lambda: 2) == lambda: 2 # would this get evaluated?

For 3 use case, would this be covered via Factory.create_factory(a=Param(default_factory=lambda : 2)? Or do you mean callable on Param itself ?

[leverage-analytics]

@adhtruong answered based on my proposal

class Factory(DataclassFactory[MyType]):
      a = Param(default_factory=lambda: 1)

# This example results in an attribute error because no parameter values get mapped 
# into the objects produced by the factory (assuming MyType doesn't have attribute 'a').
assert Factory.build(a=2).a == 2. 

# In the following case, the expression lambda: 2 would be evaluated during build 
# because the user specific a default_factory for the`Param`. 
assert Factory.build(a=lambda: 2)

[github-actions]

Documentation preview will be available shortly at https://litestar-org.github.io/polyfactory-docs-preview/650

[adhtruong]

Thanks again for clarifying. Appreciate the patience on back and forth and explaining to help me understand!

I think 3 makes is my preference as most in line with existing fields and result can be achieved by user with

1. Call a callable before passing in
2. Or override if need this repeating as a factory which can avoid repetition.

I think can make a minor tweak (commented in line) to make this work for coverage form too