Introspection: Introduce TypeHint struct

[Tpt]

inspect::TypeHint is composed of an "annotation" string and a list of "imports" ("from X import Y" kind)

The type is expected to be built using the macros type_hint!(module, name), type_hint_union!(*args) and type_hint_subscript(main, *args) that take care of maintaining the import list

Introspection data generation is done using the hidden type_hint_json macro to avoid that the proc macros generate too much code

Sadly, outside type_hint these macros can't be converted into const functions because they need to do some concatenation. I introduced type_hint! for consistency, happy to convert it to a const function.

Miscellaneous changes:

• Rename PyType{Info,Check}::TYPE_INFO into TYPE_HINT
• Drop redundant PyClassImpl::TYPE_NAME

[Tpt]

I would have preferred it to be an enum but sadly I don't see how to serialize it: const functions cannot concatenate and recursive macros are not possible

[davidhewitt]

What would the enum definition be? I can see if I can come up with any crazy ideas if I know the preferred form.

[Tpt]

Something like:

enum TypeHint {clashes
   BuiltIn { name: &'static str },
   Member { module: &'static str, name: &'static str },
   Union(&[TypeHint]),
   Subscript { value: TypeHint, parameters: &[TypeHint] },
   Callable { arguments: &[TypeHint], return: TypeHint } // Special case because of the nested [] syntax
}

This way it's easy to ensure the syntax is valid and to do manipulation like introducing aliases in case of name conflicts.

[davidhewitt]

Hmm, I think if we split the const fn up into one which (recursively) calculates the final length, and a second which serializes to it with &mut [u8] similar to what I did in the concat.rs module, I think we could probably make the const fn work?

Unfortunately this would require MSRV 1.83. But this is currently an experimental feature, I would personally be ok with having it limited to MSRV 1.83.

[Tpt]

Might be better as a const function. WDYT?

[davidhewitt]

I wonder, is there a way to merge all the macros into one? Maybe also with implicit syntax

type_hint!(a.A);  // basic type hint
type_hint!(a.A | b.B);  // union type hint
type_hint!(a.A[b.B]);  // parameterised type hint

we could infer all the imports using the dotted path rules, maybe in complex cases we require users to write the imports?

type_hint!(
   from a import A;
   from b import B;
   A.NestedClass | B
);

or maybe can use mixed syntax somehow

type_hint!(type_hint("a", "A.NestedClass") | b.B);

the "implicit syntax" seems nicest for the 99% of cases which users will write, so while I'm not sure exactly how to tie it all together it seems desirable to me (I think?) to have an intuitive syntax for the common case and then an escape hatch for the awkward ones.

[Tpt]

This looks indeed much nicer. I think we need a syntax for composition like in #[derive(FromPyObject)]: we want their to build type hints that is stuff like the union of type hints. So we want to write things like type_hint!(tuple[A::INPUT_TYPE, B::INPUT_TYPE]). But my guess it's mostly for auto-generated code.

What about having two macros:

• For static type hints like type_hint!(a.A[b.B]); with hopefully support for the from a import A syntax.
• For composition compose_type_hint!() where elements are all supposed to be other type hints like compose_type_hint!(PyTuple::INPUT_TYPE [ A::INPUT_TYPE, B::INPUT_TYPE ]). But I am not sure it's more readable than the current state.

WDYT?

It might also take into account in this design the type stubs in signatures like signature = (foo: collections.abs.Sequence[int]) -> list[int]. My guess is that we can allow there the same syntax than type_hint! and allow to set the from a import A in #[pymodule]. But it makes name conflict detection more fun...

[davidhewitt]

Thanks, this looks like the right direction for making the type hint definitions reliable wrt import resolution.

I think we can iterate a bit to find the best design for users, I wrote down some ideas.

[davidhewitt]

Sentence looks unfinished.

[davidhewitt]

What would the enum definition be? I can see if I can come up with any crazy ideas if I know the preferred form.

[davidhewitt]

I wonder, is there a way to merge all the macros into one? Maybe also with implicit syntax

type_hint!(a.A);  // basic type hint
type_hint!(a.A | b.B);  // union type hint
type_hint!(a.A[b.B]);  // parameterised type hint

we could infer all the imports using the dotted path rules, maybe in complex cases we require users to write the imports?

type_hint!(
   from a import A;
   from b import B;
   A.NestedClass | B
);

or maybe can use mixed syntax somehow

type_hint!(type_hint("a", "A.NestedClass") | b.B);

the "implicit syntax" seems nicest for the 99% of cases which users will write, so while I'm not sure exactly how to tie it all together it seems desirable to me (I think?) to have an intuitive syntax for the common case and then an escape hatch for the awkward ones.

[davidhewitt]

It looks like the annotation_stub stuff is still present, should we remove it at the same time?

[Tpt]

The code to extract modules was in the stubs.rs module and got removed