Skip to content

feat: new mypyc primitive for weakref.ref #19099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

BobTheBuidler
Copy link

@BobTheBuidler BobTheBuidler commented May 16, 2025

This PR adds a new mypyc primitive for weakref.ref

I wasn't able to figure out what name mypyc expects for weakref.proxy, so I took that out and will keep that for a separate PR later on. ref is more commonly used than proxy anyway.

for later, I tried:

  • weakref.proxy
  • weakref.ProxyType
  • weakref.weakproxy

no luck with those

also for later, I'll need to finish #19145 to add a primitive for weakref.ref.__call__

@BobTheBuidler
Copy link
Author

BobTheBuidler commented May 17, 2025

Done. This just needs tests now.

  • tests

@BobTheBuidler BobTheBuidler changed the title feat: mypyc weakref_ops.py feat: new mypyc primitive for weakref.ref May 23, 2025
@BobTheBuidler BobTheBuidler marked this pull request as ready for review May 23, 2025 23:57
@BobTheBuidler
Copy link
Author

BobTheBuidler commented May 24, 2025

@JukkaL I've cleaned up the commit history and this is now ready for review.

Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! Left a suggestion about how to make this more general.

name="weakref.ReferenceType",
arg_types=[object_rprimitive, object_rprimitive],
return_type=object_rprimitive,
c_function_name="PyWeakref_NewRef",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also support calls with a single argument, since this is probably quite common?

You'd need to add another function_op with arg_types=[object_rprimitive], and probably something like extra_int_constants=[(0, pointer_rprimitive)], to add an implicit NULL argument.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all makes sense to me, the pointer_rprimitive would become the NULL (i think?) but what does the 0 do in your above example?

Copy link
Author

@BobTheBuidler BobTheBuidler Jun 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though I don't understand it, it seems to be working. Is it an index? as in 'for extra int constant in slot 0, use pointer_rprimitive'?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 is the constant value of the argument (zero i.e. NULL).

@BobTheBuidler
Copy link
Author

BobTheBuidler commented Jun 2, 2025 via email

@BobTheBuidler
Copy link
Author

@JukkaL I know I said I'd do it later this week, but I've already finished. This is ready for review.

@JukkaL
Copy link
Collaborator

JukkaL commented Jun 3, 2025

Any thoughts on how I might determine the correct fullname for
weakref.proxy so I can handle that with both cases (1- and 2-arg) at the
same time?

You can look at how it's defined in mypy/typeshed/stdlib/weakref.pyi. You can also print the fullname of the relevant mypy AST node e.g. in some visitor (visit_member_expr).

@BobTheBuidler
Copy link
Author

hmm. It shows ProxyType in the typeshed though that wasn't working for me before. Let me try this again

@BobTheBuidler
Copy link
Author

It didn't work. I'll keep working on proxy in #19217 , this one is ready to merge if you agree

@BobTheBuidler
Copy link
Author

BobTheBuidler commented Jun 3, 2025

I got it to work for weakref.proxy on #19217. The C code generates correctly but I believe I've exposed a bug in mypyc's reference counting as my new pytest cases fail both with the new code and with the original builtin. The weakly-proxied object is being destroyed too early, while there should still be a strong reference to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants