Guidance around testing, including outbox and ENQUEUE_ON_COMMIT

[silviogutierrez]

I'm assuming like many others, I'm using this for sending emails.

Migration was super straightforward just wrapped my central send_email with @task and renamed all calls to send_email.enqueue(). Typing worked flawlessly as did functionality.

Problem is with my tests. I use outbox from pytest but the same issue applies to mail.outbox in regular Django.

Seems like outbox is not populated even if ImmediateBacked is used when I test a view. I imagine it has something to do with the default transaction handling because all transactions are actually reverted at the end of the test for speed benefits. This is a test runner optimization. Resets the DB after every test.

Flipping ENQUEUE_ON_COMMIT to False just for tests seemed to work. But now we're introducing possible differences between live and test behavior.

I think at the very least, this should be clarified. Or maybe even handled. But I suspect no tasks will run on tests by default because of the commit behavior.

[RealOrangeOne]

This is interesting, and something I've seen a little. The behaviour is different. I think the issue is that TestCase doesn't close the transaction until after the case finishes. TransactionTestCase should work as expected.

I think the only solution is to use ENQUEUE_ON_COMMIT = False, since in reality that's what you need in a test. The alternative would be a flush API to flush the buffered tasks, but that feels like it'd only be useful in tests, and a bit of a foot-gun in regular code.

[silviogutierrez]

Yea I'm not sure what the best approach is, ENQUEUE_ON_COMMIT works fine. Just wondering what will lead to more surprises. Flush or local-only ENQUEUE_ON_COMMIT.

Django does have some magic around running in a testing context, so maybe that's worth looking at.

TransactionTestCase works, but I believe it's dramatically slower.

Either way we go, this is great work. Just deployed it to production.

[RealOrangeOne]

I had some handling in there, to force set ENQUEUE_ON_COMMIT to False during tests, but the implementation didn't work with multiple databases, so I ripped it out. I think that's more surprising anyway. I suspect requiring the user be explicit, and instead just documenting this caveat is the way to go.