Fix disk quota middleware on non-asyncio reactor

[elacuesta]

Reported at #100 (comment)

[Copilot]

Pull request overview

This PR fixes the disk quota middleware to properly handle non-asyncio (Twisted) reactors in Scrapy versions 2.14.0 and above. Previously, the code assumed an asyncio reactor was always available when using the newer async API, which caused issues when running with Twisted's reactor.

Changes:

• Added runtime reactor detection to choose between asyncio and Twisted coroutine handling
• Imported deferred_from_coro to convert coroutines to Twisted Deferreds when needed
• Added comprehensive test coverage for both asyncio and non-asyncio reactor scenarios

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
sh_scrapy/diskquota.py	Added reactor detection logic and Twisted coroutine handling via deferred_from_coro
tests/test_diskquota.py	Added two new tests to verify correct behavior with asyncio and non-asyncio reactors
sh_scrapy/init.py	Enhanced comment documentation for the _SCRAPY_NO_SPIDER_ARG flag

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.92%. Comparing base (91f34db) to head (0cfba89).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #101      +/-   ##
==========================================
+ Coverage   88.87%   88.92%   +0.04%     
==========================================
  Files          15       15              
  Lines         890      894       +4     
==========================================
+ Hits          791      795       +4     
  Misses         99       99              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Gallaecio]

At first I thought "is it OK to create a deferred and not return it?", then I saw this is the same (yet much cleaner) than the mess I did on scrapy-zyte-api. 👍

[wRAR]

is it OK to create a deferred and not return it?

That's what _schedule_coro() in Scrapy does.

Also I remember asking you the same question at some point :)

[Gallaecio]

OK by me, although I do wonder if we should replace the 2 test functions with a single one that mocks less, e.g. perform a regular crawl with a component that raises the exception that triggers this middleware.