feat: Add use to Router for middleware support with pre-handler execution#1857
Merged
vdusek merged 4 commits intoapify:masterfrom Apr 28, 2026
Merged
feat: Add use to Router for middleware support with pre-handler execution#1857vdusek merged 4 commits intoapify:masterfrom
use to Router for middleware support with pre-handler execution#1857vdusek merged 4 commits intoapify:masterfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds router-level middleware support to the Python Router, aligning it with Crawlee JS semantics by executing registered middleware sequentially before the matched route handler.
Changes:
- Add
Router.use()to register pre-handler middleware functions. - Execute registered middleware in order on each request before dispatching to the matched/default handler.
- Add unit tests and documentation/code example demonstrating middleware usage.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/crawlee/router.py |
Introduces middleware registration/storage and executes middleware before handler dispatch. |
tests/unit/test_router.py |
Adds unit tests validating middleware execution order, label behavior, and exception interruption. |
docs/guides/request_router.mdx |
Documents router middleware behavior and links to a runnable example. |
docs/guides/code_examples/request_router/router_middleware.py |
Adds an example showing practical middleware usage (logging + timestamp). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
vdusek
approved these changes
Apr 28, 2026
Collaborator
vdusek
left a comment
vdusek
left a comment
There was a problem hiding this comment.
Two details, otherwise LGTM
vdusek
reviewed
Apr 28, 2026
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
vdusek
reviewed
Apr 28, 2026
|
|
||
| ## Middleware | ||
|
|
||
| Middlewares are functions registered with `router.use()` that execute before the matched request handler on every request, regardless of the request label. Multiple middlewares can be registered and are executed sequentially in the order they were registered. If a middleware raises an exception, the execution chain is interrupted and the handler is not called. |
Collaborator
There was a problem hiding this comment.
router.use is still not linked via ApiLink; we can link it like this class/Router#use
| ## Conclusion | ||
|
|
||
| This guide introduced you to the <ApiLink to="class/Router">`Router`</ApiLink> class and how to organize your crawling logic. You learned how to use built-in and custom routers, implement request handlers with label-based routing, handle errors with error and failed request handlers, and configure pre-navigation hooks for different crawler types. | ||
| This guide introduced you to the <ApiLink to="class/Router">`Router`</ApiLink> class and how to organize your crawling logic. You learned how to use built-in and custom routers, implement request handlers with label-based routing, add middleware with `router.use()`, handle errors with error and failed request handlers, and configure pre-navigation hooks for different crawler types. |
Collaborator
There was a problem hiding this comment.
router.use is still not linked via ApiLink; we can link it like this class/Router#use
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Router.use()for registering middleware functions that execute sequentially before the matched request handler, regardless of the request label.Issues
Testing
Router.