-
Notifications
You must be signed in to change notification settings - Fork 3
feat: add test setup with cloud-tasks-emulator #53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 6 commits
Commits
Show all changes
33 commits
Select commit
Hold shift + click to select a range
866184e
feat: add test setup with cloud-tasks-emulator
devin-ai-integration[bot] ed105d2
chore: update poetry.lock
devin-ai-integration[bot] b78b0c3
style: fix whitespace in test files
devin-ai-integration[bot] 5aa1fd7
test: add comprehensive test coverage
devin-ai-integration[bot] adc33a6
style: fix docstring and linting issues
devin-ai-integration[bot] 5edb1a0
test: add specific match parameters to pytest.raises assertions
devin-ai-integration[bot] 5d6dd67
ci: update workflow to target master branch
devin-ai-integration[bot] 922e418
fix: filter queue parameter from location_path args
devin-ai-integration[bot] 05dfb12
style: format utils.py
devin-ai-integration[bot] 7da0787
fix: update emulator client to use correct gRPC endpoint
devin-ai-integration[bot] d6182cf
fix: update emulator configuration to use consistent environment vari…
devin-ai-integration[bot] 37fade8
fix: update emulator client usage in examples
devin-ai-integration[bot] 2477cd3
fix: simplify emulator configuration to use single port
devin-ai-integration[bot] ab09524
fix: update docker configuration to match emulator port setup
devin-ai-integration[bot] 38a5924
fix: add socat to emulator container and update port configuration
devin-ai-integration[bot] f9b9013
fix: update emulator configuration to bind gRPC port to all interfaces
devin-ai-integration[bot] b1ce29d
fix: remove unused import and update emulator port configuration
devin-ai-integration[bot] 21a5792
fix: bind emulator to all interfaces for both HTTP and gRPC
devin-ai-integration[bot] b92df9f
fix: simplify emulator configuration to use single port for both HTTP…
devin-ai-integration[bot] 8bb1717
fix: remove unused port mapping and ensure consistent port configuration
devin-ai-integration[bot] 77e9145
fix: ensure both HTTP and gRPC interfaces bind to all network interfaces
devin-ai-integration[bot] 12d556c
fix: update emulator connection configuration in docker-compose.yml
devin-ai-integration[bot] 21476c5
fix: update emulator client to handle host and port configuration sep…
devin-ai-integration[bot] a77fdcd
fix: enable insecure gRPC connections for emulator
devin-ai-integration[bot] b2731da
fix: update environment variables for emulator connection
devin-ai-integration[bot] 19bba44
fix: add grpc_bind_all flag to ensure emulator binds to all interfaces
devin-ai-integration[bot] f631db4
fix: separate host and port configuration for emulator connection
devin-ai-integration[bot] ca02c60
fix: disable HTTP proxy and retries in emulator client
devin-ai-integration[bot] 83e7113
fix: add bind_all flag to ensure emulator binds to all interfaces
devin-ai-integration[bot] 909938a
fix: update emulator command to bind to all interfaces
devin-ai-integration[bot] 744cd23
fix: add additional gRPC channel options for connection stability
devin-ai-integration[bot] 59ff6b8
fix: update network configuration and add UDP port mapping
devin-ai-integration[bot] ee8c2f0
fix: add DNS resolution configuration for Docker networking
devin-ai-integration[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
name: Tests | ||
|
||
on: | ||
push: | ||
branches: [ main ] | ||
pull_request: | ||
branches: [ main ] | ||
|
||
jobs: | ||
test: | ||
runs-on: ubuntu-latest | ||
|
||
services: | ||
cloud-tasks-emulator: | ||
image: ghcr.io/aertje/cloud-tasks-emulator:latest | ||
ports: | ||
- 8123:8123 | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Set up Python | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: "3.11" | ||
|
||
- name: Install Poetry | ||
uses: snok/install-poetry@v1 | ||
with: | ||
version: "1.8.2" | ||
|
||
- name: Install dependencies | ||
run: | | ||
poetry install --with test | ||
poetry install --with dev | ||
|
||
- name: Run linting | ||
run: poetry run sh scripts/lint.sh | ||
|
||
- name: Run type checking | ||
run: poetry run mypy fastapi_gcp_tasks | ||
|
||
- name: Run tests | ||
run: poetry run pytest --cov=fastapi_gcp_tasks --cov-report=xml | ||
env: | ||
IS_LOCAL: "true" | ||
CLOUD_TASKS_EMULATOR_URL: http://localhost:8123 | ||
TASK_LISTENER_BASE_URL: http://localhost:8000 | ||
TASK_PROJECT_ID: test-project | ||
TASK_LOCATION: us-central1 | ||
TASK_QUEUE: test-queue | ||
|
||
- name: Upload coverage reports to Codecov | ||
uses: codecov/codecov-action@v4 | ||
with: | ||
file: ./coverage.xml | ||
fail_ci_if_error: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
# Testing Guide | ||
|
||
## Prerequisites | ||
- Python 3.11 or higher | ||
- Poetry for dependency management | ||
- cloud-tasks-emulator (install from https://github.com/aertje/cloud-tasks-emulator) | ||
|
||
## Environment Variables | ||
Required environment variables for testing: | ||
```bash | ||
# Local development flag | ||
IS_LOCAL=true | ||
|
||
# Task emulator settings | ||
CLOUD_TASKS_EMULATOR_URL=http://localhost:8123 | ||
TASK_LISTENER_BASE_URL=http://localhost:8000 | ||
|
||
# Task queue settings | ||
TASK_PROJECT_ID=test-project | ||
TASK_LOCATION=us-central1 | ||
TASK_QUEUE=test-queue | ||
``` | ||
|
||
## Running Tests | ||
|
||
1. Install dependencies: | ||
```bash | ||
poetry install --with test | ||
``` | ||
|
||
2. Start the cloud-tasks-emulator: | ||
```bash | ||
cloud-tasks-emulator | ||
``` | ||
|
||
3. Run tests: | ||
```bash | ||
# Run all tests | ||
poetry run pytest | ||
|
||
# Run with coverage report | ||
poetry run pytest --cov=fastapi_gcp_tasks | ||
|
||
# Run specific test file | ||
poetry run pytest tests/test_delayed_route.py | ||
``` | ||
|
||
## Test Structure | ||
|
||
The test suite is organized into several key areas: | ||
|
||
### Core Functionality Tests | ||
- DelayedRouteBuilder | ||
* Basic task creation and execution | ||
* Queue auto-creation | ||
* Task options (countdown, task_id) | ||
* Error handling and validation | ||
|
||
- ScheduledRouteBuilder | ||
* Basic job creation | ||
* Cron schedule validation | ||
* Time zone handling | ||
* Job updates and uniqueness | ||
|
||
### Hook and Dependency Tests | ||
- Task Hooks | ||
* OIDC token hooks | ||
* Deadline hooks | ||
* Chained hooks | ||
* Custom hook creation | ||
- Dependencies | ||
* max_retries functionality | ||
* CloudTasksHeaders integration | ||
* Error propagation | ||
|
||
### Example Implementation Tests | ||
- Simple Example | ||
* Local mode functionality | ||
* Task queueing | ||
* Environment variable handling | ||
* Default settings | ||
|
||
- Full Example | ||
* Chained hook configuration | ||
* Retry mechanisms | ||
* Scheduled tasks | ||
* Environment-specific behavior | ||
|
||
### Environment-Specific Tests | ||
- Local Development | ||
* Emulator integration | ||
* Default configurations | ||
* Environment variable handling | ||
|
||
- Deployed Environment | ||
* OIDC token integration | ||
* Cloud Scheduler integration | ||
* Job scheduling | ||
* Production settings | ||
|
||
## Contributing Tests | ||
|
||
When adding new tests: | ||
1. Follow existing test patterns | ||
2. Ensure proper type hints are used | ||
3. Run formatting and linting: | ||
```bash | ||
sh scripts/format.sh | ||
sh scripts/lint.sh | ||
``` | ||
4. Add appropriate docstrings for complex test cases | ||
5. Consider edge cases and error scenarios |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
import pytest | ||
from fastapi import FastAPI | ||
from fastapi.testclient import TestClient | ||
|
||
from fastapi_gcp_tasks import DelayedRouteBuilder, ScheduledRouteBuilder | ||
from fastapi_gcp_tasks.utils import emulator_client, queue_path | ||
|
||
|
||
@pytest.fixture | ||
def app(): | ||
"""Create a fresh FastAPI application for each test.""" | ||
return FastAPI() | ||
|
||
|
||
@pytest.fixture | ||
def delayed_route(): | ||
"""Create a DelayedRouteBuilder configured for testing with emulator.""" | ||
return DelayedRouteBuilder( | ||
client=emulator_client(), | ||
base_url="http://localhost:8000", | ||
queue_path=queue_path( | ||
project="test-project", | ||
location="us-central1", | ||
queue="test-queue", | ||
), | ||
) | ||
|
||
|
||
@pytest.fixture | ||
def scheduled_route(): | ||
"""Create a ScheduledRouteBuilder configured for testing.""" | ||
return ScheduledRouteBuilder( | ||
base_url="http://localhost:8000", | ||
location_path="projects/test-project/locations/us-central1", | ||
) | ||
|
||
|
||
@pytest.fixture | ||
def test_client(app): | ||
"""Create a TestClient instance for the FastAPI app.""" | ||
return TestClient(app) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the branch name is
master