Skip to content

feat: add test setup with cloud-tasks-emulator #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
866184e
feat: add test setup with cloud-tasks-emulator
devin-ai-integration[bot] Feb 8, 2025
ed105d2
chore: update poetry.lock
devin-ai-integration[bot] Feb 8, 2025
b78b0c3
style: fix whitespace in test files
devin-ai-integration[bot] Feb 8, 2025
5aa1fd7
test: add comprehensive test coverage
devin-ai-integration[bot] Feb 8, 2025
adc33a6
style: fix docstring and linting issues
devin-ai-integration[bot] Feb 8, 2025
5edb1a0
test: add specific match parameters to pytest.raises assertions
devin-ai-integration[bot] Feb 8, 2025
5d6dd67
ci: update workflow to target master branch
devin-ai-integration[bot] Feb 8, 2025
922e418
fix: filter queue parameter from location_path args
devin-ai-integration[bot] Feb 8, 2025
05dfb12
style: format utils.py
devin-ai-integration[bot] Feb 8, 2025
7da0787
fix: update emulator client to use correct gRPC endpoint
devin-ai-integration[bot] Feb 8, 2025
d6182cf
fix: update emulator configuration to use consistent environment vari…
devin-ai-integration[bot] Feb 8, 2025
37fade8
fix: update emulator client usage in examples
devin-ai-integration[bot] Feb 8, 2025
2477cd3
fix: simplify emulator configuration to use single port
devin-ai-integration[bot] Feb 8, 2025
ab09524
fix: update docker configuration to match emulator port setup
devin-ai-integration[bot] Feb 8, 2025
38a5924
fix: add socat to emulator container and update port configuration
devin-ai-integration[bot] Feb 8, 2025
f9b9013
fix: update emulator configuration to bind gRPC port to all interfaces
devin-ai-integration[bot] Feb 8, 2025
b1ce29d
fix: remove unused import and update emulator port configuration
devin-ai-integration[bot] Feb 8, 2025
21a5792
fix: bind emulator to all interfaces for both HTTP and gRPC
devin-ai-integration[bot] Feb 8, 2025
b92df9f
fix: simplify emulator configuration to use single port for both HTTP…
devin-ai-integration[bot] Feb 8, 2025
8bb1717
fix: remove unused port mapping and ensure consistent port configuration
devin-ai-integration[bot] Feb 8, 2025
77e9145
fix: ensure both HTTP and gRPC interfaces bind to all network interfaces
devin-ai-integration[bot] Feb 8, 2025
12d556c
fix: update emulator connection configuration in docker-compose.yml
devin-ai-integration[bot] Feb 8, 2025
21476c5
fix: update emulator client to handle host and port configuration sep…
devin-ai-integration[bot] Feb 8, 2025
a77fdcd
fix: enable insecure gRPC connections for emulator
devin-ai-integration[bot] Feb 8, 2025
b2731da
fix: update environment variables for emulator connection
devin-ai-integration[bot] Feb 8, 2025
19bba44
fix: add grpc_bind_all flag to ensure emulator binds to all interfaces
devin-ai-integration[bot] Feb 8, 2025
f631db4
fix: separate host and port configuration for emulator connection
devin-ai-integration[bot] Feb 8, 2025
ca02c60
fix: disable HTTP proxy and retries in emulator client
devin-ai-integration[bot] Feb 8, 2025
83e7113
fix: add bind_all flag to ensure emulator binds to all interfaces
devin-ai-integration[bot] Feb 8, 2025
909938a
fix: update emulator command to bind to all interfaces
devin-ai-integration[bot] Feb 8, 2025
744cd23
fix: add additional gRPC channel options for connection stability
devin-ai-integration[bot] Feb 8, 2025
59ff6b8
fix: update network configuration and add UDP port mapping
devin-ai-integration[bot] Feb 8, 2025
ee8c2f0
fix: add DNS resolution configuration for Docker networking
devin-ai-integration[bot] Feb 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Tests

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the branch name is master


jobs:
test:
runs-on: ubuntu-latest

services:
cloud-tasks-emulator:
image: ghcr.io/aertje/cloud-tasks-emulator:latest
ports:
- 8123:8123

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: "1.8.2"

- name: Install dependencies
run: |
poetry install --with test
poetry install --with dev

- name: Run linting
run: poetry run sh scripts/lint.sh

- name: Run type checking
run: poetry run mypy fastapi_gcp_tasks

- name: Run tests
run: poetry run pytest --cov=fastapi_gcp_tasks --cov-report=xml
env:
IS_LOCAL: "true"
CLOUD_TASKS_EMULATOR_URL: http://localhost:8123
TASK_LISTENER_BASE_URL: http://localhost:8000
TASK_PROJECT_ID: test-project
TASK_LOCATION: us-central1
TASK_QUEUE: test-queue

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
fail_ci_if_error: true
112 changes: 112 additions & 0 deletions docs/tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
# Testing Guide

## Prerequisites
- Python 3.11 or higher
- Poetry for dependency management
- cloud-tasks-emulator (install from https://github.com/aertje/cloud-tasks-emulator)

## Environment Variables
Required environment variables for testing:
```bash
# Local development flag
IS_LOCAL=true

# Task emulator settings
CLOUD_TASKS_EMULATOR_URL=http://localhost:8123
TASK_LISTENER_BASE_URL=http://localhost:8000

# Task queue settings
TASK_PROJECT_ID=test-project
TASK_LOCATION=us-central1
TASK_QUEUE=test-queue
```

## Running Tests

1. Install dependencies:
```bash
poetry install --with test
```

2. Start the cloud-tasks-emulator:
```bash
cloud-tasks-emulator
```

3. Run tests:
```bash
# Run all tests
poetry run pytest

# Run with coverage report
poetry run pytest --cov=fastapi_gcp_tasks

# Run specific test file
poetry run pytest tests/test_delayed_route.py
```

## Test Structure

The test suite is organized into several key areas:

### Core Functionality Tests
- DelayedRouteBuilder
* Basic task creation and execution
* Queue auto-creation
* Task options (countdown, task_id)
* Error handling and validation

- ScheduledRouteBuilder
* Basic job creation
* Cron schedule validation
* Time zone handling
* Job updates and uniqueness

### Hook and Dependency Tests
- Task Hooks
* OIDC token hooks
* Deadline hooks
* Chained hooks
* Custom hook creation
- Dependencies
* max_retries functionality
* CloudTasksHeaders integration
* Error propagation

### Example Implementation Tests
- Simple Example
* Local mode functionality
* Task queueing
* Environment variable handling
* Default settings

- Full Example
* Chained hook configuration
* Retry mechanisms
* Scheduled tasks
* Environment-specific behavior

### Environment-Specific Tests
- Local Development
* Emulator integration
* Default configurations
* Environment variable handling

- Deployed Environment
* OIDC token integration
* Cloud Scheduler integration
* Job scheduling
* Production settings

## Contributing Tests

When adding new tests:
1. Follow existing test patterns
2. Ensure proper type hints are used
3. Run formatting and linting:
```bash
sh scripts/format.sh
sh scripts/lint.sh
```
4. Add appropriate docstrings for complex test cases
5. Consider edge cases and error scenarios
893 changes: 582 additions & 311 deletions poetry.lock

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,13 @@ mypy = "^1.10.0"
types-ujson = "^5.10.0.20240515"
types-protobuf = "^5.26.0.20240422"

[tool.poetry.group.test.dependencies]
pytest = "^8.0.0"
pytest-asyncio = "^0.23.0"
pytest-cov = "^4.1.0"
httpx = "^0.26.0"
pytest-env = "^1.1.0"

[tool.mypy]
plugins = ["pydantic.mypy"]
ignore_missing_imports = true
Expand Down
41 changes: 41 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient

from fastapi_gcp_tasks import DelayedRouteBuilder, ScheduledRouteBuilder
from fastapi_gcp_tasks.utils import emulator_client, queue_path


@pytest.fixture
def app():
"""Create a fresh FastAPI application for each test."""
return FastAPI()


@pytest.fixture
def delayed_route():
"""Create a DelayedRouteBuilder configured for testing with emulator."""
return DelayedRouteBuilder(
client=emulator_client(),
base_url="http://localhost:8000",
queue_path=queue_path(
project="test-project",
location="us-central1",
queue="test-queue",
),
)


@pytest.fixture
def scheduled_route():
"""Create a ScheduledRouteBuilder configured for testing."""
return ScheduledRouteBuilder(
base_url="http://localhost:8000",
location_path="projects/test-project/locations/us-central1",
)


@pytest.fixture
def test_client(app):
"""Create a TestClient instance for the FastAPI app."""
return TestClient(app)
Loading