Commit 2f69531
authored
Break the database.users import cycle with utils.subscription and honor the dev goals limit (#11621)
# Problem
Two reliability defects, rebuilt from #9966 and #9977, which had gone
stale against current main. Both bugs are still present on `origin/main`
today; I re-verified each against the current code before rebuilding.
**1. `database/users.py` and `utils/subscription.py` form an import
cycle.**
`backend/database/users.py` imported back into `utils.subscription` at
module scope:
```python
from models.other import Person
from utils.subscription import get_default_basic_subscription
import logging
```
while `backend/utils/subscription.py` does `import database.users as
users_db` at module scope. Whichever module is imported first decides
whether the pair works. Importing `utils.subscription` first raises:
```
ImportError: cannot import name 'get_default_basic_subscription' from partially initialized
module 'utils.subscription' (most likely due to a circular import)
```
Importing `database.users` first happens to succeed, which is why this
surfaces intermittently as unrelated files change what gets imported
first. The database -> utils edge is also the reverse of the documented
backend layering (database/ -> utils/ -> routers/).
**2. `GET /v1/dev/user/goals` ignores its documented `limit` when
`include_inactive=true`.**
The route clamps `limit` and then drops it on one branch:
```python
# oversized limit cannot stream the whole collection. Mirrors the GET /v3/memories hardening.
limit = max(1, min(limit, 1000))
if include_inactive:
goals = goals_db.get_all_goals(uid, include_inactive=True)
else:
goals = goals_db.get_user_goals(uid, limit=limit)
```
`get_all_goals` had no limit parameter and streams the whole goals
collection, so the clamp is dead code on that branch and the endpoint
returns every goal the user has ever created, ignoring its own
documented "**limit**: Maximum number of goals to return".
# Reachability / failure scenario
- Import cycle: any entry point that imports `utils.subscription` before
`database.users` fails at import time with the error above. Reproduced
with a fresh interpreter running `import utils.subscription` from
`backend/`.
- Goals limit: `GET /v1/dev/user/goals?include_inactive=true&limit=10`
from any developer API consumer returns the full collection and reads
every goal document in Firestore. The path that lets it through is
`routers/developer.py::get_goals` -> `database/goals.py::get_all_goals`.
# Fix
**Import cycle.** The `get_default_basic_subscription` import in
`database/users.py` moves from module scope to its call sites: inside
`get_user_subscription`, and at the top of
`get_user_valid_subscription`. Since #9966 was written, main added a
`provision=False` branch to `get_user_valid_subscription` that also
calls `get_default_basic_subscription`, earlier in the function than the
old fallback site, so the function-level import sits above the first
call site and covers both. Each deferred import carries a comment
recording why it must not be folded back to the top.
**Goals limit.** `get_all_goals` accepts an opt-in `limit` applied after
the in-Python newest-first sort. The bound is deliberately not pushed
into the Firestore query: `order_by('created_at')` excludes documents
that lack the field entirely, and legacy or manually created goals can
lack `created_at`, so a query-level order+limit would silently drop
them. Goals without `created_at` coerce to `datetime.min` and sort last,
so the bounded page is the newest goals with dateless legacy goals
appearing only once dated goals run out, and the response honours the
documented limit.
Deliberately not changed: every other `get_all_goals` caller (the
goal-by-id lookup, `routers/goals.py`, and the MCP goal reads) omits
`limit` and keeps its fetch-everything behavior; bounding those is a
behavior change with no bug attached.
# Tests
- `backend/tests/unit/test_subscription_import_cycle.py`: runs `import
utils.subscription` (and separately `import database.users`) first in a
fresh interpreter and asserts both succeed standalone. A fresh
interpreter is the only honest seam here: once pytest has imported
either module, the order is decided for the whole session, so an
in-process assertion would pass either way.
- `backend/tests/unit/test_dev_goals_limit_include_inactive.py`:
router-level tests assert the clamp is delegated (captured kwargs for
limit 5, ceiling clamp to 1000, and the active-only branch unchanged);
database-level tests assert the bounded page is the newest goals, that a
legacy goal without `created_at` is retained and sorts last instead of
being dropped, that dated goals fill the page first, and that callers
omitting `limit` keep the full unordered fetch with nothing pushed into
the query.
# Verification
All commands run from `backend/` on Windows (Git Bash), `PYTHONUTF8=1`.
Prove-fail. Reverted the three product files to `origin/main` (`git diff
origin/main -- <files>` printed nothing, byte-identical), then ran both
test files:
```
FAILED tests/unit/test_subscription_import_cycle.py::test_utils_subscription_imports_standalone
AssertionError: utils.subscription is not importable on its own:
ImportError: cannot import name 'get_default_basic_subscription' from partially initialized
module 'utils.subscription' (most likely due to a circular import)
FAILED tests/unit/test_dev_goals_limit_include_inactive.py::test_get_all_goals_bounds_the_query_when_limit_is_given
TypeError: get_all_goals() got an unexpected keyword argument 'limit'
5 failed, 3 passed
```
The three passes on the unfixed tree are the behavior-preservation
guards (importing `database.users` first, the active-only branch, and
the unbounded default), which must pass on both sides. Restored the fix:
`8 passed`.
Static and contract gate:
```
black --line-length 120 --skip-string-normalization --check <5 files> -> 5 files would be left unchanged
python -m pyright -p pyrightconfig.json database/goals.py -> 0 errors (users.py and routers/developer.py are in the pyright exclude list)
python scripts/check_module_stub_pollution.py -> 898 test files, 0 violations
python scripts/scan_async_blockers.py --dirs routers utils -> 0 findings in fail scope
python scripts/scan_import_time_side_effects.py -> 804 files, 0 violations
python scripts/check_conversation_lifecycle_writes.py -> passed
```
Workflow contracts: the touched files match three high-risk workflows
(goals, users, developer). Ran all 13 unit and service test files listed
by those workflows: `391 passed, 2 failed`, and the sync workflow file
separately: `77 passed, 10 failed`. All 12 failures reproduce
identically on a clean unmodified `origin/main` checkout in the same
batch order (`tests/services/users/test_account_deletion.py` pair passes
standalone on both trees), so they are pre-existing local-environment
and test-ordering issues, not caused by this change.
`testing/e2e/test_account_deletion_cloud_tasks.py` needs live services
and errors locally on both trees.
`scripts/pr-preflight --suggest`: Product invariants affected: none. No
`fix:` commits, so no failure-class declaration is required.
Full `scripts/pr-preflight --pr-body-file` run: every selected manifest
check passes (including backend-route-policy-baseline,
backend-async-blockers, backend-import-purity,
backend-workflow-contracts, the line-count ratchet with the declarations
below) except `desktop-backend-candidate-probe-fixtures`, which fails on
this Windows machine with "gemini_proxy: total response deadline is
unsupported on this runner". The files that check exercises are
byte-identical to `origin/main` in this branch (the diff is five backend
files), so it is a local runner-capability issue of the same class as
the known Windows-only `detect_platform` false failure; CI runs on
Linux.
# Impact
- The import cycle breaks any standalone consumer of
`utils.subscription` and makes backend import order fragile: whether it
bites depends on which module a given entry point or test session
happens to import first. Developer-facing reliability, low direct user
impact.
- The goals endpoint bug affects developer API consumers who pass
`include_inactive=true`: response size and Firestore read cost grow with
the user's total historical goal count instead of respecting the
documented bound. Narrow trigger, real cost on goal-heavy accounts.
Line-Count-Exception: backend/database/users.py | 2131 -> 2140 |
Breaking the utils.subscription import cycle moves one module-level
import to deferred call sites with comments recording why each must stay
deferred so the import is not folded back to the top and silently
re-broken.
Line-Count-Exception: backend/routers/developer.py | 2107 -> 2111 | The
four added lines are the comment recording why the goals bound is
applied after the in-Python sort rather than pushed into a Firestore
order_by that would drop legacy goals lacking created_at.5 files changed
Lines changed: 269 additions & 3 deletions
File tree
- backend
- database
- routers
- tests/unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
337 | 337 | | |
338 | 338 | | |
339 | 339 | | |
| 340 | + | |
340 | 341 | | |
341 | 342 | | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
342 | 355 | | |
343 | 356 | | |
344 | 357 | | |
345 | 358 | | |
346 | 359 | | |
347 | 360 | | |
348 | | - | |
| 361 | + | |
349 | 362 | | |
350 | 363 | | |
351 | 364 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
32 | | - | |
33 | 32 | | |
34 | 33 | | |
35 | 34 | | |
| |||
1547 | 1546 | | |
1548 | 1547 | | |
1549 | 1548 | | |
| 1549 | + | |
| 1550 | + | |
| 1551 | + | |
| 1552 | + | |
| 1553 | + | |
1550 | 1554 | | |
1551 | 1555 | | |
1552 | 1556 | | |
| |||
1678 | 1682 | | |
1679 | 1683 | | |
1680 | 1684 | | |
| 1685 | + | |
| 1686 | + | |
| 1687 | + | |
| 1688 | + | |
| 1689 | + | |
1681 | 1690 | | |
1682 | 1691 | | |
1683 | 1692 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1942 | 1942 | | |
1943 | 1943 | | |
1944 | 1944 | | |
1945 | | - | |
| 1945 | + | |
| 1946 | + | |
| 1947 | + | |
| 1948 | + | |
| 1949 | + | |
1946 | 1950 | | |
1947 | 1951 | | |
1948 | 1952 | | |
| |||
Lines changed: 178 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
0 commit comments