perf(api): DB-level pagination + drop N+1 message-count + adapter logging (Batch B)#2875
Merged
Conversation
Backend hardening (Batch B):
- DB-level pagination: /trends and /ideas?status= pushed offset/limit into
the query instead of fetching `limit+offset` rows and slicing in memory.
Added TrendRepository.{count_by_category,count_by_period} and offset params
to get_latest/get_by_category/get_all and IdeaRepository.get_by_status, so
`total` is a COUNT rather than the page size.
- Removed N+1 in the /debates list: DebateSession.to_dict() no longer
lazy-loads `self.messages` just to count them. The endpoint fetches all
counts in one DebateRepository.count_messages_for_sessions() COUNT...GROUP BY
and passes message_count into to_dict().
- Signal adapters (10 files) now log via a module logger instead of print(),
so per-adapter fetch errors flow through the logging infrastructure.
- tests/test_pagination.py: covers offset paging (no overlap), COUNT-based
totals, the bulk message-count, and the /debates message_count.
Verified: ruff + black clean, 196 tests pass (incl. 7 new).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial review caught a regression: after to_dict() stopped lazy-loading
messages (defaulting message_count to 0), the two nested-debate serializations
in get_idea_detail still called to_dict() without a count, so /ideas/{id}
reported message_count: 0 despite the messages being present. Pass
message_count=len(messages) at both sites; add a regression test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary — Backend hardening (Batch B)
Addresses the verified high/medium backend findings from the codebase health assessment.
1. DB-level pagination (
/trends,/ideas?status=)Both endpoints fetched
limit + offsetrows and sliced in memory. Now offset/limit are pushed into the SQL query, andtotalcomes from aCOUNTinstead of the page size.TrendRepository: addedoffsettoget_latest/get_by_category/get_all, and newcount_by_category/count_by_period.IdeaRepository.get_by_status: addedoffset.2. Removed N+1 in
/debatesDebateSession.to_dict()calledlen(self.messages), lazy-loading every session's full message list just to count — one extra query (and full row fetch) per session in the list. Now:DebateRepository.count_messages_for_sessions()returns all counts in oneCOUNT ... GROUP BY.to_dict(message_count=...)takes the count from the caller and never triggers a relationship load.3. Adapter logging
10 signal adapters used
print()for fetch errors. They now use a moduleloggerso errors flow through the logging infrastructure (consistent with the rest of the codebase).Tests
tests/test_pagination.py(7 new): offset pages don't overlap,totalis a COUNT (not page size) for category/period, idea-status paging, the bulk message-count (incl. empty input + zero-message sessions), and/debatesmessage_count.Notes
🤖 Generated with Claude Code