[feat] 백오피스 대시보드: 사용량·LLM 품질 모니터링 - #52
Merged
Merged
Conversation
- generation_log 테이블(V9) + 생성 시 best-effort 기록(GenerationLogRecorder) → 호출 수·재시도율·지연·토큰·에러를 comment_status로 못 보던 부분까지 집계 - 사용량/품질 집계 서비스 분리(UsageStatsService·QualityStatsService, SRP) · 사용량: 오늘 대화·메시지·카드·가입, DAU/WAU, 감정 분포, 일별 활동 추이 · 품질: 생성 성공률·호출·재시도율·avg/p95 지연·토큰·막힌 PENDING - GET /api/admin/dashboard/usage·/quality - admin-web 대시보드 페이지(사용량·품질 2섹션, recharts) + 라우트·네비 활성화 - KST 날짜 경계 계산 중앙화(KstDashboardDates)
- '오늘 메시지'를 '유저당 평균 메시지'(오늘 유저 발화 / DAU)로 교체 - UsageStatsResponse: avgMessagesPerUser 추가, todayCharacterMessages 제거
Test Coverage
|
좁은 카드에서 6종 범례가 5+1로 어색하게 접히던 것을 3+3 균등 2줄로 변경
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Walkthrough관리자 대시보드에 사용량·LLM 품질 통계 API와 웹 UI를 추가했습니다. 생성 로그를 저장해 성공률, 재시도, 지연, 토큰 및 일별 추이를 집계하고 Changes대시보드 모니터링 기능
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant AdminWeb
participant AdminDashboardController
participant UsageStatsService
participant QualityStatsService
participant Repositories
AdminWeb->>AdminDashboardController: 사용량 또는 품질 통계 요청
AdminDashboardController->>UsageStatsService: days 전달
AdminDashboardController->>QualityStatsService: days 전달
UsageStatsService->>Repositories: 사용량 집계 쿼리 실행
QualityStatsService->>Repositories: 생성 로그 품질 집계 조회
Repositories-->>UsageStatsService: 사용량 원시 결과
Repositories-->>QualityStatsService: 품질 원시 결과
UsageStatsService-->>AdminDashboardController: UsageStatsResponse
QualityStatsService-->>AdminDashboardController: QualityStatsResponse
AdminDashboardController-->>AdminWeb: ApiResponse 응답
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt (1)
40-51: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win생성 로그 계약을 검증하는 테스트를 추가하세요.
현재
relaxedmock은 주입만 검증하므로 타입·성공 여부·재시도 횟수·토큰·최종 실패 기록이 깨져도 테스트가 통과합니다. 빠른 단위 테스트에서record()인자를 검증하는 방안은 변경 비용이 낮고 계약을 명확히 보장합니다. 저장소 통합 테스트는 DB 매핑까지 확인할 수 있지만 실행 비용이 더 큽니다.검증 예시
verify(exactly = 1) { generationLogRecorder.record( type = GenerationType.COMMENT, success = true, attemptCount = 1, latencyMs = any(), usedTokens = 123, ) } verify(exactly = 1) { generationLogRecorder.record( type = GenerationType.COMMENT, success = false, attemptCount = 2, latencyMs = any(), usedTokens = null, failureReason = any(), ) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt` around lines 40 - 51, generationLogRecorder is relaxed without verifying the generation-log contract. Add unit-test assertions around CommentGenerationService scenarios that verify GenerationLogRecorder.record() receives the expected COMMENT type, success status, attempt count, token usage, latency, and failureReason for both successful and final-failure outcomes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@admin-web/src/pages/dashboard/quality-section.tsx`:
- Line 72: Update the quality stats request in the component using useCustom to
consume its error state and render an explicit failure state when the API
request fails, instead of allowing undefined stats to return null. Preserve the
existing loading and successful-data rendering behavior, and apply the same
handling to the usage section’s corresponding useCustom flow.
- Around line 45-69: Remove the duplicated short() and GenerationTooltip
implementations from the quality section, and reuse the existing shared
equivalents from usage-section.tsx, including ActivityTooltip where applicable.
Update the quality section’s references and imports so its current chart
behavior remains unchanged.
In `@admin-web/src/pages/dashboard/usage-section.tsx`:
- Around line 59-96: Remove the duplicated short() and ActivityTooltip
implementations from usage-section.tsx and reuse the corresponding shared
symbols already defined in quality-section.tsx, preserving the existing tooltip
behavior and formatting.
- Line 99: Update the useCustom destructuring and rendering in the usage section
to handle error and isError, matching the pattern in quality-section.tsx.
Display an API failure state separately from the existing no-data state,
including the applicable handling around the logic at lines 118-120.
- Around line 154-178: Update the emotion legend rendering alongside the
hasEmotion check in the dashboard usage section so it is shown only when
hasEmotion is true; when no emotion data exists, display only the existing “데이터
없음” state and omit the emotionData entries. Keep the chart and legend rendering
unchanged for populated data.
In `@src/main/kotlin/com/nexters/gamss/admin/service/KstDashboardDates.kt`:
- Around line 14-26: Update the KstDashboardDates calculation flow so one
LocalDate is captured at the service entry point and reused for sinceDaysAgo(),
startOfToday(), startOfTomorrow(), and dateAxis() instead of each method calling
today(). Prefer injecting a Clock for obtaining the captured KST date, and pass
that reference date through the related service calculations to keep KPI ranges
and date axes consistent.
In `@src/main/kotlin/com/nexters/gamss/admin/service/QualityStatsService.kt`:
- Around line 27-50: Update QualityStatsService.getQualityStats so aggregate
success, retry, latency, token, and percentile statistics are computed by the
database rather than loading every GenerationLog through findAllSince(since).
Add or reuse a repository-level aggregation query/projection, while preserving
the existing response values and dailyGeneration behavior.
In `@src/main/kotlin/com/nexters/gamss/admin/service/UsageStatsService.kt`:
- Around line 64-85: Update UsageStatsService.buildDailyActivity and the
corresponding QualityStatsService aggregation to use database-side date GROUP BY
queries instead of loading all timestamps through findCreatedAtsSince and
grouping with bucketByDate. Preserve the existing KST date buckets and
DailyActivityResponse results, and reuse shared query logic where appropriate.
In
`@src/main/kotlin/com/nexters/gamss/conversation/service/CommentGenerationService.kt`:
- Around line 161-164: 재시도 루프가 예상 밖 예외도 품질 로그에 남기도록 수정하세요.
src/main/kotlin/com/nexters/gamss/conversation/service/CommentGenerationService.kt의
161-164 및 207-210에서 각 루프에 별도 catch (e: Exception)를 추가해 현재 시도 횟수와 지연 시간을 기록한 뒤 즉시
재던지며, 기존 CommentGenerationFailedException 재시도 정책은 유지하세요. 같은 파일 176-182의
failureReasonOf는 Throwable?을 받아 예상 밖 예외의 메시지 또는 안전한 요약을 반환하도록 변경하세요.
In
`@src/test/kotlin/com/nexters/gamss/admin/service/DashboardStatsIntegrationTest.kt`:
- Around line 40-121: 보정 지표의 0건 분모 null 반환 경로를 통합 테스트로 검증하세요. 기존 `사용량 지표를 집계한다`
및 `LLM 품질 지표를 집계한다` 테스트와 별도로 데이터가 없는 기간을 조회해 `avgMessagesPerUser`,
`successRate`, `retryRate`가 각각 null인지 assert하고, 기존 비어 있지 않은 케이스 검증은 유지하세요.
---
Outside diff comments:
In
`@src/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt`:
- Around line 40-51: generationLogRecorder is relaxed without verifying the
generation-log contract. Add unit-test assertions around
CommentGenerationService scenarios that verify GenerationLogRecorder.record()
receives the expected COMMENT type, success status, attempt count, token usage,
latency, and failureReason for both successful and final-failure outcomes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1a643299-2f69-4bd2-963e-d07142dbe8e9
📒 Files selected for processing (28)
admin-web/src/App.tsxadmin-web/src/components/layout/AdminLayout.tsxadmin-web/src/components/metric-card.tsxadmin-web/src/pages/dashboard/index.tsxadmin-web/src/pages/dashboard/quality-section.tsxadmin-web/src/pages/dashboard/usage-section.tsxsrc/main/kotlin/com/nexters/gamss/admin/controller/AdminDashboardController.ktsrc/main/kotlin/com/nexters/gamss/admin/controller/dto/DailyActivityResponse.ktsrc/main/kotlin/com/nexters/gamss/admin/controller/dto/DailyGenerationResponse.ktsrc/main/kotlin/com/nexters/gamss/admin/controller/dto/EmotionCountResponse.ktsrc/main/kotlin/com/nexters/gamss/admin/controller/dto/QualityStatsResponse.ktsrc/main/kotlin/com/nexters/gamss/admin/controller/dto/UsageStatsResponse.ktsrc/main/kotlin/com/nexters/gamss/admin/service/KstDashboardDates.ktsrc/main/kotlin/com/nexters/gamss/admin/service/QualityStatsService.ktsrc/main/kotlin/com/nexters/gamss/admin/service/UsageStatsService.ktsrc/main/kotlin/com/nexters/gamss/card/repository/CardRepository.ktsrc/main/kotlin/com/nexters/gamss/card/repository/EmotionCountProjection.ktsrc/main/kotlin/com/nexters/gamss/conversation/repository/ConversationRepository.ktsrc/main/kotlin/com/nexters/gamss/conversation/repository/MessageRepository.ktsrc/main/kotlin/com/nexters/gamss/conversation/service/CommentGenerationService.ktsrc/main/kotlin/com/nexters/gamss/member/repository/MemberRepository.ktsrc/main/kotlin/com/nexters/gamss/monitoring/domain/GenerationLog.ktsrc/main/kotlin/com/nexters/gamss/monitoring/domain/GenerationType.ktsrc/main/kotlin/com/nexters/gamss/monitoring/repository/GenerationLogRepository.ktsrc/main/kotlin/com/nexters/gamss/monitoring/service/GenerationLogRecorder.ktsrc/main/resources/db/migration/V9__generation_log.sqlsrc/test/kotlin/com/nexters/gamss/admin/service/DashboardStatsIntegrationTest.ktsrc/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt
- 생성 재시도 루프에서 예상 밖 예외도 generation_log에 기록 후 재던짐 (기존엔 CommentGenerationFailedException만 기록돼 성공률이 과대 집계) - KstDashboardDates: 요청당 기준 날짜(today)를 한 번만 캡처해 전달 (자정 경계에서 KPI 범위와 일별 축 기준일 어긋남 방지) - 프론트: API 실패 시 에러 상태(재시도) 표시, 감정 범례 hasEmotion 가드 - 프론트: shortDate·SeriesTooltip·SectionError 공통 모듈로 중복 제거 - 데이터 0건일 때 평균·비율 null 반환 테스트 추가
# Conflicts: # src/main/kotlin/com/nexters/gamss/conversation/service/CommentGenerationService.kt
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
admin-web/src/pages/dashboard/quality-section.tsx (1)
15-24: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win빈 기간의 지연 시간을
nullms로 표시하지 않도록 nullable 계약을 반영하세요.Line 22-23이
number로 고정되어 있어 API가 빈 기간에null을 반환하면 Line 36에서nullms가 렌더링됩니다.avgLatencyMs/p95LatencyMs를number | null로 선언하고latency(null)은—를 반환하세요.수정 예시
-interface QualityStats { - avgLatencyMs: number - p95LatencyMs: number +interface QualityStats { + avgLatencyMs: number | null + p95LatencyMs: number | null } -function latency(ms: number): string { +function latency(ms: number | null): string { + if (ms === null) return '—' if (ms >= 1000) {Also applies to: 32-37
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@admin-web/src/pages/dashboard/quality-section.tsx` around lines 15 - 24, Update the QualityStats interface so avgLatencyMs and p95LatencyMs accept number | null, and adjust the latency rendering logic to return an em dash for null values instead of producing “nullms”; preserve the existing millisecond formatting for numeric values.src/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt (1)
40-51: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win생성 로그 기록 계약도 검증하세요.
GenerationLogRecorder가 relaxed mock이라 기록 호출이 빠지거나attemptCount가 틀려도 테스트가 통과합니다. 성공, 재시도 소진, 비재시도 예외 각각에 대해type,success,attemptCount를verify해 대시보드 집계의 핵심 입력을 보호하세요.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt` around lines 40 - 51, CommentGenerationServiceTest에서 relaxed mock인 generationLogRecorder에 대한 계약 검증이 빠져 있다. 성공, 재시도 소진, 비재시도 예외를 다루는 각 테스트에 GenerationLogRecorder 호출 검증을 추가하고, 기록된 type·success·attemptCount가 각 시나리오의 기대값과 일치하는지 확인하라.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@admin-web/src/pages/dashboard/quality-section.tsx`:
- Around line 15-24: Update the QualityStats interface so avgLatencyMs and
p95LatencyMs accept number | null, and adjust the latency rendering logic to
return an em dash for null values instead of producing “nullms”; preserve the
existing millisecond formatting for numeric values.
In
`@src/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt`:
- Around line 40-51: CommentGenerationServiceTest에서 relaxed mock인
generationLogRecorder에 대한 계약 검증이 빠져 있다. 성공, 재시도 소진, 비재시도 예외를 다루는 각 테스트에
GenerationLogRecorder 호출 검증을 추가하고, 기록된 type·success·attemptCount가 각 시나리오의 기대값과
일치하는지 확인하라.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d1d0b8f0-0635-415e-a77c-50c31389a5f2
📒 Files selected for processing (9)
admin-web/src/pages/dashboard/chart-shared.tsxadmin-web/src/pages/dashboard/quality-section.tsxadmin-web/src/pages/dashboard/usage-section.tsxsrc/main/kotlin/com/nexters/gamss/admin/service/KstDashboardDates.ktsrc/main/kotlin/com/nexters/gamss/admin/service/QualityStatsService.ktsrc/main/kotlin/com/nexters/gamss/admin/service/UsageStatsService.ktsrc/main/kotlin/com/nexters/gamss/conversation/service/CommentGenerationService.ktsrc/test/kotlin/com/nexters/gamss/admin/service/DashboardStatsIntegrationTest.ktsrc/test/kotlin/com/nexters/gamss/conversation/service/CommentGenerationServiceTest.kt
kite707
reviewed
Jul 27, 2026
kite707
approved these changes
Jul 27, 2026
kite707
left a comment
kite707
left a comment
Collaborator
There was a problem hiding this comment.
질문에 잘 답변해주셔서 감사합니다! 늦은 시간까지 고생하셨어요!
청소 스케줄러·대시보드 막힌 PENDING 지표가 comment_status + comment_status_updated_at 조건으로 조회하는데 인덱스가 없어 messages 전체를 스캔(스케줄러는 1분마다)한다. 가장 빠르게 커지는 테이블이라 스캔 비용 누적을 막기 위해 복합 인덱스를 둔다.
Merged
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 연관 이슈
📌 개요
백오피스 대시보드('준비중'이던 자리)를 구현한다. 서비스 사용량과 LLM 생성 품질/안정성을 한 화면에서 모니터링한다. (보안 지표는 별도 이슈)
🔧 주요 변경사항
관측 로그 (
generation_log)GenerationLog엔티티, 생성 시GenerationLogRecorder로 한 줄씩 best-effort 기록(기록 실패가 생성 흐름을 깨지 않음)집계 (SRP: 사용량/품질 서비스 분리)
UsageStatsService— 오늘 대화·유저 발화, 유저당 평균 메시지, 카드·가입, DAU/WAU, 감정 분포, 일별 활동 추이QualityStatsService— 생성 성공률·호출 수·재시도율·avg/p95 지연·토큰·막힌 PENDING, 일별 성공/실패 추이KstDashboardDates로 중앙화GET /api/admin/dashboard/usage·/quality백오피스 UI (admin-web)
/dashboard페이지 — 사용량·품질 2섹션, recharts(감정 파이·활동 추이 이중축·성공/실패 스택바), 위험 신호 강조(막힌 PENDING·성공률 저하)🌐 API · DB 영향
GET /api/admin/dashboard/usage,GET /api/admin/dashboard/qualitygeneration_log테이블 신설모습
💬 리뷰 포인트
예상 후속
Summary by CodeRabbit
Summary by CodeRabbit
/dashboard로 정리했습니다.