Skip to content

fix: migration 042 — purge 111K low-quality patterns #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
thebtf merged 1 commit into from
Mar 21, 2026
Merged

fix: migration 042 — purge 111K low-quality patterns #38

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions internal/db/gorm/migrations.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1415,6 +1415,24 @@ func runMigrations(db *gorm.DB, embeddingDims int) error {
return nil
},
},
// Migration 042: Purge low-quality patterns (frequency < 5).
// With 111K+ patterns accumulated from garbage SDK extraction, most are noise.
// MinFrequency threshold was raised to 5 in T019 — patterns below this are worthless.
{
ID: "042_purge_low_quality_patterns",
Migrate: func(tx *gorm.DB) error {
result := tx.Exec(`DELETE FROM patterns WHERE frequency < 5`)
if result.Error != nil {
log.Warn().Err(result.Error).Msg("migration 042: pattern purge failed (non-fatal)")
return nil
}
Comment on lines +1425 to +1428
Copy link

@coderabbitai coderabbitai bot Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Не скрывайте ошибку миграции на Line 1425—1428.

Если DELETE падает и функция всё равно возвращает nil, миграция может быть отмечена как выполненная без фактической очистки. Для one-time cleanup это ломает детерминизм состояния БД. Верните ошибку, чтобы миграция не считалась успешно применённой.

Предлагаемый фикс 
 		Migrate: func(tx *gorm.DB) error {
 			result := tx.Exec(`DELETE FROM patterns WHERE frequency < 5`)
 			if result.Error != nil {
-				log.Warn().Err(result.Error).Msg("migration 042: pattern purge failed (non-fatal)")
-				return nil
+				return fmt.Errorf("migration 042: purge low-quality patterns: %w", result.Error)
 			}
 			log.Info().Int64("patterns_deleted", result.RowsAffected).Msg("migration 042: low-quality pattern purge complete")
 			return nil
 		},
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if result.Error != nil {
log.Warn().Err(result.Error).Msg("migration 042: pattern purge failed (non-fatal)")
return nil
}
if result.Error != nil {
return fmt.Errorf("migration 042: purge low-quality patterns: %w", result.Error)
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@internal/db/gorm/migrations.go` around lines 1425 - 1428, В блоке проверки
результата миграции (где проверяется result.Error и логируется "migration 042:
pattern purge failed (non-fatal)") не подавляйте ошибку — вместо возвращения nil
верните саму ошибку; замените текущее поведение в этой ветке так, чтобы функция
возвращала result.Error (и при желании подняла уровень лога на error), чтобы
миграция не считалась успешно выполненной при падении DELETE.

log.Info().Int64("patterns_deleted", result.RowsAffected).Msg("migration 042: low-quality pattern purge complete")
return nil
},
Rollback: func(tx *gorm.DB) error {
return nil
},
},
})
if err := m.Migrate(); err != nil {
return fmt.Errorf("run gormigrate migrations: %w", err)
Expand Down
1 change: 1 addition & 0 deletions internal/worker/handlers_maintenance.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,3 +343,4 @@ func (s *Service) handlePatternCleanup(w http.ResponseWriter, r *http.Request) {
"deleted": deleted,
})
}

Loading