PDF Processor #46
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| CREATE TYPE "public"."document_status" AS ENUM('queued', 'processing', 'ready', 'failed');--> statement-breakpoint | ||
| CREATE TABLE "document_chunks" ( | ||
| "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, | ||
| "document_id" uuid NOT NULL, | ||
| "path" varchar(1024) NOT NULL, | ||
| "description" text DEFAULT '' NOT NULL, | ||
| "content" text NOT NULL, | ||
| "sort_order" integer DEFAULT 0 NOT NULL, | ||
| "created_at" timestamp DEFAULT now() NOT NULL | ||
| ); | ||
| --> statement-breakpoint | ||
| CREATE TABLE "documents" ( | ||
| "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, | ||
| "title" varchar(512) NOT NULL, | ||
| "resource_name" varchar(512) NOT NULL, | ||
| "summary" text, | ||
| "gcs_pdf_path" varchar(1024) NOT NULL, | ||
| "status" "document_status" DEFAULT 'queued' NOT NULL, | ||
| "error_message" text, | ||
| "uploaded_by_idp_uuid" varchar(255) NOT NULL, | ||
| "is_active" boolean DEFAULT true NOT NULL, | ||
| "created_at" timestamp DEFAULT now() NOT NULL, | ||
| "updated_at" timestamp DEFAULT now() NOT NULL, | ||
| "processed_at" timestamp | ||
| ); | ||
| --> statement-breakpoint | ||
| ALTER TABLE "document_chunks" ADD CONSTRAINT "document_chunks_document_id_documents_id_fk" FOREIGN KEY ("document_id") REFERENCES "public"."documents"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint | ||
| CREATE INDEX "document_chunks_document_id_idx" ON "document_chunks" USING btree ("document_id");--> statement-breakpoint | ||
| CREATE INDEX "document_chunks_document_sort_idx" ON "document_chunks" USING btree ("document_id","sort_order");--> statement-breakpoint | ||
| CREATE UNIQUE INDEX "documents_resource_name_active_unique" ON "documents" USING btree ("resource_name") WHERE "documents"."is_active" = true;--> statement-breakpoint | ||
| CREATE INDEX "documents_status_idx" ON "documents" USING btree ("status");--> statement-breakpoint | ||
| CREATE INDEX "documents_uploaded_by_idp_uuid_idx" ON "documents" USING btree ("uploaded_by_idp_uuid");--> statement-breakpoint | ||
| CREATE INDEX "documents_is_active_idx" ON "documents" USING btree ("is_active");--> statement-breakpoint | ||
| CREATE INDEX "documents_created_at_idx" ON "documents" USING btree ("created_at"); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| ALTER TYPE "public"."document_status" ADD VALUE 'uploading' BEFORE 'queued';--> statement-breakpoint | ||
| ALTER TABLE "documents" ADD COLUMN "processing_token" uuid; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ALTER TABLE "documents" ADD COLUMN "last_reprocessed_at" timestamp; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| ALTER TABLE "documents" ADD COLUMN "expires_at" timestamp;--> statement-breakpoint | ||
| CREATE INDEX "documents_expires_at_idx" ON "documents" USING btree ("expires_at"); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| CREATE INDEX "document_chunks_path_idx" ON "document_chunks" USING btree ("path");--> statement-breakpoint | ||
| CREATE UNIQUE INDEX "document_chunks_document_id_path_unique" ON "document_chunks" USING btree ("document_id","path"); | ||
|
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# document_chunks 삽입 경로와 중복 처리 방식을 확인한다.
set -euo pipefail
rg -n --type=ts -C 6 'documentChunks' src | head -200Repository: gsainfoteam/chatbot-be Length of output: 12619 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== migration =="
cat -n drizzle/0013_aberrant_alex_wilder.sql
echo
echo "== schema relevant =="
sed -n '324,348p' src/db/schema.ts | cat -n
echo
echo "== document chunk insert/reprocess paths =="
sed -n '170,235p' src/pdf-processor/documents.repository.ts | cat -n
echo
echo "== organizations delete chunks path =="
sed -n '860,888p' src/organizations/organizations.repository.ts | cat -n
echo
echo "== all documentChunks insert/delete/upsert occurrences =="
rg -n --type=ts \
"insert\\(documentChunks\\)|delete\\(documentChunks\\)|sql`|onConflict|onConflictDoUpdate|replaceChunks|setShare|setDocumentStatus" srcRepository: gsainfoteam/chatbot-be Length of output: 4968 중복 현재 🤖 Prompt for AI Agents |
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| CREATE TYPE "public"."organization_membership_status" AS ENUM('PENDING', 'ACCEPTED');--> statement-breakpoint | ||
| CREATE TYPE "public"."organization_role" AS ENUM('MANAGER', 'MEMBER');--> statement-breakpoint | ||
| CREATE TABLE "document_organization_shares" ( | ||
| "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, | ||
| "document_id" uuid NOT NULL, | ||
| "organization_id" uuid NOT NULL, | ||
| "shared_by_idp_uuid" varchar(255) NOT NULL, | ||
| "created_at" timestamp DEFAULT now() NOT NULL | ||
| ); | ||
| --> statement-breakpoint | ||
| CREATE TABLE "document_ownership_transfers" ( | ||
| "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, | ||
| "document_id" uuid NOT NULL, | ||
| "source_organization_id" uuid NOT NULL, | ||
| "target_organization_id" uuid NOT NULL, | ||
| "actor_idp_uuid" varchar(255) NOT NULL, | ||
| "transferred_at" timestamp DEFAULT now() NOT NULL | ||
| ); | ||
| --> statement-breakpoint | ||
| CREATE TABLE "organization_memberships" ( | ||
| "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, | ||
| "organization_id" uuid NOT NULL, | ||
| "invitee_email" varchar(255) NOT NULL, | ||
| "member_idp_uuid" varchar(255), | ||
| "role" "organization_role" DEFAULT 'MEMBER' NOT NULL, | ||
| "status" "organization_membership_status" DEFAULT 'PENDING' NOT NULL, | ||
| "invited_by_idp_uuid" varchar(255) NOT NULL, | ||
| "accepted_at" timestamp, | ||
| "created_at" timestamp DEFAULT now() NOT NULL, | ||
| "updated_at" timestamp DEFAULT now() NOT NULL | ||
| ); | ||
| --> statement-breakpoint | ||
| CREATE TABLE "organizations" ( | ||
| "id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL, | ||
| "name" varchar(255) NOT NULL, | ||
| "slug" varchar(255) NOT NULL, | ||
| "is_default" boolean DEFAULT false NOT NULL, | ||
| "created_by_idp_uuid" varchar(255), | ||
| "created_at" timestamp DEFAULT now() NOT NULL, | ||
| "updated_at" timestamp DEFAULT now() NOT NULL, | ||
| CONSTRAINT "organizations_slug_unique" UNIQUE("slug") | ||
| ); | ||
| --> statement-breakpoint | ||
| INSERT INTO "organizations" ("name", "slug", "is_default", "created_by_idp_uuid") | ||
| VALUES ('인포팀', 'infoteam', true, NULL) | ||
| ON CONFLICT ("slug") DO UPDATE | ||
| SET "name" = EXCLUDED."name", | ||
| "is_default" = true, | ||
| "updated_at" = now();--> statement-breakpoint | ||
| INSERT INTO "organization_memberships" ( | ||
| "organization_id", | ||
| "invitee_email", | ||
| "member_idp_uuid", | ||
| "role", | ||
| "status", | ||
| "invited_by_idp_uuid", | ||
| "accepted_at" | ||
| ) | ||
| SELECT | ||
| o."id", | ||
| lower(trim(a."email")), | ||
| a."idp_uuid", | ||
| 'MANAGER'::"organization_role", | ||
| 'ACCEPTED'::"organization_membership_status", | ||
| a."idp_uuid", | ||
| now() | ||
| FROM "admins" a | ||
| JOIN "organizations" o ON o."slug" = 'infoteam' | ||
| WHERE a."role" = 'SUPER_ADMIN' | ||
| AND NOT EXISTS ( | ||
| SELECT 1 | ||
| FROM "organization_memberships" existing | ||
| WHERE existing."organization_id" = o."id" | ||
| AND ( | ||
| existing."invitee_email" = lower(trim(a."email")) | ||
| OR existing."member_idp_uuid" = a."idp_uuid" | ||
| ) | ||
| );--> statement-breakpoint | ||
| ALTER TABLE "documents" ADD COLUMN "owner_organization_id" uuid;--> statement-breakpoint | ||
| UPDATE "documents" | ||
| SET "owner_organization_id" = ( | ||
| SELECT "id" FROM "organizations" WHERE "slug" = 'infoteam' | ||
| ) | ||
| WHERE "owner_organization_id" IS NULL;--> statement-breakpoint | ||
| DO $$ | ||
| BEGIN | ||
| IF EXISTS (SELECT 1 FROM "documents" WHERE "owner_organization_id" IS NULL) THEN | ||
| RAISE EXCEPTION 'documents.owner_organization_id backfill failed'; | ||
| END IF; | ||
| END $$;--> statement-breakpoint | ||
| ALTER TABLE "documents" ALTER COLUMN "owner_organization_id" SET NOT NULL;--> statement-breakpoint | ||
| ALTER TABLE "document_organization_shares" ADD CONSTRAINT "document_organization_shares_document_id_documents_id_fk" FOREIGN KEY ("document_id") REFERENCES "public"."documents"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint | ||
| ALTER TABLE "document_organization_shares" ADD CONSTRAINT "document_organization_shares_organization_id_organizations_id_fk" FOREIGN KEY ("organization_id") REFERENCES "public"."organizations"("id") ON DELETE restrict ON UPDATE no action;--> statement-breakpoint | ||
| ALTER TABLE "document_ownership_transfers" ADD CONSTRAINT "document_ownership_transfers_document_id_documents_id_fk" FOREIGN KEY ("document_id") REFERENCES "public"."documents"("id") ON DELETE restrict ON UPDATE no action;--> statement-breakpoint | ||
|
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# documents 테이블의 하드 삭제 경로를 확인한다.
set -euo pipefail
rg -n --type=ts -C 6 'delete\(documents\)|from\(documents\)\s*\)|\.delete\(' src | head -200Repository: gsainfoteam/chatbot-be Length of output: 13940 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== migration constraints around document_ownership_transfers =="
sed -n '1,140p' drizzle/0014_unique_dracula.sql | nl -ba
echo
echo "== upload/delete controller and service outlines =="
fd -a 'upload\.(controller|service)\.ts$' src | sed 's#^\./##' | while read -r f; do
echo "--- $f"
ast-grep outline "$f" --match delete --view compact || true
done
echo
echo "== relevant source snippets =="
sed -n '340,410p' src/upload/upload.controller.ts | nl -ba
sed -n '1,220p' src/upload/upload.service.ts | nl -ba
echo
echo "== repository hardDelete usages =="
rg -n -C 4 'hardDelete\(' src | sed -n '1,220p'
echo
echo "== all delete(documents) occurrences =="
rg -n -C 4 'delete\(documents\)' src drizzle | sed -n '1,220p'Repository: gsainfoteam/chatbot-be Length of output: 264 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== migration constraints around document_ownership_transfers =="
sed -n '1,140p' drizzle/0014_unique_dracula.sql
echo
echo "== upload/delete controller snippets =="
sed -n '340,410p' src/upload/upload.controller.ts
echo
echo "== upload service snippets =="
sed -n '1,220p' src/upload/upload.service.ts
echo
echo "== hardDelete usages =="
rg -n -C 4 'hardDelete\(' src
echo
echo "== delete(documents) occurrences =="
rg -n -C 4 'delete\(documents\)' src drizzleRepository: gsainfoteam/chatbot-be Length of output: 19218 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== rollbackUpload references =="
rg -n -C 6 'rollbackUpload|rollback' src/upload/upload.service.ts
echo
echo "== DocumentsRepository source around chunk ownership and soft delete =="
sed -n '1,260p' src/pdf-processor/documents.repository.ts
echo
echo "== OrganizationsRepository around cancelAndSoftDeleteDocument =="
sed -n '820,920p' src/organizations/organizations.repository.ts
echo
echo "== ownership transfer insert context =="
rg -n -C 8 'documentOwnershipTransfers|ownershipTransfers|ownership transfer|transfer' src | sed -n '1,260p'Repository: gsainfoteam/chatbot-be Length of output: 29483
이 마이그레이션은 🤖 Prompt for AI Agents |
||
| ALTER TABLE "document_ownership_transfers" ADD CONSTRAINT "document_ownership_transfers_source_organization_id_organizations_id_fk" FOREIGN KEY ("source_organization_id") REFERENCES "public"."organizations"("id") ON DELETE restrict ON UPDATE no action;--> statement-breakpoint | ||
| ALTER TABLE "document_ownership_transfers" ADD CONSTRAINT "document_ownership_transfers_target_organization_id_organizations_id_fk" FOREIGN KEY ("target_organization_id") REFERENCES "public"."organizations"("id") ON DELETE restrict ON UPDATE no action;--> statement-breakpoint | ||
| ALTER TABLE "organization_memberships" ADD CONSTRAINT "organization_memberships_organization_id_organizations_id_fk" FOREIGN KEY ("organization_id") REFERENCES "public"."organizations"("id") ON DELETE restrict ON UPDATE no action;--> statement-breakpoint | ||
| CREATE INDEX "document_organization_shares_document_id_idx" ON "document_organization_shares" USING btree ("document_id");--> statement-breakpoint | ||
| CREATE INDEX "document_organization_shares_organization_id_idx" ON "document_organization_shares" USING btree ("organization_id");--> statement-breakpoint | ||
| CREATE UNIQUE INDEX "document_organization_shares_document_id_organization_id_unique" ON "document_organization_shares" USING btree ("document_id","organization_id");--> statement-breakpoint | ||
| CREATE INDEX "document_ownership_transfers_document_id_idx" ON "document_ownership_transfers" USING btree ("document_id");--> statement-breakpoint | ||
| CREATE INDEX "document_ownership_transfers_source_organization_id_idx" ON "document_ownership_transfers" USING btree ("source_organization_id");--> statement-breakpoint | ||
| CREATE INDEX "document_ownership_transfers_target_organization_id_idx" ON "document_ownership_transfers" USING btree ("target_organization_id");--> statement-breakpoint | ||
| CREATE INDEX "organization_memberships_organization_id_idx" ON "organization_memberships" USING btree ("organization_id");--> statement-breakpoint | ||
| CREATE INDEX "organization_memberships_member_idp_uuid_status_idx" ON "organization_memberships" USING btree ("member_idp_uuid","status");--> statement-breakpoint | ||
| CREATE INDEX "organization_memberships_invitee_email_status_idx" ON "organization_memberships" USING btree ("invitee_email","status");--> statement-breakpoint | ||
| CREATE UNIQUE INDEX "organization_memberships_organization_id_invitee_email_unique" ON "organization_memberships" USING btree ("organization_id","invitee_email") WHERE "organization_memberships"."status" = 'PENDING';--> statement-breakpoint | ||
| CREATE UNIQUE INDEX "organization_memberships_organization_id_member_idp_uuid_unique" ON "organization_memberships" USING btree ("organization_id","member_idp_uuid") WHERE "organization_memberships"."member_idp_uuid" IS NOT NULL;--> statement-breakpoint | ||
| CREATE UNIQUE INDEX "organizations_single_default_unique" ON "organizations" USING btree ("is_default") WHERE "organizations"."is_default" = true;--> statement-breakpoint | ||
| CREATE INDEX "organizations_created_by_idp_uuid_idx" ON "organizations" USING btree ("created_by_idp_uuid");--> statement-breakpoint | ||
| ALTER TABLE "documents" ADD CONSTRAINT "documents_owner_organization_id_organizations_id_fk" FOREIGN KEY ("owner_organization_id") REFERENCES "public"."organizations"("id") ON DELETE restrict ON UPDATE no action;--> statement-breakpoint | ||
| CREATE INDEX "documents_owner_organization_id_idx" ON "documents" USING btree ("owner_organization_id"); | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.