feat: Kubernetes log export ingestion

[leo-aa88]

Implements #3: first-class ingestion of Kubernetes log exports through the existing SourceAdapter pipeline (no live cluster API).

Supported inputs

• Concatenated kubectl logs --prefix --timestamps output
• Fluent Bit / Vector JSON lines with a nested kubernetes object
• CRI (kubelet / containerd) node logs: TIMESTAMP stdout|stderr F|P message
• .gz files and tarballs (.tar, .tar.gz, .tgz, …)
• Kubelet path conventions: /var/log/pods/<ns>_<pod>_<uid>/<container>/<n>.log (and .log.gz)

Mapping

K8s metadata	raglogs field
labels.app / app.kubernetes.io/name / container_name	service
namespace	environment
pod	host

Path-inferred identity is a fallback on RawLogLine; CLI --service / --env still win over adapter defaults. Parsed line fields win over both.

Usage

kubectl logs -n production -l app=billing-worker --all-containers \
  --prefix --timestamps --since=1h > /tmp/billing-export.log
raglogs ingest --adapter k8s /tmp/billing-export.log

raglogs ingest --adapter k8s --recursive ./var/log/pods
raglogs ingest --adapter k8s ./node-logs.tar.gz

API: POST /ingestions with "adapter": "k8s" and paths.

Review follow-up

• LEVEL_SERVICE_PATTERN runs only on CRI-unwrapped inner messages.
• Gzipped kubelet paths (…/0.log.gz) infer namespace/pod/container.
• CRI P/F fragments are concatenated in the k8s adapter read() path.

Merge conflicts with sibling adapter PRs

#31 (Loki), #32 (Datadog), and this PR all touched registry / ingest CLI / API / README / the same unit tests. This branch now merges both of those heads and keeps all five adapters (file, cloudwatch, datadog, loki, k8s) wired together so it is conflict-free against main and against those PRs.

CloudWatch env remains ADAPTER_CLOUDWATCH_REGION (as on main). Loki adds LOKI_*; Datadog adds DATADOG_*.

Tests

Verified: pytest tests/unit/ — 369 passed.

  

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[cursor]

Code review

Adapter + gzip/tar discovery look solid, and RawLogLine defaults (default_service / host / extra) are the right extension. Nested Fluent Bit / Vector kubernetes JSON aliases in core are fine — that is a field-alias problem, not an adapter branch.

Do not merge yet. k8s-specific parsing was pushed into shared parse_text_line, which every file ingest uses.

Blocking

1. LEVEL_SERVICE_PATTERN mis-parses ordinary text logs (src/core/parsing/text_parser.py). A line like ERROR failed to connect to database today keeps the full message and uses --service / filename. After this PR it becomes service="failed", message="to connect to database". Fingerprints, clustering, and explain all shift for non-k8s file ingest. Apply that pattern only on the CRI recurse path (or inside the k8s adapter), and add a regression test.

CRI (TIMESTAMP stdout|stderr F|P) and kubectl [pod/container] prefixes are specific enough to live in core if you want; the untimestamped LEVEL service message rule is not.

Suggestions

2. CRI P/F fragments are not reassembled — one kubelet error becomes two fingerprints.
3. Gzipped kubelet paths (.../0.log.gz) are read as gzip but miss PODS_PATH (\d+\.log$), so namespace/pod/container stay empty.

Merge note

#31, #32, and #33 all touch registry.py, ingest CLI/API, README, and the same unit tests. Land one and rebase the others. Suggested order: #32, then #31, then this PR after the parser fix.

[cursor]

Blocking: this runs for every text log, not just k8s.

ERROR failed to connect to database currently keeps the full message and uses --service / filename. After this match it becomes service="failed", message="to connect to database" — fingerprints and clustering change for ordinary file ingest.

Keep LEVEL_SERVICE_PATTERN on the CRI recurse path (or in the k8s adapter) only. Add a regression test that parse_text_line("ERROR failed to connect to database", default_service="api").service == "api" and that the message is unchanged.

[cursor]

Suggestion: .../0.log.gz is ingested as gzip (_is_gzip_log) but this regex requires .log$, so default_service / namespace / pod stay empty. Strip a trailing .gz before inferring metadata, or allow .log.gz here.

[cursor]

Suggestion: kubelet splits long lines into P (partial) then F (full). Each fragment becomes its own LogEntry, so one error becomes two fingerprints. If node-log dumps are a real input, concatenate P…F in the k8s reader before parsing.

[cursor]

Re-review (follow-up)

Previous blockers and the two suggestions are fixed. CI is green.

• LEVEL_SERVICE_PATTERN is gated on cri_unwrapped=True, and test_untimestamped_error_does_not_steal_service_token covers ERROR failed to connect to database.
• CRI P/F fragments are joined in _reassemble_cri_fragments.
• _strip_gzip_log_suffix infers kubelet metadata from 0.log.gz.

No remaining blockers. Fine to merge after rebasing onto whatever lands first among #31/#32.

Suggestions (non-blocking)

1. CLI vs path-default precedence. Ingestion uses raw.default_service or spec.service, so a kubelet path wins over --service. Add Loki source adapter for bounded-window ingest #31 does the opposite (spec.service or raw.default_service). Prefer CLI/API --service/--env as the override when you reconcile RawLogLine + _process_line with Loki — both PRs extend those types and will conflict.

2. KUBECTL_PREFIX_PATTERN still runs on every text line ([app/worker] … can look like kubectl). Acceptable given issue feat: Kubernetes log export ingestion #3; just be aware it is not k8s-adapter-only.

CRI/kubectl handling in parse_text_line plus Fluent Bit aliases in extract_kubernetes_fields are in scope for #3. The CRI-inner gate keeps generic file ingest intact.