Skip to content

fix(clustered): clarify compactor scaling guidance for CPU and memory #6302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/pr-link-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ jobs:
curl -L -H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-o link-checker-info.json \
"https://api.github.com/repos/influxdata/docs-v2/releases/tags/link-checker-v1.0.0"
"https://api.github.com/repos/influxdata/docs-v2/releases/tags/link-checker-v1.2.2"

# Extract download URL for linux binary
DOWNLOAD_URL=$(jq -r '.assets[] | select(.name | test("link-checker.*linux")) | .url' link-checker-info.json)
Expand Down
68 changes: 68 additions & 0 deletions .github/workflows/sync-link-checker-binary.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Sync Link Checker Binary from docs-tooling

on:
workflow_dispatch:
inputs:
version:
description: 'Link checker version to sync (e.g., v1.2.2)'
required: true
type: string

jobs:
sync-binary:
name: Sync link-checker binary from docs-tooling
runs-on: ubuntu-latest

steps:
- name: Download binary from docs-tooling release
run: |
echo "Downloading link-checker ${{ inputs.version }} from docs-tooling..."

# Download binary from docs-tooling release
curl -L -H "Accept: application/octet-stream" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-o link-checker-linux-x86_64 \
"https://github.com/influxdata/docs-tooling/releases/download/link-checker-${{ inputs.version }}/link-checker-linux-x86_64"

# Download checksums
curl -L -H "Accept: application/octet-stream" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
-o checksums.txt \
"https://github.com/influxdata/docs-tooling/releases/download/link-checker-${{ inputs.version }}/checksums.txt"

# Verify downloads
ls -la link-checker-linux-x86_64 checksums.txt

- name: Create docs-v2 release
run: |
echo "Creating link-checker-${{ inputs.version }} release in docs-v2..."

gh release create \
--title "Link Checker Binary ${{ inputs.version }}" \
--notes "Link validation tooling binary for docs-v2 GitHub Actions workflows.

This binary is distributed from the docs-tooling repository release link-checker-${{ inputs.version }}.

### Usage in GitHub Actions

The binary is automatically downloaded by docs-v2 workflows for link validation.

### Manual Usage

\`\`\`bash
# Download and make executable
curl -L -o link-checker https://github.com/influxdata/docs-v2/releases/download/link-checker-${{ inputs.version }}/link-checker-linux-x86_64
chmod +x link-checker

# Verify installation
./link-checker --version
\`\`\`

### Changes in ${{ inputs.version }}

See the [docs-tooling release](https://github.com/influxdata/docs-tooling/releases/tag/link-checker-${{ inputs.version }}) for detailed changelog." \
link-checker-${{ inputs.version }} \
link-checker-linux-x86_64 \
checksums.txt
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
66 changes: 66 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,47 @@ cargo build --release
cp target/release/link-checker /usr/local/bin/
```

#### Binary Release Process

**For maintainers:** To create a new link-checker release in docs-v2:

1. **Create release in docs-tooling** (builds and releases binary automatically):
```bash
cd docs-tooling
git tag link-checker-v1.2.x
git push origin link-checker-v1.2.x
```

2. **Manually distribute to docs-v2** (required due to private repository access):
```bash
# Download binary from docs-tooling release
curl -L -H "Authorization: Bearer $(gh auth token)" \
-o link-checker-linux-x86_64 \
"https://github.com/influxdata/docs-tooling/releases/download/link-checker-v1.2.x/link-checker-linux-x86_64"

curl -L -H "Authorization: Bearer $(gh auth token)" \
-o checksums.txt \
"https://github.com/influxdata/docs-tooling/releases/download/link-checker-v1.2.x/checksums.txt"

# Create docs-v2 release
gh release create \
--repo influxdata/docs-v2 \
--title "Link Checker Binary v1.2.x" \
--notes "Link validation tooling binary for docs-v2 GitHub Actions workflows." \
link-checker-v1.2.x \
link-checker-linux-x86_64 \
checksums.txt
```

3. **Update workflow reference** (if needed):
```bash
# Update .github/workflows/pr-link-check.yml line 98 to use new version
sed -i 's/link-checker-v[0-9.]*/link-checker-v1.2.x/' .github/workflows/pr-link-check.yml
```

> [!Note]
> The manual distribution is required because docs-tooling is a private repository and the default GitHub token doesn't have cross-repository access for private repos.

#### Core Commands

```bash
Expand All @@ -184,6 +225,31 @@ link-checker check public/path/to/file.html
link-checker config
```

### Link Resolution Behavior

The link-checker automatically handles relative link resolution based on the input type:

**Local Files → Local Resolution**
```bash
# When checking local files, relative links resolve to the local filesystem
link-checker check public/influxdb3/core/admin/scale-cluster/index.html
# Relative link /influxdb3/clustered/tags/kubernetes/ becomes:
# → /path/to/public/influxdb3/clustered/tags/kubernetes/index.html
```

**URLs → Production Resolution**
```bash
# When checking URLs, relative links resolve to the production site
link-checker check https://docs.influxdata.com/influxdb3/core/admin/scale-cluster/
# Relative link /influxdb3/clustered/tags/kubernetes/ becomes:
# → https://docs.influxdata.com/influxdb3/clustered/tags/kubernetes/
```

**Why This Matters**
- **Testing new content**: Tag pages generated locally will be found when testing local files
- **Production validation**: Production URLs validate against the live site
- **No false positives**: New content won't appear broken when testing locally before deployment

### Content Mapping Workflows

#### Scenario 1: Map and check InfluxDB 3 Core content
Expand Down
11 changes: 8 additions & 3 deletions content/influxdb3/clustered/admin/scale-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ menu:
parent: Administer InfluxDB Clustered
name: Scale your cluster
weight: 207
influxdb3/clustered/tags: [scale]
influxdb3/clustered/tags: [scale, performance, Kubernetes]
related:
- /influxdb3/clustered/reference/internals/storage-engine/
- /influxdb3/clustered/write-data/best-practices/data-lifecycle/
- /influxdb3/clustered/query-data/troubleshoot-and-optimize/optimize-queries/
- https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits, Kubernetes resource requests and limits
---

Expand Down Expand Up @@ -559,11 +561,14 @@ concurrency demands or reaches the hardware limits of your underlying nodes.

### Compactor

- **Recommended**: Maintain **1 Compactor pod** and use [vertical scaling](#vertical-scaling) (especially
increasing the available CPU) for the Compactor.
- **Recommended**: Maintain **1 Compactor pod** and use [vertical scaling](#vertical-scaling) for the Compactor.
Scale CPU and memory resources together, as compactor concurrency settings scale based on memory, not CPU count.
- Because compaction is a compute-heavy process, horizontal scaling increases compaction throughput, but not as
efficiently as vertical scaling.

> [!Important]
> When scaling the Compactor, scale CPU and memory resources together.

### Garbage collector

The [Garbage collector](/influxdb3/clustered/reference/internals/storage-engine/#garbage-collector) is a lightweight process that typically doesn't require
Expand Down