Skip to content

Commit af869a7

Browse files
sravottosravotto
authored
cli: add command to validate S3 object storage (#5)
Introduce a new `s3` subcommand that performs validation tests against an S3-compatible object store. The command: - Initializes an S3 store from environment variables. - Runs validation checks via the `validate` package. - Outputs a formatted report to stdout. This provides users with an easy way to confirm connectivity and configuration of their S3 object storage directly from the CLI.
1 parent 6b377f9 commit af869a7

File tree

16 files changed

+663
-32
lines changed

16 files changed

+663
-32
lines changed

‎.github/docker-compose.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,9 @@ services:
3939
image: minio/minio
4040
healthcheck:
4141
test: ["CMD", "curl", "--fail", "http://localhost:29001/"]
42-
interval: 30s
43-
timeout: 10s
44-
retries: 3
42+
interval: 2s
43+
timeout: 2s
44+
retries: 10
4545
start_period: 5s
4646
command: server --console-address ":29001" --address ":29000" /data
4747
network_mode: host

‎README.md‎

Lines changed: 111 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,111 @@
1-
# blobcheck
1+
# blobcheck
2+
3+
**blobcheck** is a diagnostic tool for validating object storage connectivity and integration with CockroachDB backup/restore workflows. It verifies that the storage provider is correctly configured, runs synthetic workloads, and produces network performance statistics.
4+
5+
---
6+
7+
## Usage
8+
9+
```bash
10+
blobcheck s3 [flags]
11+
```
12+
13+
### Flags
14+
15+
```
16+
-h, --help help for s3
17+
```
18+
19+
### Global Flags
20+
21+
```
22+
--db string PostgreSQL connection URL
23+
(default "postgresql://root@localhost:26257?sslmode=disable")
24+
--endpoint string http endpoint, if uri is not specified
25+
--path string destination path (e.g. bucket/folder), if uri is not specified
26+
--uri string in the [scheme]://[host]/[path]?[parameters] format
27+
--verbose increase logging verbosity to debug
28+
```
29+
30+
### Credentials
31+
32+
Credentials must be provided in one of the locations supported by `config.LoadDefaultConfig`.
33+
For example, they can be exported before running:
34+
35+
```bash
36+
export AWS_ACCESS_KEY_ID=..
37+
export AWS_SECRET_ACCESS_KEY=..
38+
```
39+
40+
---
41+
42+
## Examples
43+
44+
### Using endpoint and path
45+
46+
```bash
47+
blobcheck s3 --endpoint http://provider:9000 --path mybucket/cluster1_backup
48+
```
49+
50+
### Using full URI
51+
52+
```bash
53+
blobcheck s3 --uri 's3://mybucket/cluster1_backup?AWS_ACCESS_KEY_ID=..&AWS_SECRET_ACCESS_KEY=..&AWS_ENDPOINT=http://provider:9000'
54+
```
55+
56+
### Sample Output
57+
58+
```text
59+
┌────────────────────────────────────────────────┐
60+
│ Suggested Parameters │
61+
├───────────────────────┬────────────────────────┤
62+
│ parameter │ value │
63+
├───────────────────────┼────────────────────────┤
64+
│ AWS_ACCESS_KEY_ID │ AKIA... │
65+
│ AWS_ENDPOINT │ https://s3.example.com │
66+
│ AWS_REGION │ us-west-2 │
67+
│ AWS_SECRET_ACCESS_KEY │ ****** │
68+
│ AWS_SKIP_CHECKSUM │ true │
69+
└───────────────────────┴────────────────────────┘
70+
┌──────────────────────────────────────────┐
71+
│ Statistics │
72+
├──────┬────────────┬─────────────┬────────┤
73+
│ node │ read speed │ write speed │ status │
74+
├──────┼────────────┼─────────────┼────────┤
75+
│ 1 │ 100MB/s │ 50MB/s │ OK │
76+
│ 2 │ 200MB/s │ 100MB/s │ OK │
77+
└──────┴────────────┴─────────────┴────────┘
78+
```
79+
80+
---
81+
82+
## High-Level Architecture
83+
84+
### Components
85+
86+
- **Validator (`internal/validate`)**
87+
The central orchestrator for validation. Responsible for:
88+
- Database and table creation (source and restored)
89+
- Running synthetic workloads
90+
- Initiating full and incremental backups
91+
- Restoring from backups
92+
- Comparing original and restored table fingerprints for integrity verification
93+
94+
- **Database Layer (`internal/db`)**
95+
- Manages creation, dropping, and schema definition for test databases/tables
96+
- Handles external connections to the object store
97+
98+
- **Blob Storage Layer (`internal/blob`)**
99+
- Abstracts interactions with the S3 provider
100+
- Executes backup/restore commands
101+
- Performs quick tests directly on the S3 storage (put/get/list)
102+
103+
- **Workload Generator (`internal/workload`)**
104+
- Populates the source table with synthetic data during tests
105+
- Simulates table activity between backups to ensure incremental backups are meaningful
106+
107+
---
108+
109+
## License
110+
111+
This project is licensed under the Apache 2.0 License. See [LICENSE](LICENSE.txt) for details.

‎cmd/root.go‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
// Copyright 2025 Cockroach Labs, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package cmd
16+
17+
import (
18+
"errors"
19+
"fmt"
20+
"log/slog"
21+
"os"
22+
23+
"github.com/spf13/cobra"
24+
25+
"github.com/cockroachlabs-field/blobcheck/cmd/s3"
26+
"github.com/cockroachlabs-field/blobcheck/internal/env"
27+
)
28+
29+
var verbosity int
30+
var envConfig = &env.Env{
31+
DatabaseURL: "postgresql://root@localhost:26257?sslmode=disable",
32+
LookupEnv: os.LookupEnv,
33+
}
34+
35+
// rootCmd represents the base command when called without any subcommands
36+
var rootCmd = &cobra.Command{
37+
Use: "blobcheck",
38+
Short: "blobcheck validates backup/restore operation against blob storage",
39+
Long: `blobcheck is a diagnostic tool for validating object storage connectivity
40+
and integration with CockroachDB backup/restore workflows.
41+
It verifies that the storage provider is correctly configured,
42+
runs synthetic workloads, and produces network performance statistics.`,
43+
PersistentPreRunE: func(_ *cobra.Command, _ []string) error {
44+
if envConfig.DatabaseURL == "" {
45+
return errors.New("database URL cannot be blank")
46+
}
47+
if envConfig.URI != "" {
48+
if envConfig.Endpoint != "" || envConfig.Path != "" {
49+
return errors.New("URI and (endpoint + path) cannot be set simultaneously")
50+
}
51+
} else {
52+
if envConfig.Endpoint == "" {
53+
return errors.New("set (endpoint + path) or URI")
54+
}
55+
if envConfig.Path == "" {
56+
return errors.New("set (endpoint + path) or URI")
57+
}
58+
}
59+
if verbosity > 0 {
60+
slog.SetLogLoggerLevel(slog.LevelDebug)
61+
}
62+
if verbosity > 1 {
63+
envConfig.Verbose = true
64+
}
65+
return nil
66+
},
67+
}
68+
69+
// Execute runs the root command.
70+
func Execute() {
71+
s3.Add(envConfig, rootCmd)
72+
f := rootCmd.PersistentFlags()
73+
f.StringVar(&envConfig.DatabaseURL, "db", envConfig.DatabaseURL, "PostgreSQL connection URL")
74+
f.StringVar(&envConfig.Path, "path", envConfig.Path, "destination path (e.g. bucket/folder)")
75+
f.StringVar(&envConfig.Endpoint, "endpoint", envConfig.Path, "http endpoint")
76+
f.StringVar(&envConfig.URI, "uri", envConfig.URI, "S3 URI")
77+
f.CountVarP(&verbosity, "verbosity", "v", "increase logging verbosity to debug")
78+
err := rootCmd.Execute()
79+
80+
if err != nil {
81+
fmt.Println(err)
82+
os.Exit(1)
83+
}
84+
}

‎cmd/s3/s3.go‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
// Copyright 2025 Cockroach Labs, Inc.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License");
4+
// you may not use this file except in compliance with the License.
5+
// You may obtain a copy of the License at
6+
//
7+
// http://www.apache.org/licenses/LICENSE-2.0
8+
//
9+
// Unless required by applicable law or agreed to in writing, software
10+
// distributed under the License is distributed on an "AS IS" BASIS,
11+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
// See the License for the specific language governing permissions and
13+
// limitations under the License.
14+
15+
package s3
16+
17+
import (
18+
"github.com/spf13/cobra"
19+
20+
"github.com/cockroachdb/field-eng-powertools/stopper"
21+
"github.com/cockroachlabs-field/blobcheck/internal/blob"
22+
"github.com/cockroachlabs-field/blobcheck/internal/env"
23+
"github.com/cockroachlabs-field/blobcheck/internal/format"
24+
"github.com/cockroachlabs-field/blobcheck/internal/validate"
25+
)
26+
27+
func command(env *env.Env) *cobra.Command {
28+
cmd := &cobra.Command{
29+
Use: "s3",
30+
Short: "Performs a validation test for a s3 object store",
31+
RunE: func(cmd *cobra.Command, args []string) error {
32+
ctx := stopper.WithContext(cmd.Context())
33+
store, err := blob.S3FromEnv(ctx, env)
34+
if err != nil {
35+
return err
36+
}
37+
validator, err := validate.New(ctx, env, store)
38+
if err != nil {
39+
return err
40+
}
41+
defer validator.Clean(ctx)
42+
report, err := validator.Validate(ctx)
43+
if err != nil {
44+
return err
45+
}
46+
format.Report(cmd.OutOrStdout(), report)
47+
return nil
48+
},
49+
}
50+
return cmd
51+
}
52+
53+
// Add the command.
54+
func Add(env *env.Env, parent *cobra.Command) {
55+
cmd := command(env)
56+
parent.AddCommand(cmd)
57+
}

‎go.mod‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,16 +42,20 @@ require (
4242
github.com/go-ini/ini v1.67.0 // indirect
4343
github.com/goccy/go-json v0.10.5 // indirect
4444
github.com/gogo/protobuf v1.3.2 // indirect
45+
github.com/inconshreveable/mousetrap v1.1.0 // indirect
4546
github.com/klauspost/compress v1.18.0 // indirect
4647
github.com/klauspost/cpuid/v2 v2.2.11 // indirect
4748
github.com/kr/text v0.2.0 // indirect
49+
github.com/mattn/go-runewidth v0.0.16 // indirect
4850
github.com/minio/crc64nvme v1.0.2 // indirect
4951
github.com/minio/md5-simd v1.1.2 // indirect
5052
github.com/philhofer/fwd v1.2.0 // indirect
5153
github.com/pkg/errors v0.9.1 // indirect
5254
github.com/pmezard/go-difflib v1.0.0 // indirect
55+
github.com/rivo/uniseg v0.4.7 // indirect
5356
github.com/rogpeppe/go-internal v1.13.1 // indirect
5457
github.com/rs/xid v1.6.0 // indirect
58+
github.com/spf13/pflag v1.0.6 // indirect
5559
github.com/tinylib/msgp v1.3.0 // indirect
5660
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 // indirect
5761
golang.org/x/mod v0.27.0 // indirect
@@ -72,7 +76,9 @@ require (
7276
github.com/jackc/pgpassfile v1.0.0 // indirect
7377
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
7478
github.com/jackc/puddle/v2 v2.2.2 // indirect
79+
github.com/jedib0t/go-pretty/v6 v6.6.8
7580
github.com/minio/minio-go/v7 v7.0.95
81+
github.com/spf13/cobra v1.9.1
7682
github.com/stretchr/testify v1.10.0
7783
golang.org/x/crypto v0.40.0 // indirect
7884
golang.org/x/sync v0.16.0 // indirect

‎go.sum‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b h1:r6VH0faHjZe
5050
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b/go.mod h1:Vz9DsVWQQhf3vs21MhPMZpMGSht7O/2vFW2xusFUVOs=
5151
github.com/cockroachdb/redact v1.1.5 h1:u1PMllDkdFfPWaNGMyLD1+so+aq3uUItthCFqzwPJ30=
5252
github.com/cockroachdb/redact v1.1.5/go.mod h1:BVNblN9mBWFyMyqK1k3AAiSxhvhfK2oOZZ2lK+dpvRg=
53+
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
5354
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
5455
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
5556
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
@@ -72,6 +73,8 @@ github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
7273
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
7374
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
7475
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
76+
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
77+
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
7578
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
7679
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
7780
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
@@ -80,6 +83,8 @@ github.com/jackc/pgx/v5 v5.7.5 h1:JHGfMnQY+IEtGM63d+NGMjoRpysB2JBwDr5fsngwmJs=
8083
github.com/jackc/pgx/v5 v5.7.5/go.mod h1:aruU7o91Tc2q2cFp5h4uP3f6ztExVpyVv88Xl/8Vl8M=
8184
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
8285
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
86+
github.com/jedib0t/go-pretty/v6 v6.6.8 h1:JnnzQeRz2bACBobIaa/r+nqjvws4yEhcmaZ4n1QzsEc=
87+
github.com/jedib0t/go-pretty/v6 v6.6.8/go.mod h1:YwC5CE4fJ1HFUDeivSV1r//AmANFHyqczZk+U6BDALU=
8388
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
8489
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
8590
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
@@ -91,6 +96,8 @@ github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
9196
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
9297
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
9398
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
99+
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
100+
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
94101
github.com/minio/crc64nvme v1.0.2 h1:6uO1UxGAD+kwqWWp7mBFsi5gAse66C4NXO8cmcVculg=
95102
github.com/minio/crc64nvme v1.0.2/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
96103
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
@@ -106,11 +113,19 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
106113
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
107114
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
108115
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
116+
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
117+
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
118+
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
109119
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
110120
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
111121
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
112122
github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
113123
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
124+
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
125+
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
126+
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
127+
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
128+
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
114129
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
115130
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
116131
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=

0 commit comments

Comments
 (0)