Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google secret manager store implementation #1034

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
4cc1c19
Google secret manager store implementation
shirkevich Feb 6, 2025
dfab44d
Fix example to make it consistent with AWS
shirkevich Feb 6, 2025
6e490f9
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 6, 2025
7088c15
Fixing coderabbitai suggestions
shirkevich Feb 10, 2025
21051e7
Merge branch 'main' into google-secret-manager
shirkevich Feb 10, 2025
5c5213c
Ordering stores, removed redundant examples
shirkevich Feb 11, 2025
a594edb
Merge branch 'main' into google-secret-manager
shirkevich Feb 11, 2025
65e9ab8
Fixing documentation to properly name delimiter param
shirkevich Feb 11, 2025
afebd17
Merge branch 'main' into google-secret-manager
osterman Feb 12, 2025
e605636
Merge branch 'main' into google-secret-manager
shirkevich Feb 13, 2025
2c61183
Merge branch 'main' into google-secret-manager
mcalhoun Feb 14, 2025
dd49233
[autofix.ci] apply automated fixes
autofix-ci[bot] Feb 14, 2025
b5a966c
Removing unused import
shirkevich Feb 14, 2025
0dc538c
Merge branch 'main' into google-secret-manager
mcalhoun Feb 17, 2025
daa8b1d
Merge branch 'main' into google-secret-manager
mcalhoun Feb 19, 2025
c355610
Merge branch 'main' into google-secret-manager
osterman Feb 19, 2025
33ae030
Do not require google application credentials for test
shirkevich Feb 20, 2025
297c229
Merge branch 'main' into google-secret-manager
shirkevich Feb 20, 2025
c545b86
Properly testing stack delimiter
shirkevich Feb 20, 2025
c21ac6c
Merge branch 'main' into google-secret-manager
osterman Feb 20, 2025
4a13579
Merge branch 'main' into google-secret-manager
osterman Feb 20, 2025
bc0f0bb
Fixing test to work without google credentials finally
shirkevich Feb 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions go.mod

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

248 changes: 248 additions & 0 deletions pkg/store/google_secret_manager_store.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,248 @@
package store

import (
"context"
"fmt"
"strings"
"time"

secretmanager "cloud.google.com/go/secretmanager/apiv1"
secretmanagerpb "cloud.google.com/go/secretmanager/apiv1/secretmanagerpb"
"github.com/charmbracelet/log"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Import log package with alias.

The log package should be imported with an alias to avoid confusion with the standard log package.

Apply this diff:

-	"github.com/charmbracelet/log"
+	clog "github.com/charmbracelet/log"

Committable suggestion skipped: line range outside the PR's diff.

🧰 Tools
🪛 golangci-lint (1.62.2)

11-11: import "github.com/charmbracelet/log" imported without alias but must be with alias "log" according to config

(importas)

"github.com/googleapis/gax-go/v2"
"google.golang.org/api/option"
)

const (
gsmOperationTimeout = 30 * time.Second
)

// GSMClient is the interface that wraps the Google Secret Manager client methods we use
type GSMClient interface {
CreateSecret(ctx context.Context, req *secretmanagerpb.CreateSecretRequest, opts ...gax.CallOption) (*secretmanagerpb.Secret, error)
AddSecretVersion(ctx context.Context, req *secretmanagerpb.AddSecretVersionRequest, opts ...gax.CallOption) (*secretmanagerpb.SecretVersion, error)
AccessSecretVersion(ctx context.Context, req *secretmanagerpb.AccessSecretVersionRequest, opts ...gax.CallOption) (*secretmanagerpb.AccessSecretVersionResponse, error)
Close() error
}

// GSMStore is an implementation of the Store interface for Google Secret Manager.
type GSMStore struct {
client GSMClient
projectID string
prefix string
stackDelimiter *string
}

type GSMStoreOptions struct {
Prefix *string `mapstructure:"prefix"`
ProjectID string `mapstructure:"project_id"`
StackDelimiter *string `mapstructure:"stack_delimiter"`
Credentials *string `mapstructure:"credentials"` // Optional JSON credentials
}

// Ensure GSMStore implements the store.Store interface.
var _ Store = (*GSMStore)(nil)

// NewGSMStore initializes a new Google Secret Manager Store.
func NewGSMStore(options GSMStoreOptions) (Store, error) {
ctx := context.Background()

if options.ProjectID == "" {
return nil, fmt.Errorf("project_id is required in Google Secret Manager store configuration")

Check failure on line 51 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"project_id is required in Google Secret Manager store configuration\")" (err113) Raw Output: pkg/store/google_secret_manager_store.go:51:15: do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"project_id is required in Google Secret Manager store configuration\")" (err113) return nil, fmt.Errorf("project_id is required in Google Secret Manager store configuration") ^
}

var clientOpts []option.ClientOption
if options.Credentials != nil {
clientOpts = append(clientOpts, option.WithCredentialsJSON([]byte(*options.Credentials)))
}

client, err := secretmanager.NewClient(ctx, clientOpts...)
if err != nil {
// Close the client to prevent resource leaks
if client != nil {
client.Close()
}
return nil, fmt.Errorf("failed to create Secret Manager client: %w", err)
}

store := &GSMStore{
client: client,
projectID: options.ProjectID,
}

if options.Prefix != nil {
store.prefix = *options.Prefix
}

if options.StackDelimiter != nil {
store.stackDelimiter = options.StackDelimiter
} else {
defaultDelimiter := "-"
store.stackDelimiter = &defaultDelimiter
}

return store, nil
}

func (s *GSMStore) getKey(stack string, component string, key string) (string, error) {
if s.stackDelimiter == nil {
return "", fmt.Errorf("stack delimiter is not set")

Check failure on line 89 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"stack delimiter is not set\")" (err113) Raw Output: pkg/store/google_secret_manager_store.go:89:14: do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"stack delimiter is not set\")" (err113) return "", fmt.Errorf("stack delimiter is not set") ^
}

// Get the base key using the common getKey function
baseKey, err := getKey(s.prefix, *s.stackDelimiter, stack, component, key, "_")
if err != nil {
return "", err
}

// Replace any remaining slashes with underscores as Secret Manager doesn't allow slashes
baseKey = strings.ReplaceAll(baseKey, "/", "_")
// Remove any double underscores that might have been created
baseKey = strings.ReplaceAll(baseKey, "__", "_")
// Trim any leading or trailing underscores
baseKey = strings.Trim(baseKey, "_")

Check failure on line 103 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 add-constant: string literal "_" appears, at least, 4 times, create a named constant for it (revive) Raw Output: pkg/store/google_secret_manager_store.go:103:34: add-constant: string literal "_" appears, at least, 4 times, create a named constant for it (revive) baseKey = strings.Trim(baseKey, "_") ^

return baseKey, nil
}

// Set stores a key-value pair in Google Secret Manager.
func (s *GSMStore) Set(stack string, component string, key string, value interface{}) error {

Check failure on line 109 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 function-length: maximum number of lines per function exceeded; max 60 but got 90 (revive) Raw Output: pkg/store/google_secret_manager_store.go:109:1: function-length: maximum number of lines per function exceeded; max 60 but got 90 (revive) func (s *GSMStore) Set(stack string, component string, key string, value interface{}) error { ctx, cancel := context.WithTimeout(context.Background(), gsmOperationTimeout) defer cancel() if stack == "" { return fmt.Errorf("stack cannot be empty") } if component == "" { return fmt.Errorf("component cannot be empty") } if key == "" { return fmt.Errorf("key cannot be empty") } // Convert value to string strValue, ok := value.(string) if !ok { return fmt.Errorf("value must be a string") } // Get the secret ID using getKey secretID, err := s.getKey(stack, component, key) if err != nil { return fmt.Errorf("failed to get key: %w", err) } // Create the secret if it doesn't exist parent := fmt.Sprintf("projects/%s", s.projectID) createSecretReq := &secretmanagerpb.CreateSecretRequest{ Parent: parent, SecretId: secretID, Secret: &secretmanagerpb.Secret{ Replication: &secretmanagerpb.Replication{ Replication: &secretmanagerpb.Replication_Automatic_{ Automatic: &secretmanagerpb.Replication_Automatic{}, }, }, }, } log.Debug("creating/updating Google Secret Manager secret", "project", s.projectID, "secret_id", secretID, "stack", stack, "component", component, "key", key) secret, err := s.client.CreateSecret(ctx, createSecretReq) if err != nil { // Ignore error if secret already exists if !strings.Contains(err.Error(), "already exists") { log.Debug("failed to create secret", "project", s.projectID, "secret_id", secretID, "error", err) return fmt.Errorf("failed to create secret: %w", err) } log.Debug("secret already exists", "project", s.projectID, "secret_id", secretID) // If the secret already exists, construct the name manually secret = &secretmanagerpb.Secret{ Name: fmt.Sprintf("projects/%s/secrets/%s", s.projectID, secretID), } } else { log.Debug("successfully created secret", "name", secret.GetName()) } // Add new version with the secret value addVersionReq := &secretmanagerpb.AddSecretVersionRequest{ Parent: secret.GetName(), Payload: &secretmanagerpb.SecretPayload{ Data: []byte(strValue), }, } log.Debug("adding new version to secret", "name", secret.GetName()) _, err = s.client.AddSecretVersion(ctx, addVersionReq) if err != nil { log.Debug("failed to add version to secret", "name", secret.GetName(), "error", err) return fmt.Errorf("failed to add secret version: %w", err) } log.Debug("successfully added new version to secret", "name", secret.GetName()) return nil }
ctx, cancel := context.WithTimeout(context.Background(), gsmOperationTimeout)
defer cancel()

if stack == "" {
return fmt.Errorf("stack cannot be empty")

Check failure on line 114 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"stack cannot be empty\")" (err113) Raw Output: pkg/store/google_secret_manager_store.go:114:10: do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"stack cannot be empty\")" (err113) return fmt.Errorf("stack cannot be empty") ^
}
if component == "" {
return fmt.Errorf("component cannot be empty")

Check failure on line 117 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"component cannot be empty\")" (err113) Raw Output: pkg/store/google_secret_manager_store.go:117:10: do not define dynamic errors, use wrapped static errors instead: "fmt.Errorf(\"component cannot be empty\")" (err113) return fmt.Errorf("component cannot be empty") ^
}
if key == "" {
return fmt.Errorf("key cannot be empty")
}

// Convert value to string
strValue, ok := value.(string)
if !ok {
return fmt.Errorf("value must be a string")
}

// Get the secret ID using getKey
secretID, err := s.getKey(stack, component, key)
if err != nil {
return fmt.Errorf("failed to get key: %w", err)
}

// Create the secret if it doesn't exist
parent := fmt.Sprintf("projects/%s", s.projectID)
createSecretReq := &secretmanagerpb.CreateSecretRequest{
Parent: parent,
SecretId: secretID,
Secret: &secretmanagerpb.Secret{
Replication: &secretmanagerpb.Replication{
Replication: &secretmanagerpb.Replication_Automatic_{
Automatic: &secretmanagerpb.Replication_Automatic{},
},
},
},
}

log.Debug("creating/updating Google Secret Manager secret",
"project", s.projectID,
"secret_id", secretID,
"stack", stack,
"component", component,
"key", key)

secret, err := s.client.CreateSecret(ctx, createSecretReq)
if err != nil {
// Ignore error if secret already exists
if !strings.Contains(err.Error(), "already exists") {
log.Debug("failed to create secret",
"project", s.projectID,
"secret_id", secretID,
"error", err)
return fmt.Errorf("failed to create secret: %w", err)
}
log.Debug("secret already exists",
"project", s.projectID,
"secret_id", secretID)
// If the secret already exists, construct the name manually
secret = &secretmanagerpb.Secret{
Name: fmt.Sprintf("projects/%s/secrets/%s", s.projectID, secretID),
}
} else {
log.Debug("successfully created secret",
"name", secret.GetName())
}

// Add new version with the secret value
addVersionReq := &secretmanagerpb.AddSecretVersionRequest{
Parent: secret.GetName(),
Payload: &secretmanagerpb.SecretPayload{
Data: []byte(strValue),
},
}

log.Debug("adding new version to secret",
"name", secret.GetName())

_, err = s.client.AddSecretVersion(ctx, addVersionReq)
if err != nil {
log.Debug("failed to add version to secret",
"name", secret.GetName(),
"error", err)
return fmt.Errorf("failed to add secret version: %w", err)
}

log.Debug("successfully added new version to secret",
"name", secret.GetName())

Check failure on line 198 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 add-constant: string literal "name" appears, at least, 4 times, create a named constant for it (revive) Raw Output: pkg/store/google_secret_manager_store.go:198:3: add-constant: string literal "name" appears, at least, 4 times, create a named constant for it (revive) "name", secret.GetName()) ^
return nil
}

// Get retrieves a value by key from Google Secret Manager.
func (s *GSMStore) Get(stack string, component string, key string) (interface{}, error) {
ctx, cancel := context.WithTimeout(context.Background(), gsmOperationTimeout)
defer cancel()

if stack == "" {
return nil, fmt.Errorf("stack cannot be empty")
}
if component == "" {
return nil, fmt.Errorf("component cannot be empty")
}
if key == "" {
return nil, fmt.Errorf("key cannot be empty")
}

// Get the secret ID using getKey
secretID, err := s.getKey(stack, component, key)
if err != nil {
return nil, fmt.Errorf("failed to get key: %w", err)
}

// Build the resource name for the latest version
name := fmt.Sprintf("projects/%s/secrets/%s/versions/latest", s.projectID, secretID)

log.Debug("accessing Google Secret Manager secret",
"name", name,
"project", s.projectID,

Check failure on line 228 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 add-constant: string literal "project" appears, at least, 4 times, create a named constant for it (revive) Raw Output: pkg/store/google_secret_manager_store.go:228:3: add-constant: string literal "project" appears, at least, 4 times, create a named constant for it (revive) "project", s.projectID, ^
"secret_id", secretID,

Check failure on line 229 in pkg/store/google_secret_manager_store.go

View workflow job for this annotation

GitHub Actions / [lint] golangci

[golangci] reported by reviewdog 🐶 add-constant: string literal "secret_id" appears, at least, 4 times, create a named constant for it (revive) Raw Output: pkg/store/google_secret_manager_store.go:229:3: add-constant: string literal "secret_id" appears, at least, 4 times, create a named constant for it (revive) "secret_id", secretID, ^
"stack", stack,
"component", component,
"key", key)

// Access the secret version
result, err := s.client.AccessSecretVersion(ctx, &secretmanagerpb.AccessSecretVersionRequest{
Name: name,
})
if err != nil {
log.Debug("failed to access secret",
"name", name,
"error", err)
return nil, fmt.Errorf("failed to access secret version: %w", err)
}

log.Debug("successfully accessed secret",
"name", name)
return string(result.Payload.Data), nil
}
Loading
Loading