go-masker

go-masker is a Go library for fail-closed masking of sensitive data before it reaches logs, diagnostics, traces, or other observability systems. It plugs into log/slog, zap and zerolog without importing either third-party logger.

One MaskJSON call on a payload, with the default policy and no configuration:

Field	In	Out
password	hunter2	[REDACTED]
card	4111 1111 1111 1111	**** **** **** 1111
email	alice@example.com	a***@example.com
user_id	884213	**4213
amount	1499	1499

Each field got the treatment its name implies, amount was left alone, and nothing had to be listed by hand.

It provides one policy and rule model for:

• strings and scalar values;
• arbitrary nested Go values;
• JSON documents and readers;
• URLs, JSON bodies and forms carried inside string values;
• struct tags;
• HTTP headers and URLs through httpmask;
• log/slog attributes through slogmask;
• JSON log lines from zerolog or any JSON-line logger through zerologmask, and from zap's JSON encoder through zapmask.

The module has no third-party dependencies, not even in its tests, and does not depend on an HTTP framework or logging library.

Why a library instead of a field filter

A list of field names in a logger config covers the fields you remembered, at the depth you remembered them. This library is built around the cases that list misses:

• Errors never fall back to the input. Every operation returns a safe marker on failure. A filter that errors typically logs the raw value, which is the moment you needed it least.
• Depth is not special. The same policy applies to a nested JSON object, a struct field, a map inside a slice, a URL query parameter and an HTTP header.
• Hostile input stays bounded. Traversal depth, visited nodes and input size are capped, and the depth limit itself cannot exceed 10,000, so deeply nested input cannot exhaust the goroutine stack.
• Output does not drift. Masking a fixed corpus under Go 1.23 through 1.27 produces byte-identical results; the digests are recorded in PERFORMANCE.md.

It is not a substitute for not collecting the secret in the first place, and it cannot prove that a custom rule you wrote is safe.

Each of those claims is checked by the suite; see How it is tested.

Table of contents

• Why a library instead of a field filter
• Supported Go versions
• Installation
• Quick start
• What is masked
• Documents inside strings
• Secrets inside text
• Core concepts
• JSON
• Struct tags
• HTTP headers and URLs
• log/slog
• zerolog
• zap
• Errors and fail-closed behavior
• Limits and security
• How it is tested
• How this was built
• Performance
• Releases
• Documentation map
• Compatibility and stability
• Contributing
• License

Supported Go versions

The library requires Go 1.23 or newer and uses only the standard library. The CI workflow runs the tests, the race suite, the correctness matrix and a fuzz smoke pass on every supported minor release — 1.23.x, 1.24.x, 1.25.x, 1.26.x, 1.27.x — plus stable, so a new Go release is covered on the day it ships. A separate job runs govulncheck on every push.

Installation

go get github.com/icntswm/go-masker

Import the core package as masker:

import "github.com/icntswm/go-masker"

The adapters are separate packages in the same module:

import (
	"github.com/icntswm/go-masker/httpmask"    // HTTP headers and URLs
	"github.com/icntswm/go-masker/slogmask"    // log/slog attributes
	"github.com/icntswm/go-masker/zerologmask" // zerolog and other JSON-line loggers
	"github.com/icntswm/go-masker/zapmask"     // zap's JSON encoder
)

Quick start

package main

import (
	"fmt"

	"github.com/icntswm/go-masker"
)

func main() {
	m, err := masker.New(masker.DefaultPolicy())
	if err != nil {
		panic(err)
	}

	value, err := m.MaskValue("password", "correct-horse-battery-staple")
	if err != nil {
		panic(err)
	}
	fmt.Println(value)
	// Output: [REDACTED]
}

Masker instances are immutable and safe for concurrent use after successful construction. Operations copy input containers and never mutate the source.

What is masked

DefaultPolicy matches complete field names case-insensitively. The comparison also ignores the separator characters _, -, and ., so accessToken, access-token, and ACCESS.TOKEN all match the access_token binding. Its default bindings are:

Keys	Rule
password, passwd, passphrase, pwd	full
token, access_token, refresh_token, api_key, apikey, api_token, secret, client_secret, secret_key, secret_access_key, aws_secret_access_key, id_token, private_key, private_token, session_id, credentials, auth_token, otp	token
email, e-mail	email
phone, phone_number, mobile	phone
id, user_id, customer_id	ID
card, card_number, pan	card
authorization, cookie, set-cookie, x-api-key, x-api-token, x-access-token, x-auth-token, proxy-authorization, x-csrf-token, cvv, cvc	full

Built-in rules are available directly through PasswordRule, TokenRule, FullRule, EmailRule, PhoneRule, IDRule, and CardRule.

Partial rules preserve only a deliberately limited safe shape. Short or ambiguous phone/card values and values containing unexpected text are fully redacted.

Documents inside strings

A secret often reaches a log inside a value whose own name is harmless: a callback URL, a request body logged as a string, a form. When the policy leaves a string field alone, the masker looks at the value itself, and if the whole value is one of these documents, masks it by the keys inside it:

m.MaskValue("note", "https://u:pass@h/cb?token=abc#state")
// https://%5BREDACTED%5D@h/cb?token=%5BREDACTED%5D#%5BREDACTED%5D
m.MaskValue("note", `{"password":"secret","user":"alice"}`)
// {"password":"[REDACTED]","user":"alice"}
m.MaskValue("note", "user=alice&password=secret")
// password=%5BREDACTED%5D&user=alice

Recognition is strict, so prose is left alone:

• a URL is a single absolute scheme://host token without spaces; its userinfo and fragment are replaced as httpmask replaces them, and each query parameter is masked by its key;
• a JSON document is a string that is, apart from surrounding whitespace, one valid object or array; it is masked exactly as MaskJSON masks it;
• a form is key=value pairs joined by &, with no spaces, no ; and valid percent-encoding.

A value is rewritten only when something in it was masked; otherwise it comes back byte for byte. A document may hold another one, such as a JSON body with a redirect_uri, and is inspected to the same depth, node and input limits as the value around it. A string that looks like a URL but whose query does not parse becomes the marker. A field whose own key the policy masks or omits is decided by that key and is never inspected. WithoutEmbeddedDocuments() turns the inspection off.

Secrets inside text

A string that is not a whole document, such as a log message or an error text, is searched for secrets written into it. Only the secret is replaced; the rest of the text is kept byte for byte:

m.MaskValue("message", "login failed: password=hunter2")
// login failed: password=[REDACTED]
m.MaskValue("message", "upstream said: Bearer eyJhbGciOi... rejected")
// upstream said: Bearer [REDACTED] rejected
m.MaskValue("message", "dial postgres://app:pass@db:5432/app failed")
// dial postgres://[REDACTED]@db:5432/app failed

Two kinds of detectors run by default:

• key=value and key: value pairs, with an optionally quoted key or value; a key may start with _, - or ., so --password=... is a pair too. The policy judges the key as a field with Source SourceText, so the same rules that mask a password field mask password=... in a sentence. An omitted value becomes the marker, since text has no member to drop. After Authorization: the value takes the Bearer/Basic/Token credential with it.
• Secrets recognizable by shape, masked whatever key stands before them: the credential after Bearer or Basic, the body of a PEM private key (its BEGIN and END lines stay), a JWT, provider tokens with a documented prefix (ghp_, github_pat_, glpat-, xox?-, sk_live_, AIza, npm_, …) and the userinfo of a URL written inside a sentence.

Two detectors are opt-in, because ordinary text matches them by chance: WithCardNumberDetection() masks 13–19 digit numbers that pass the Luhn check with CardRule, keeping the last four digits, and WithAWSKeyIDDetection() masks AWS access key ids. WithoutTextDetectors() turns the text detectors off, and WithoutValueInspection() turns off both them and the documents above.

Detection is a heuristic on top of the policy, not a replacement for it: a secret with no key and no known shape, such as a bare random password, is not recognized. Text without a candidate costs no allocation.

Core concepts

Policies decide whether a field should be masked; rules transform the selected scalar value. A policy can be assembled from bindings or written as a function:

policy := masker.PolicyFunc(func(field masker.Field) (masker.Decision, error) {
	if field.Key == "tax_id" {
		return masker.Decision{Rule: masker.FullRule()}, nil
	}
	return masker.Decision{}, nil
})

m, err := masker.New(masker.Chain(masker.DefaultPolicy(), policy))

Policies in a Chain are evaluated in order. The first policy that gives an opinion wins. New rejects empty chains and chains containing nil policies.

A zero Decision means "no opinion". Decision{Omit: true} removes an object or map member entirely; array elements keep their position and become null, so the shape of a list is never altered.

Values are seen the way encoding/json would render them: a time.Time, net.IP, or any other encoding.TextMarshaler is masked as its text, and a []byte as its base64 form. A json.RawMessage, which encoding/json embeds as is, is decoded and masked by its keys like any other map; one that is not valid JSON fails closed. A MarshalText error or panic fails closed, and the method runs on a copy of the value, so it cannot change your data; a marshaler that holds pointers, maps, or locks is walked field by field instead.

NewKeyPolicy rejects empty keys (including keys that are made of separator characters only), nil rules, and fold-equivalent keys bound to different rules, so an ambiguous configuration fails at construction instead of resolving silently at run time.

Named custom rules make the rule name visible in diagnostics:

rule, err := masker.NewRule("tenant-id", func(input masker.RuleInput) (string, error) {
	return input.Redaction, nil
})

Partial custom rules must slice on runes, not bytes: a byte offset can split a multi-byte character, and the library rejects rule output that is not valid UTF-8.

Option is intentionally a closed type. Use the exported With* constructors rather than implementing options against the internal configuration type.

Reflection results are normalized for logging: safe scalars become strings unless WithPreserveSafeTypes keeps booleans, integers, unsigned integers, floats, and strings in their concrete types. Sensitive values are always strings, and a non-sensitive json.Number is always retained as json.Number to avoid precision loss.

Runnable examples for every exported constructor, option, rule, and adapter are in the package documentation. They are executed and output-checked by go test, so they cannot drift from the implementation.

JSON

MaskJSON accepts exactly one valid JSON document and returns valid JSON. It uses json.Number semantics for safe numbers, preserves last-wins behavior for duplicate object keys, and uses a streaming walker with depth, node, and input limits.

input := []byte(`{"user":"alice","password":"secret","balance":9007199254740993}`)
output, err := m.MaskJSON(input)
if err != nil {
	// output is a safe JSON root fallback, never the original input.
}
fmt.Println(string(output))
// {"balance":9007199254740993,"password":"[REDACTED]","user":"alice"}

Object keys are sorted in the output, and balance keeps its exact value. That number is 2^53+1, which float64 rounds down to 2^53; safe numbers are carried through as json.Number instead, so the digits survive.

MaskJSONReader reads the complete reader into memory before processing. Use WithMaxInputBytes to bound accepted input:

m, err := masker.New(masker.DefaultPolicy(), masker.WithMaxInputBytes(2<<20))
output, err := m.MaskJSONReader(reader)

There is intentionally no writer API for a single document: a writer cannot retract an unsafe prefix if a later parse error is found. zerologmask and zapmask are writers only for complete lines, each masked as its own document.

Struct tags

Struct tags select a built-in rule explicitly:

type Event struct {
	User     string `json:"user"`
	Password string `mask:"password"`
	Internal string `json:"-"`
}

masked, err := m.MaskAny(Event{
	User: "alice", Password: "secret", Internal: "not logged",
})

Supported tag values are full, email, phone, id, card, password, token, and omit. The precedence is:

1. omit;
2. full or another explicit tag rule;
3. the configured policy;
4. ordinary safe traversal.

The tag grammar is strict: an unsupported value or a comma-separated option is a configuration error rather than a silent fallback. The result is normalized to map[string]any/[]any containers. Struct field promotion follows the relevant encoding/json rules, including same-depth tagged-field selection and ignored unexported embedded non-struct fields.

When the masked value is going to be encoded anyway, MaskJSONValue returns the JSON directly. It gives the same bytes as json.Marshal of the MaskAny result, sorted keys included, at about half the cost, and on any error returns the marker as JSON:

line, err := m.MaskJSONValue(Event{User: "alice", Password: "secret"})
// {"Password":"[REDACTED]","user":"alice"}

A custom rule can join the grammar through WithTagRule. The name must not be empty, contain a comma, a space, or a quote, and built-in names and omit cannot be redefined:

lastTwo, err := masker.NewRule("last2", func(input masker.RuleInput) (string, error) {
	runes := []rune(input.Value)
	if len(runes) < 2 {
		return "**", nil
	}
	return string(runes[len(runes)-2:]), nil
})
m, err := masker.New(masker.DefaultPolicy(), masker.WithTagRule("last2", lastTwo))

HTTP headers and URLs

The httpmask adapter copies headers and URLs before applying the core policy:

core, err := masker.New(masker.DefaultPolicy())
adapter, err := httpmask.New(core)

headers, err := adapter.Headers(http.Header{
	"Authorization": {"Bearer secret"},
	"X-Trace":       {"trace-id"},
})
maskedURL, err := adapter.URLString(
	"https://user:password@example.com/?token=secret&keep=value#fragment",
)

Cookie and Set-Cookie values and URL userinfo are always fully redacted. Sensitive query parameters use the core policy. Query keys are sorted and URL escaping may be normalized. URL fragments are redacted by default, because an OAuth implicit-flow token arrives there; WithPreserveFragment keeps them.

log/slog

The slogmask adapter plugs the core policy into any slog handler through ReplaceAttr:

logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
	ReplaceAttr: slogmask.ReplaceAttr(m),
}))
logger.Info("login", "user", "alice", "password", "hunter2")
// {"time":"…","level":"INFO","msg":"login","user":"alice","password":"[REDACTED]"}

Attributes in groups are matched by their own key, with the group names in the policy path. The policy also decides each group as an object, so a group named like a secret, such as credentials, masks every member. Attributes added through logger.With and WithGroup, and the values returned by LogValue, are masked the same way. A safe scalar keeps its type, a masked one is logged as a string, an error is masked as its text, and an Omit decision drops a top-level attribute. Inside a group it logs the marker instead: log/slog writes a broken line when ReplaceAttr drops every member of a group and another attribute follows it. The message is masked as a string attribute named msg, so it is searched as described in Secrets inside text; that is a safety net, so still pass secrets as attributes rather than in the message. The built-in time, level, and source attributes are not masked.

Masking happens only in a handler that calls ReplaceAttr. The standard TextHandler and JSONHandler do; the default handler behind slog.Info before slog.SetDefault, and third-party handlers that ignore HandlerOptions, log attributes unmasked.

zerolog

The zerologmask adapter wraps the logger's writer, because zerolog serializes its fields as they are added and its hooks cannot change what is written:

logger := zerolog.New(zerologmask.NewWriter(os.Stdout, m))
logger.Info().Str("user", "alice").Str("password", "hunter2").Msg("login")
// {"level":"info","message":"login","password":"[REDACTED]","user":"alice"}

Masking the output line covers every field, including those added through Interface or RawJSON. The line is re-encoded, so its keys come out sorted, as in every JSON document the masker writes. Any logger that writes one JSON object per line works the same way. For human-readable output put the masking writer in front of zerolog.ConsoleWriter, so the console formats an already masked line. A writer that routes or filters by level, such as a zerolog.MultiLevelWriter or zerolog.FilteredLevelWriter, loses that when wrapped and receives every level: wrap each destination instead. zerolog built with the binary_log tag writes CBOR, not JSON, and every line is replaced. A line the masker cannot parse is replaced by {"message":"[REDACTED]"}. The message is searched like any other string, as described in Secrets inside text; that is a safety net, so still pass secrets as fields rather than in the message.

zap

zapmask masks the lines zap's JSON encoder writes, after encoding, so the library does not import zap. Its WriteSyncer goes to zapcore.NewCore directly:

sink := zapmask.NewWriteSyncer(os.Stdout, m)
logger := zap.New(zapcore.NewCore(zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()), sink, zap.InfoLevel))
logger.Info("login", zap.String("user", "alice"), zap.String("password", "hunter2"))
// {"level":"info","msg":"login","password":"[REDACTED]","ts":...,"user":"alice"}

Fields added through With, zap.Dict and zap.Any values are masked like any nested object, and a zap.Namespace is decided as an object, so a credentials namespace becomes the marker as a whole. Sampling, level filtering, zapcore.Tee and zapcore.BufferedWriteSyncer keep working, because the writer only sees what zap has decided to write. zap writes an error's %+v text under keyVerbose, a multi-error's parts under keyCauses and a panic while encoding a field under keyError; each of these is also decided as its base key, so zap.NamedError("token", err) does not log the token again under tokenVerbose. The msg text is searched by the text detectors like any other string. The console encoder does not write JSON, and every one of its lines is replaced by the marker line.

Errors and fail-closed behavior

Every public operation returns a safe fallback on an error; it never returns the original unsafe value. Use errors.Is for a category and errors.As for safe operation context:

var detail *masker.MaskError
output, err := m.MaskJSON([]byte(`{"password":`))
if err != nil && errors.Is(err, masker.ErrInvalidJSON) {
	fmt.Println(string(output)) // [REDACTED] as a JSON string
}
if errors.As(err, &detail) {
	fmt.Println(detail.Code, detail.Path)
}

Exported Err* values identify categories such as ErrInvalidJSON, ErrInvalidUTF8, ErrDepthLimit, ErrNodeLimit, ErrCycle, and ErrPanic. ErrorCode constants use the corresponding Code* names.

Never substitute the original input for an error result in application code or in an adapter. The public operations already return a safe fallback, and replacing it is the one change that reintroduces the leak the library prevents.

Limits and security

Default limits are depth 32, 100,000 visited nodes, and 8 MiB of JSON input. Traversal recurses once per nesting level, so WithMaxDepth rejects values above 10,000: a Go stack overflow is fatal and could not fail closed. Input skipped after a limit trips is scanned iteratively. Memory still grows with the input because the public byte-slice and reader APIs retain the complete document during masking.

Review custom policies and rules as security-sensitive code. A custom rule's output is checked for valid UTF-8, but the library cannot prove that arbitrary custom logic removed every secret.

Read SECURITY.md for vulnerability reporting and THREAT_MODEL.md for the security assumptions and failure model.

How it is tested

A masking library is only worth what its test suite proves, so the evidence is listed rather than asserted. There are 9,969 lines of tests against 7,555 lines of shipped code.

Check	Evidence
Masking scenarios	260 generated cases across JSON, reflection, URLs and headers; each checks the masked result, not just that nothing panicked
Security goldens	45 recorded decisions in 8 files, covering rules, key casing, limits, nesting, errors and URLs
Fuzzing	6 targets: JSON, strings, case-folded policy lookup, JSON/reflection parity, URLs, text detectors
Logger adapters	slogmask through the real log/slog handlers; zerologmask and zapmask against lines captured from the real zerolog and zap, so the module keeps no dependency
Examples	38, executed and output-checked, so documentation cannot drift from behavior
Coverage	84.1% core, 88.5% httpmask, 90.2% slogmask, 95.9% for the text detectors, 90.8% for the line-masking engine behind zerologmask and zapmask
Go versions	tests, race suite, matrix and fuzz smoke on 1.23.x through 1.27.x plus stable
Supply chain	govulncheck on every push, reporting standard-library advisories the code actually reaches

The version matrix earns its cost: it caught a change in encoding/json string escaping in Go 1.27 on the day stable moved. A separate check then confirmed what mattered — masking a fixed corpus under every supported release still produces byte-identical output.

How this was built

This library was written with AI assistance, using Anthropic's Claude Code and OpenAI's Codex. Everything above is how that is kept honest: a change has to survive the same suite on every supported Go release before it ships. Provenance is not a substitute for review - the maintainer answers for what is here regardless of how it was produced, and you should read it the way you would read any dependency that handles secrets.

Performance

The streaming JSON walker, bounded key cache, direct encoder, and per-masker reflection metadata cache are designed for predictable behavior on nested and wide payloads. On an M3 Pro, full redaction of a string costs 16 ns and allocates nothing, and a 10,000-record JSON document is masked at roughly 125 MB/s. Throughput stays flat as documents grow wider or longer, which matters more than the absolute numbers; the method and the full tables are in PERFORMANCE.md.

Run local checks and benchmarks with:

make test
make race
make bench
make bench-matrix

Releases

Releases carry a source archive and a SLSA build attestation, so the archive can be traced to the workflow and the tag that produced it:

slsa-verifier verify-artifact go-masker-vX.Y.Z.tar.gz \
  --provenance-path go-masker-vX.Y.Z.tar.gz.intoto.jsonl \
  --source-uri github.com/icntswm/go-masker --source-tag vX.Y.Z

Taking the module with go get needs none of this: the Go checksum database already verifies what you download. The attestation is for anyone who takes the archive instead.

Documentation map

• package documentation — API reference and runnable examples for every exported constructor, option, rule, and adapter;
• ARCHITECTURE.md — implementation architecture and design decisions;
• PERFORMANCE.md — benchmark methodology and references;
• SECURITY.md — vulnerability reporting and user expectations;
• THREAT_MODEL.md — threat model and security boundaries;
• CHANGELOG.md — unreleased and released changes;
• CONTRIBUTING.md — development and pull request workflow;
• AGENTS.md — the two checks in this repository that pass without running;
• RELEASING.md — tagging, the module proxy, and retractions.

The package godoc contains runnable examples for construction, JSON, reflection, custom rules, struct tags, and the HTTP and logger adapters.

Compatibility and stability

The project is pre-1.0. Public API changes will be called out in the changelog before the first stable release. The supported Go window is Go 1.23 and newer compatible releases.

Contributing

Bug reports and pull requests are welcome. Please read CONTRIBUTING.md before making changes. Security issues must be reported privately according to SECURITY.md.

License

This project is licensed under the MIT License.