Repository navigation
Conversation
5 of 14 tasks
Cover building a parser, parsing each element kind on a warm parser, and the FromStringFact helper, which builds a new parser on every call.
Every FromString* call built a new parser: six participle parsers,
about 2.5 ms and 2 MiB per call, for a parse that takes 40 µs.
Keep one parser behind a sync.Once and reuse it. A built participle
parser is stateless and documented as safe for concurrent use.
FromStringFact 2669 µs -> 40 µs
2.1 MiB -> 16 KiB
14674 allocs -> 266
New() is unchanged for callers that want their own instance.
ExprTerm.ToExpr dropped the error from Term.ToBiscuit (there was a
FIXME). An unbound {parameter} or any other bad term inside an
expression produced a Value with a nil Term. Parsing reported
success, and the nil dereference happened later, when the check or
rule was converted for a token or an authorizer.
Make the ToExpr chain return an error and propagate it from
Rule.ToBiscuit and CheckQuery.ToBiscuit. Operator.ToExpr now also
reports an unknown operator instead of appending a nil op.
ToExpr is exported but only used inside this package.
The term list was `(@@ ("," @@)*)*`. The outer repetition let the
whole group match again without a separator, so `f(1 2)` parsed as
`f(1, 2)` and `f(1, 2 3, 4)` as `f(1, 2, 3, 4)`.
Make the group optional instead of repeated. Zero terms are still
accepted.
The lexer takes the first rule that matches, and the Function and
Bool rules had no word boundary. `prefixed("a")` lexed as `prefix`
followed by `ed`, `truename("x")` as `true` followed by `name`, and
the parse failed with "unexpected token".
Drop the Function rule: method names now lex as Ident, and the
grammar matches them by value after a Dot, so `.length()` and
friends still work. The rule also listed prefix and suffix, which
are not the method names (starts_with, ends_with). Add \b around the
Keyword and Bool patterns.
Value and Set were left over from an earlier grammar. Nothing references them; terms and sets are parsed through Term.
Parse every block and authorizer snippet from samples.json and compare the outcome, accepted with this AST or rejected, against a checked-in golden file. Any lexer or grammar change that alters what the parser accepts or produces for the 85 snippets shows up as a diff in review. The golden file is identical when generated from upstream/main and from this branch: none of the fixes on this branch changes the outcome of any sample. The 10 rejected snippets use v3.1+ syntax the grammar does not support yet; they will change to ok as support lands. Regenerate with: go test ./parser/ -run TestSamplesCorpus -update
Generate pb/schema.pb.go with buf from the schema.proto published in the spec repository, instead of a vendored copy of the proto file and a protoc install script. The v3.3 schema adds the fields later changes need (scopes, check kinds, public key table, external signatures); none of them is used yet. `make generate` runs script/generate.sh. Co-authored-by: Antonio Ojea <aojea@google.com>
…sion The samples.json published with the v3.3 spec changed shape: facts, rules and checks are grouped by origin, authorizer checks appear next to block checks, and new error kinds (Execution, block signature deserialization) are reported. Update the test structs to match. Samples whose blocks exceed biscuit.MaxSchemaVersion must be refused by Unmarshal; the test checks that and then skips. An explicit unsupported list covers the samples inside the supported range that cannot pass yet, so that each later change removes its entry.
Sets are written {a, b} in the spec and the empty set {,}; the operator
printed and parsed as === is the Equal opcode. biscuit-go accepted [a, b]
only, had no syntax for the empty set, rejected empty sets when converting
blocks, and printed the Equal opcode as ==.
Accept the spec forms and print them. {name} stays a parameter: the
grammar now disambiguates {…} instead of a lexer token, so {true} is a
set and {abc} a parameter. [a, b] is still accepted as a set, since
existing callers use it; the spec reuses that syntax for arrays, which
will replace it when array support lands.
Enables samples test013_block_rules and test017_expressions.
Tokens may be signed with ECDSA over secp256r1 as well as Ed25519, and each block declares which payload format its signature covers (SignedBlock.version). Version 1 tags every field and binds each signature to the previous one; the spec requires it for non-Ed25519 keys, for third-party blocks and for datalog v3.3 blocks. Move the key handling to internal/crypto, with one Signer/Verifier pair per algorithm and the encodings of the spec (compressed SEC1 point, big endian scalar, DER signature over SHA-256 for ECDSA). Sign and verify through it, choosing the payload format per block the way biscuit-rust does: v1 when required, otherwise the format of the previous blocks, so Ed25519-only tokens keep producing the same bytes. Each block may use a different algorithm; the next key defaults to the signer's. Public API: Authorizer and WithSingularRootPublicKey take a crypto.PublicKey (ed25519.PublicKey, *ecdsa.PublicKey, or []byte as an Ed25519 key, which existing callers pass). NewBuilderWithSigner accepts an *ecdsa.PrivateKey root. Unmarshal validates key and signature sizes per algorithm instead of assuming Ed25519. Enables sample test036_secp256r1. Part of this ports the internal/crypto package from eclipse-biscuit#178. Bump the module to go 1.25, which ecdsa.ParseRawPrivateKey requires. Co-authored-by: Steve Sloan <steve.sloan@apple.com>
The debugger looked up strings and variables in the symbol table only at
the top level of a term. Elements of a set were printed through
Set.String, so a fact such as allowed({"A", "B"}) came out as
allowed({#1024, #1025}), in facts and in expression values alike.
Print every term through one function that recurses into sets.
`check all` (datalog v3.1) passes when a query has at least one match and every match of that query satisfies its expressions, where `check if` needs a single match. Add the check kind to the datalog, biscuit and parser layers, encode it in CheckV2.kind (left unset for `check if`, as the other implementations do) and evaluate it in the authorizer. A block using `check all` must declare version 4, so the builder now computes the block version from its content instead of stamping the maximum supported one; blocks without v3.1 features keep version 3. MaxSchemaVersion becomes 4. A v4 block is rejected when it uses a v3.1 feature this library does not support yet (scopes), so that no token is accepted with altered semantics; the v3.1 operators are already rejected by the expression converter. A v3 block declaring a check kind is rejected, as the spec requires. Enables sample test025_check_all. Samples 027 and 028 are now in range and listed as unsupported until the v3.1 operators land.
World.QueryRule discarded the error returned by Rule.Apply, so an expression that overflowed, divided by zero or mixed types made the query look like it had no match. A failing check was then reported where the spec requires authorization to stop with an execution error. QueryRule now returns the error, and the authorizer wraps it in ErrExecution and returns at the first one, from token checks, authorizer checks and policies alike. A stray debug print in the evaluation loop is removed.
Datalog v3.1 adds strict inequality (!==) and the bitwise operators &, | and ^ on integers. Add them to the datalog, biscuit and parser layers, with the precedence of the spec (^ binds loosest, then |, then &, between comparisons and arithmetic), and encode them with the OpBinary kinds of the schema. Using one of them makes the block version 4; a v3 block carrying one is rejected, so the content-based version check now covers operators as well as check kinds. The parser accepted only non-negative integers. Accept a leading minus on a literal, parsed from the full text so that the smallest int64 is representable; the sign is a separate token, so "1 -2" stays a subtraction. Enables samples test027_integer_wraparound and test028_expressions_v4. The sample test maps the spec's Execution/Overflow error to ErrExecution and datalog.ErrInt64Overflow.
Every fact in the world now carries its origin, the set of blocks it derives from: the block that stated it, or for a derived fact the blocks of every fact the rule matched plus the block of the rule. Every rule, check and policy reads only the facts whose origin is within the origins it trusts. Without scopes those are the authority block, the authorizer and the rule's own block, which is the execution scope the spec defines and the model that scopes (`trusting`) and third-party blocks build on. The authorizer used one world per block, cloned from the authority world, to emulate that scope. It now loads the whole token into a single world and evaluates each check with the trusted origins of its block. World.Clone copied the slice header only, so clones shared their fact storage; it copies the facts now. World API: AddFact takes the origin, AddRule the block and the trusted origins, Query/QueryRule/QueryMatchAll the trusted origins; Facts returns the facts grouped by origin and Rules the rules with their block. PrintWorld lists facts by origin and rules by block, in the layout of the reference implementation. The unused Biscuit.generateWorld and its test are removed. The sample test now compares the whole world, with origins, against the expectation of the spec samples instead of the authority and authorizer part only.
A rule, check or policy may end with `trusting authority`, `trusting previous` or `trusting <public key>`, and a block or an authorizer may set that as the default for its own bodies. The clause selects the blocks whose facts the body reads: the authority block, every block up to the current one, or the third-party blocks signed by the key. Without a clause the defaults apply: authority, authorizer and the current block. Public keys in scopes are interned like strings: a block carries the keys it introduces, and a scope refers to a key by its index in the token's key table (datalog v3.1, block version 4). The parser reads `algorithm/hex` keys, the printers write them, and the converters resolve indices against the table that Unmarshal, Build and Append maintain. The guard that rejected v4 blocks with scopes is removed. In the authorizer, each block's default trusted origins come from its scopes, each body resolves its own on top, and the map from third-party keys to block ids is in place for third-party blocks to fill. Public API: Scope and PublicKey (aliases of the datalog types), PublicKeyFrom, ParsePublicKey, TrustingPublicKey, Rule.Scopes, ParsedBlock.Scopes, BlockBuilder.AddScope, Builder.AddAuthorityScope, Authorizer.AddScope.
A third party can sign a block for a token it does not hold: the holder sends it a ThirdPartyBlockRequest (the signature of the token's last block), the third party answers with a ThirdPartyBlockContents (the serialized block and its external signature), and the holder appends it with AppendThirdPartyBlock. Facts and rules of such a block are visible to the bodies that trust the third party's key (`trusting <key>`). Reading: a SignedBlock with an external signature is a third-party block. Its symbol table and key table are its own, starting from the defaults, and do not extend the token's tables (the next blocks ignore them). Its version must be at least 5 (datalog v3.2), and MaxSchemaVersion becomes 5. The external signature covers the block and the previous block's signature (external payload v1); the block signature covers the external signature as well, in both payload versions. The authorizer maps external keys to block ids, which the scopes of the previous commit resolve against. Writing: ThirdPartyRequest, NewThirdPartyBlockBuilder, CreateBlock, AppendThirdPartyBlock, and Serialize/Unmarshal for the two messages. AppendThirdPartyBlock verifies the external signature against the expected key before appending, so a block made for another token or signed by another key is refused. Enables samples test024_third_party, test026_public_keys_interning and test037_secp256r1_third_party.
Append and Seal rebuilt the container without RootKeyId, so a token created with WithRootKeyID lost the id at the first attenuation and the verifier could no longer select the root key by id.
`reject if` (datalog v3.3) passes when none of its queries has a match, the opposite of `check if`. Add the check kind to the datalog, biscuit and parser layers, encode it as CheckV2.Reject and evaluate it in the authorizer. A block using it is version 6. MaxSchemaVersion becomes 6. The other v3.3 features are not supported yet; a v6 block using them is rejected by the converters (unknown term types, operators and closures), so no token is accepted with altered semantics. Their samples are listed as unsupported until they land. Enables sample test029_reject_if.
Since datalog v3.3, == and != compare values of different types as not equal, where === and !== (the strict forms) make that a type error. Add the two operators, encode them with the OpBinary kinds of the schema, and make the parser read == and != as them, as the reference parser does; a block using one is version 6. Source that needs a v3.0 block must use === and !==, which is also how older tokens print. Enables sample test031_heterogeneous_equal.
null (datalog v3.3) is the absence of a value: a term equal only to itself, which the heterogeneous == and != compare with any other type without error. Add it to the datalog, biscuit and parser layers and encode it as TermV2.null. A block holding one, in a fact, a predicate or an expression, is version 6; the version computation now looks at facts and predicates as well as operators. Enables sample test030_null.
Datalog v3.3 evaluates the right side of && and || only when the left side does not decide the result, and adds .all($p -> ...) and .any($p -> ...) on sets. Both rely on closures: an expression pushed on the evaluation stack and run by the operator that consumes it, with its parameters bound by that operator. A parameter may not have the name of a variable already bound (ErrShadowedVariable, an execution error). Add the Closure op and the LazyAnd, LazyOr, All and Any operators to the datalog, biscuit and parser layers, and encode them with the schema's Op.closure and OpBinary kinds. The parser emits the lazy forms for && and ||, as the reference parser does, so a block using them is version 6; the eager And and Or stay for older tokens. The parser needs a lookahead of two tokens to tell `$p -> body` from an argument that starts with a variable. Enables sample test032_laziness_closures.
Datalog v3.3 adds two collection types: arrays, ordered sequences of
terms of any type, written [a, b]; and maps from integer or string keys
to terms of any type, written {k: v}. Add them to the datalog, biscuit
and parser layers, encode them as TermV2.array and TermV2.map, and extend
the operators: .length() and .contains() (an element for arrays, a key
for maps), .starts_with() and .ends_with() on arrays, .get() returning
null when there is nothing at the index or key, equality, and .all() /
.any(), which see a map as [key, value] arrays. Map entries are kept
sorted by key, integers first, so that equal maps print the same.
[a, b] used to be accepted as a set; it is an array now, as in the spec.
Sets are written {a, b}; the README and the tests that meant a set are
updated, including the README's `{{read}}`, a set of one parameter. A
block holding an array or a map is version 6.
Enables sample test034_array_map.
Datalog v3.3 adds the unary .type() operator, which returns the name of the type of a value as a string: "integer", "string", "date", "bytes", "bool", "set", "null", "array" or "map". The result is interned in the symbol table like any other string, so it can be compared with string literals. Blocks that use .type() are version 6. Sample test033_typeof passes.
Datalog v3.3 lets an expression call a function provided by the authorizer: $x.extern::name() (unary) or $x.extern::name($y) (binary). The function name is a symbol (OpUnary.ffiName / OpBinary.ffiName). Authorizer option WithExternFuncs registers the functions by name; they receive and return biscuit terms. In the datalog package the registry (ExternFuncs) is held by the World and passed down to Expression.EvaluateWith, including inside closures. A call to a name that is not registered fails with ErrUndefinedExtern, reported by the authorizer as ErrExecution. The parser lexes extern::name as an identifier and emits biscuit.ExternUnary / biscuit.ExternBinary. Blocks using extern:: are version 6. The samples test registers the test function of the sample generator; sample test035_ffi passes.
The v3.3 samples use .try_or() (test038), but OpBinary.TryOr was added to schema.proto after the v3.3 tag, in the same commit that renames the *V2 messages (FactV2 -> Fact, RuleV2 -> Rule, CheckV2 -> Check, PredicateV2 -> Predicate, TermV2 -> Term, ExpressionV2 -> Expression) and the Block fields facts_v2/rules_v2/checks_v2 -> facts/rules/checks. The field numbers do not change, so the wire format is the same. script/generate.sh pins that commit instead of the tag; the Go code follows the renamed identifiers.
Datalog v3.3 adds receiver.try_or(fallback): the value of the receiver, or fallback when evaluating the receiver fails. The receiver is encoded as a closure without parameters so that it is evaluated lazily; the fallback is an ordinary operand, so an error in it is not caught (the "right-hand side does not catch errors" validation of the sample). Expression evaluation now accepts a closure on either side of a binary operator, as biscuit-rust does; the term operand and the closure are passed to the operator. The parser wraps the receiver chain (the term and the method calls before .try_or()) in the closure. Blocks using .try_or() are version 6. Sample test038_try_op passes; all 38 current samples now run.
The secp256r1 change made Biscuit.Authorizer and WithSingularRootPublicKey take a crypto.PublicKey instead of an ed25519.PublicKey, and changed the result type of PublickKeyByIDProjection. Every known user calls Biscuit.Authorizer with an Ed25519 key, and a custom projection written against v2.2 no longer compiles. Restore the v2 signatures and mark them deprecated; they keep their behavior. Add the provider the specification implementations share: RootKeyProvider selects the root key from the root key ID of the token, as biscuit-rust's RootKeyProvider does; RootKey and RootKeysByID build one from a key or a map; Biscuit.AuthorizerFrom verifies the token with it, for Ed25519 and P-256 alike. Biscuit.Authorizer, AuthorizerFor, WithSingularRootPublicKey and WithRootPublicKeys forward to them. The datalog.World methods keep the origin-aware signatures: the v2 ones could not honor the origins and trusted origins the world now tracks, no caller of them exists outside this module, and the new ones match biscuit-rust's World (add_fact, add_rule, query_rule, query_match_all). The tests, examples, samples and README use AuthorizerFrom; a dedicated test covers the deprecated entry points.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Alternative to #178, in case @polygloton is not able to continue it. This is #178 rebased on top of the fixes that bring the CI to a stable state (#183, #184, #185), keeping the original commits and authorship, plus four small follow-up commits on top:
Makefile: expose the protobuf generation asmake generate, throughscript/generate.sh.samples: drop the root key files, nothing reads them.datalog: re-apply the byte-array fixes from datalog: fix panics on byte-array terms, harden Set.Equal, add benchmarks #185 on the new datalog code and port its tests.datalog: hashSetelements in sorted order. Equal sets hashed differently depending on map iteration order, so duplicate facts were kept. Found by the ported tests.The first 8 commits belong to #183, #184 and #185; review from
Foundational Infrastructureonwards.Known issues, same as #178: the
inspect_with_biscuit_clisubtests require thebiscuitCLI inPATH,TestCombineItIteratoris flaky (map iteration order), andmake lintreports findings in the new code.