V2: Declarative Management API

[markphelps]

New Declarative Management API Design

Overview

Design and implement a new declarative management API for Flipt v2 that supports GitOps workflows and multi-environment configurations while maintaining compatibility with v1 flag evaluation.

Requirements

Environments

• Environment Support
  • Environments as top-level resources
  • Allow different storage locations per environment
  • Support configuration of multiple remote repositories/folders via environment settings (see: Gitops with multiple repositories #2975)
  • Example: Production state in one repository/folder, staging in another

API Structure

• Resource Hierarchy

  Environment
  └── Namespace
      ├── Flags
      ├── Segments
      └── Other resources
  

• API Routes

  • Introduce environment-aware routing
  • Proposed structure: /v2/management/environments/{env}/...
  • Maintain all existing v1 flag features and nouns (segments, constraints, variants, etc.)
  • Use existing flag state format for backward compatibility

• New Features

  • Support for resource tagging (implementation details TBD) ie: Flag Grouping / Tagging inside Namespace #3739

Validation

• Service-side Validation

  • Validate all configurations before writing to storage
  • Return HTTP 400 Bad Request for invalid configurations with detailed error messages

• Client-side Tools

  • (optionally) Provide CLI validation tool as part of Flipt binary
  • Implement form validation in UI

State Management

• Updates

  • Require full flag state for updates (no partial updates)
  • Implement "last write wins" approach for conflict resolution
  • Version control through git commit history

• Future Considerations

  • Rollback support (planned for future release)
  • Enhanced conflict resolution strategies

Non-Requirements

• Partial state updates
• Initial rollback support
• Complex conflict resolution strategies

Implementation Notes

1. Flag State Format

   • Use existing flag state format: Flag State Documentation
   • Maintain namespace structure: Namespace Documentation

2. Error Handling

   • Clear error messages for validation failures
   • Proper HTTP status codes for different error scenarios
   • Detailed logging for debugging purposes

Questions for Discussion

1. What is the preferred implementation approach for resource tagging?
2. Are there specific GitOps workflows we should optimize for?
3. Is the proposed URL structure sufficient? /v2/management/environments/{env}/namespaces/{namespace}/resource it seems quite verbose to me

[GeorgeMac]

It is a lame optimization but we could shorten management to just manage:

/v2/manage/environments/{env}/namespaces/{namespace}/resource

Also, worth noting that there is also the resource type that needs to go in there, which makes this longer.
It Flipt Cloud we even include the fully qualified protobuf type:

# list flags example:

GET /v2/manage/environments/{env}/flipt.core.Flag/namespaces/{namespace}/resources

# or perhaps:

GET /v2/manage/environments/{env}/namespaces/{namespace}/flipt.core.Flag

# or just:

GET /v2/manage/environments/{env}/namespaces/{namespace}/flags

Which I appreciate is even more verbose. But we do need something to identify the resource type on.
Whicever we choose, it could be nice if there is space for more resource types in the future, or v2 resources types if we choose to evolve the format and evaluation capabilities.

[GeorgeMac]

We could also drop manage altogether like we had with v1.
The operations are effectively scoped beneath environments:

GET /v2/environments/{env}/flipt.core.Flag/namespaces/{namespace}/resources

[GeorgeMac]

Some more thoughts on the implementation details of the API (i.e. protobuf and gRPC shape)

Prior Art (Flipt Cloud)

In Flipt Cloud we planned for the future by using a google.protobuf.Any slot and incorporating a type into the URL (as shown about). This is where we would slot Flag and Segment as types (more specifically called flipt.core.Flag and flipt.core.Segment respectively).

This allowed us to write a single (generic) resource server / storage implementation that routed to the details for marshalling via looking up the implementation type name in a map.

The resulting RPCs looked like so:

service ManagementService {
  // ...
  rpc GetResource(GetResourceRequest) returns (ResourceResponse) {}
  rpc ListResources(ListResourcesRequest) returns (ListResourcesResponse) {}
  rpc CreateResource(UpdateResourceRequest) returns (ResourceResponse) {}
  rpc UpdateResource(UpdateResourceRequest) returns (ResourceResponse) {}
  rpc DeleteResource(DeleteResourceRequest) returns (DeleteResourceResponse) {}
}

Where many of the payloads had to duplicate the type (known as TypeUrl in Any) from google.protobuf.Any:

message Resource {
  // type is the fully-qualified protobuf message name
  string type = 1;
  string namespace = 2;
  string key = 3;
  google.protobuf.Any payload = 4;
}

This was so that we could substitute these from the URL paths into the payload via Gateway.
Also, it was so that we had consistency across the various CRUD operations.
This was required for Get / List / Delete, but for Create and Update it was redundant because its in the Any payload again.

It was a little weird and I wonder if we can do any better.

Possible Paths Forward

URL structure

1. Repeat sins of the past

Just do the same again, incorporating environment scope into the protobuf types and gRPC gateway routes.

For example:

message Resource {
  string environment = 1;
  string type = 2;
  string namespace = 3;
  string key = 4;
  google.protobuf.Any payload = 5;
}

and

# Read
GET /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}/{key}

# List
GET /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}

# Create
POST /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}

# Update
PUT /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}

# Delete
DELETE /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}/{key}

2. Remove the redundency

We could remove the redundency from the Create and Update RPCs, by removing the Type from the payloads.
This will mean we can't include those in the paths and make them structured slightly differently to Read, List and Delete.

The benefit just removes some odd redundency when interfacing with the Update and Create endpoints where you need to duplicate the type information.

Something like (compared with the previous approach):

# Read
GET /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}/{key}

# List
GET /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}

# Create
-POST /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}
+POST /v2/environments/{env}/namespaces/{namespace}/resources

# Update
-PUT /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}
+PUT /v2/environments/{env}/namespaces/{namespace}/resources

# Delete
DELETE /v2/environments/{env}/namespaces/{namespace}/resources/{type=**}/{key}

3. Have N x M CRUD RPCs

Where M is the number of RPCs for the different operations (i.e. read / list / create / update / delete ~ 5) and N is the number of resource types we want to implemented (currently flags and segments ~ 2).

Instead of having a generic CRUD slot supporting both types, we just enumerate all the options explicitly.

Future resource types will need enumerating in the API contract in order to add them.
That is the one benefit of the other approach, is that resource types can be added without changing the API.

Type parameters

Given we go forward with something like (1) or (2) then we need to decide how to address types.
We chose in our prior attempt to use the protobuf package TypeURL as this was a canonical path and we could add new types in the future under different package structure naming conventions.

For example, Flag type resource was flipt.core.Flag (it lived in rpc/flipt/core directory).

Alternatively, we could alias these to something else that is more compact.
For example, Segment type resource could be just segment or segment.v1 or v1/segment.