feat: implement --dedup-plugin-configs flag for dump #581
Conversation
There was a problem hiding this comment.
Good start.
A few corner cases to think about:
- What if a plugin configuration is unique and is only present once in the entire state? The code shouldn't introduce an indirection (via plugin source name) in such a case.
- What if a plugin has multiple parents at the same time? Does this logic hold correctly in such a case? I'm not sure either way - please give that some thought and we will certainly need tests around it.
dump/dump.go
Outdated
| // } | ||
| // } | ||
| func dedupPluginsConfig(state *utils.KongRawState, plugins []*kong.Plugin) { | ||
| state.SharedPluginMap = make(map[string]map[string]utils.SharedPlugin) |
There was a problem hiding this comment.
I'd recommend making this function unit-testable:
- Take state and plugins as input
- Return the "SharedPluginMap", and then set "state.SharedPluginMap" to the returned result in the caller's code.
Avoid changing input arguments in place - this codebase doesn't follow this consistently but it is for the best. I myself didn't know better before.
There was a problem hiding this comment.
refactored a bit
| fp.ConfigSource = kong.String(name) | ||
| break | ||
| } | ||
| } |
There was a problem hiding this comment.
Suggest two functions: one takes in the plugin map and the relation IDs, and returns the name of the plugin config source (if found), and this one then sets the config to nil and source to non-nil. This way you can unit-test the function that looks through sharedPlugins map.
There was a problem hiding this comment.
refactored a bit
GGabriele
force-pushed
the
dedup_plugins
branch
5 times, most recently
from
dd3c2cd to
8ea7c1d
Compare
| if p.Route != nil && p.Route.ID != nil { | ||
| sharedPlugin.Routes = append(sharedPlugin.Routes, *p.Route.ID) | ||
| } | ||
| return sharedPlugin |
There was a problem hiding this comment.
The tracking here loses the relationship of a plugin with multiple entities.
Consider the following plugin:
plugins:
name: rate-limiting
service:
id: svc-id
consumer:
id: consumer-id
config:
day: null
fault_tolerant: true
hide_client_headers: false
hour: null
limit_by: consumer
minute: 10
month: null
policy: redis
redis_database: 0
redis_host: redis.common.svc
redis_password: null
redis_port: 6379
redis_timeout: 2000
second: null
year: null
enabled: true
protocols:
- http
- httpsIn this case, this data structure would track the plugin as a plugin that is applied to the consumer, and the same plugin is applied to the service, in reality, that is not actually true, the plugin is present only once, on a combination of service and consumer.
The code that uses this data structure, later on, is structured in a way that one never runs into this bug currently. But that is a bit too risky to rely on. Please consider a data structure that captures the entire relationship explicitly.
_plugin_configs feature of decK allows users to abstract out configuration of plugins that are used repeatedly and then reference them in services/routes/consumers. This helps with de-duplication of configuration code. While users would like to take advantage of this, most of them cannot because de-duplicating the configuration in the first place is a cumbersome and an error-prone process. This implements such flag and feature for the dump command.
|
Closing this due to lack of activity. Please re-open if needed. |
#341 (doc)
This implements such flag and feature for the dump command.
Test:
syncthis file first (before.yaml.txt), having some shared plugins across different consumers, routes and services.Result: after.yaml.txt