KongStateToFile housekeep + way to improve extensibility

[luccamendonca]

Hey Harry :)

As you may know already, we're working on using decK as a go library to help us
with a nice quickstart to manage our Kong EE and offer it as a service to our
internal customers.

Starting up has been quick and easy, but we have some specifics here such as
wanting to be able to easily dump/sync only data, without necessarily writing or
reading files.

Then, we stumbled upon a TODO break-down this giant function on file/writer.go
KongStateToFile and we thought about abstracting all the fetching and parsing/zeroing,
allowing us to fetch only the data without needing to write the file.

We could do this on our end, of course, but in our opinion it would be something
nice to have under decK (or even go-kong itself, which seems to make more sense)
to make it a bit more extensible.

What do you think about that, @hbagdi?

Thanks in advance :)

---

cc @marcelometal @MayconDionisio

[hbagdi]

Can you share some snippets of code on what exactly you are doing?
I can't get a grasp on how the code would look like with the change you are referring to. I get the idea and I like the sound of it.

[luccamendonca]

Hey Harry,

We didn't actually start coding those changes into deck. Instead, we're working around the "limitations" on our side by just copying some snippets/methods from decK, specially to make sure our approach makes sense - and until now it's going well, I'd say.

Now, we thought of creating a new layer to manipulate and "convert" the state to a more "raw" struct-based format after doing all the things that are done inside KongStateToFile.

For example:

• On state/state.go (or convert.go, or anything like that) we could have a method to parse - build the relations between entities, sanitize (zeroOut, etc) and return the data ready to be manipulated by go or Marshaled.

• Then, on file/writer.go we'd have KongStateToFileYAML instead of KongStateToFile, which would just call the "converter" on state, Marshal the data to YAML and write to a file. It would also allow us to support other file formats such as JSON (even though I prefer YAML for declarative stuff for a number of reasons).

With those changes we'd be able to extend decK not only to be a CLI, but also to be a powerful lib + difftool + sync tool for Kong.

Of course, we could also somehow take all of the logic and business rules to go-kong and let decK be only a "dumb" frontend to plug go-kong into a CLI application.

And that's pretty much what we thought for now, as we think this could be a good housekeeping to remove logic from the file writer as well as allow us to extend some key parts of decK to use whithin our application, which won't be really a CLI, therefore we're just ignoring the cmd part.

Let us know what you think and if there's anyway we can elucidade even better what our objectives are :)

Thanks!

[hbagdi]

I see what you are talking about. I actually wanted to do this but never got around to implementing it because it never got requested.

Those changes sound good to me. If you could do a quick PoC as a PR, to discuss and iron out the design before you do a full-fledged implementation, that would be best.

[luccamendonca]

Great! So you should hear from us on the next couple days with some news :)

cc @marcelometal @MayconDionisio we should be able to split our focus into advancing on our implementation as give this PoC a go, what do you guys think?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.