Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka Stream Topology visualizer #205

Open
IldarAlmakaev opened this issue Feb 20, 2021 · 2 comments
Open

Kafka Stream Topology visualizer #205

IldarAlmakaev opened this issue Feb 20, 2021 · 2 comments
Assignees
Labels
scope/backend scope/frontend status/accepted An issue which has passed triage and has been accepted type/feature A new feature
Milestone

Comments

@IldarAlmakaev
Copy link

IldarAlmakaev commented Feb 20, 2021

As a user, I would like to copy&paste a generated Kafka Streams topology from an application log and then generate a nice-looking diagram in Kafka UI application.
It would help users to review how their stream topology is built.
As an example, you can take a look at this repo.
example

We can implement a parser of this topology in the backend and then provide models for easier display in UI

Here we could add features like:

  • interactive UI elements (e.g. by clicking on a UI elements ability to display metadata of sources, sinks, processors, transformers, etc
  • Downloading this nice-looking chart/diagram
    • something else :)

The data model is a graph that consists of source connectors, sink connectors, and consumers. Consumers are processor topologies that consist of stream processors (nodes) that are connected by streams (edges).
The graph nodes have additional info:

  • the number of processors and topics at the consumer level
  • the number of input messages, output messages, and lag at the topic level
@IldarAlmakaev IldarAlmakaev added type/enhancement En enhancement to an already existing feature scope/backend scope/frontend labels Feb 20, 2021
@germanosin germanosin added this to the 0.2 milestone Mar 5, 2021
@GneyHabub GneyHabub self-assigned this Jun 15, 2021
@Flyur Flyur self-assigned this Jun 25, 2021
Flyur added a commit that referenced this issue Jul 6, 2021
Flyur added a commit that referenced this issue Jul 6, 2021
Flyur added a commit that referenced this issue Jul 7, 2021
@germanosin germanosin modified the milestones: 0.2, 0.3 Aug 24, 2021
@germanosin germanosin modified the milestones: 0.3, 0.4 Sep 16, 2021
@IldarAlmakaev IldarAlmakaev linked a pull request Oct 13, 2021 that will close this issue
12 tasks
@Haarolean Haarolean added type/feature A new feature and removed type/enhancement En enhancement to an already existing feature labels Dec 21, 2021
@Haarolean Haarolean modified the milestones: 0.4, 0.5 Feb 8, 2022
@Haarolean Haarolean removed this from the 0.5 milestone May 18, 2022
@Haarolean Haarolean added this to the 0.6 milestone May 18, 2022
@Haarolean Haarolean removed this from the 0.6 milestone Jun 8, 2022
@Haarolean Haarolean added this to the 0.10 milestone May 18, 2023
@Haarolean Haarolean added the status/accepted An issue which has passed triage and has been accepted label May 18, 2023
@AndreasKlein
Copy link

Hi everyone, what is the current status of this? Has someone started implementing it already? If not, since it would be a very helpful feature for our teams, is there a way we can help?

@Haarolean
Copy link
Contributor

@AndreasKlein #4255

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope/backend scope/frontend status/accepted An issue which has passed triage and has been accepted type/feature A new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants