Draft
Conversation
added 8 commits
March 27, 2026 10:19
Since this branch is separate from the branch where I'm consolidating the `post-processing-service` I'm not going to do its config update in this branch. Ideally this branch will merge prior to `ab/cthlu-56/consolidate-post-processing` so in that branch I can then merge the changes and transfer over the `post-processing-service` kafka topic configs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
In working on CTHLU-56 I noticed that there was a fair number of duplicate config entries for Kafka topic names when porting over
post-processing-service. In order to eliminate duplicates I went ahead and standardized the Kafka topic configuration keys organizing around the topic prefixes:elastic_*nbs_*nrt_*Given the high number of Kafka topics I wanted to make sure that I verified every one and ensured that the actual topic name wasn't lost in the shuffle. To do that I wrote a script to parse the
application.yamland output a CSV which I used to create the following spreadsheet that I used to go topic by topic:https://docs.google.com/spreadsheets/d/12RyCBvzYOP7YwFs5qrSGdfBOv9KoymOV2-CdqFwmai4/edit?usp=sharing
This PR represents 47 of the topics, leaving out the topics for
post-processing-servicethat will be handled in the main CTHLU-56 PR.Related Issue
Jira Ticket
Additional Notes
Checklist