How can I determine the basis for manually merging topics using merge_topics()?

[adiosChung]

Hi Maarten！
I am currently writing my graduation thesis on the BERTopic model. During the process of using BERTopic, I initially clustered 23 topics. However, after observing the results through visualize_topics() and visualize_hierarchy(), I found that the merging results were not satisfactory. So, I manually merged topics using merge_topics() based on the similarity of topic representations. Is it reasonable to do so?
My supervisor asked me how I determined the number of topics to merge. I am not sure whether it is convincing to say that I merged topics based on the similarity of topic feature words.May I ask if what I am doing is reasonable?

[MaartenGr]

With topic modeling, the "correctness" of a solution is typically dependent on the context, use case, involved stakeholders, evaluation metrics, etc.

The basis for manually merging topics then depends on a human-based solution, you are the one who decides that certain topics should be together. In other words, what is your thinking process for doing so? What ground, hypothesis, theory, testing, etc. do you have for wanting to merge them together?

All in all, manually merging topics is a very valid process but when somebody asks you why you did it, you generally do need a reason for doing so. The reason depends on the use case. For instance, I could consult domain experts and ask them if certain topics should be together or merged.

[adiosChung]

That is to say, as long as I can explain based on domain knowledge why these topic feature words are similar and how they reflect similar thematic content, I can manually merge the topics instead of using the results generated by visualize_topics() and hierarchical_topics(), right? Thank you for your answer, Maarten😄

[MaartenGr]

Yes, as long as you have a good reason for doing so (whether it is grounded by statistics, research, or otherwise).