`mcap merge` should deduplicate identical channels

[IvDmNe]

Description
I want to create several .mcap files and then merge them into one (as I understand, FoxGlove studio can display only single file). I successfully merge different mcap files. But in FoxGlove Studio in panel Data Source a total number of messages is incorrect, as well as frequency of messages in each topic.
It seems that the total number of messages is taken from one of .mcap files before merging.

Despite the shown number of messages is wrong, it seems that in panels all messages are visualized (I checked panels plot and 3D).

• Version: v0.0.24
• Platform: Ubuntu 20.04

Steps To Reproduce

1. Create several mcap files;
2. use mcap merge on them;
3. check statistics of the file with mcap info or in FoxGlove Studio in panel Data Source.

Expected Behavior
Statistic about messages represents the whole file.

Actual Behavior
Number of messages counts only one file's messages.

[foxhubber]

Linear: FG-965

[jtbandes]

Thanks for the report, but I'm not able to reproduce this issue in a simple test. Could you please provide example files that can be used to reproduce the issue?

Notice the new message count is 3025 + 3025 = 6050:

~/Desktop/scripts$ mcap info arrows.mcap 
library: 
profile: 
messages: 3025
duration: 49.95s
start: 0.000
end: 49.950
compression:
	zstd: [112/112 chunks] [118 MB/08 MB (93.06%)] [0.16 MB/sec] 
channels:
  	(1) scene updates    1000 msgs (20.02 Hz)   : foxglove.SceneUpdate [jsonschema]     
  	(2) scene updates 2    25 msgs (0.50 Hz)    : foxglove.SceneUpdate [jsonschema]     
  	(3) transforms       2000 msgs (40.04 Hz)   : foxglove.FrameTransform [jsonschema]  
attachments: 0
metadata: 0

~/Desktop/scripts$ mcap info cubes.mcap 
library: 
profile: 
messages: 3025
duration: 49.95s
start: 0.000
end: 49.950
compression:
	zstd: [93/93 chunks] [97 MB/07 MB (92.21%)] [0.14 MB/sec] 
channels:
  	(1) scene updates    1000 msgs (20.02 Hz)   : foxglove.SceneUpdate [jsonschema]     
  	(2) scene updates 2    25 msgs (0.50 Hz)    : foxglove.SceneUpdate [jsonschema]     
  	(3) transforms       2000 msgs (40.04 Hz)   : foxglove.FrameTransform [jsonschema]  
attachments: 0
metadata: 0

~/Desktop/scripts$ mcap merge arrows.mcap cubes.mcap -o merged.mcap

~/Desktop/scripts$ mcap info merged.mcap 
library: mcap go v0.2.0
profile: 
messages: 6050
duration: 49.9s
start: 0.050
end: 49.950
compression:
	zstd: [27/27 chunks] [216 MB/16 MB (92.34%)] [0.32 MB/sec] 
channels:
  	(1) scene updates    1000 msgs (20.04 Hz)   : foxglove.SceneUpdate [jsonschema]     
  	(2) scene updates    1000 msgs (20.04 Hz)   : foxglove.SceneUpdate [jsonschema]     
  	(3) scene updates 2    25 msgs (0.50 Hz)    : foxglove.SceneUpdate [jsonschema]     
  	(4) transforms       2000 msgs (40.08 Hz)   : foxglove.FrameTransform [jsonschema]  
  	(5) scene updates 2    25 msgs (0.50 Hz)    : foxglove.SceneUpdate [jsonschema]     
  	(6) transforms       2000 msgs (40.08 Hz)   : foxglove.SceneUpdate [jsonschema]     
attachments: 0
metadata: 0

[IvDmNe]

Oh, I missed the duplicated channels. Yes, my output is the same as yours. But it confuses me: shouldn't channels with the same name be merged together instead of duplicating?

[jtbandes]

Yes I think it would be reasonable to expect identical channels to be merged. Converting to a feature request.

[IvDmNe]

Thanks for the response!

[vrsiddarth]

Also, when I attempt to view the combined MCAP file in Foxglove, I can see the complete recording, However, the information about the data sources is incorrect. It fetches information from the first part of the file. Ignores the rest. It shows incorrect frequency and message count.

[vrsiddarth]

[aharmat]

Adding my vote for this feature (merging identical channels). I currently have an application where we stitch together time-sequential mcap files that all have the same topics, and the resulting mcap file has a huge number of duplicated channels. This makes it hard to interpret the output of mcap info, and as mentioned above, the reported topic rates are incorrect.

[mwcondino]

Bump here, I'm facing the same issue as well -- is there any plan to work on this?

[defunctzombie]

Maybe related PR: #1322