Port the ROS 2 DDS-middleware decoupling (FastDDS + CycloneDDS + Zenoh, --rmw=) from ue4-dev to ue5-dev

[JArmandoAnaya]

CARLA version: 0.10.x (ue5-dev)
Platform/OS: Ubuntu 22.04 / 24.04 (Linux), Windows 11
Target branch: ue5-dev

Summary

The ROS 2 DDS-middleware decoupling that shipped on ue4-dev (PRs #9608, #9612, #9619, #9620, #9643, #9644, #9645, plus the follow-ups #9665, #9666, #9669, #9672, #9681, #9692, #9711) is not present on ue5-dev. This issue tracks porting it so the UE5 simulator can run its native ROS 2 connector on FastDDS, CycloneDDS, or Zenoh, selectable at runtime via --rmw=.

Two later ue4-dev PRs build on top of the decoupling and are included in this port as separate trailing PRs, so the core porting strategy is unchanged: #9735 (merged) adds Zenoh as a third middleware (--rmw=zenoh), and #9778 (merged) adds a configurable ROS 2 domain id via --ros-domain-id. Both are now merged on ue4-dev, so no PR in the series is blocked on upstream.

On ue4-dev the work replaced the hard FastDDS dependency with a strategy-pattern abstraction (Middleware enum, MiddlewareFactory, IPublisherMiddleware / ISubscriberMiddleware), introduced plain-old-data message structs serialized through a single unified CDR path (CdrSerialization + a generic GenericCdrPubSubType that replaces all the per-type generated classes), added a CycloneDDS implementation over raw dds_writecdr(), and removed the fastddsgen-generated type files.

Porting this to ue5-dev is not a cherry-pick, because the two branches build ROS 2 in fundamentally different ways. This issue records the analysis and the consolidated plan.

The architecture gap (why this is an adaptation)

Concern	ue4-dev	ue5-dev
Where ROS 2 code compiles	statically into carla-server	into a standalone shared library libcarla-ros2-native.so (the Ros2Native/ ExternalProject)
DDS vendor libraries	static archives linked into carla-server	shared libraries built by ExternalProject, copied to the plugin Binaries, declared as runtime dependencies and loaded with AddDynamicLibrary
DDS symbols in carla-server	allowed (monolithic)	forbidden, kept behind an opaque PImpl, only the ROS2 facade compiles into the server
Publisher header shape	exposes the templated impl directly in the header	opaque forward-declared impl, the template lives only in the .cpp inside the shared library
Build driver	make plus LibCarla/cmake/{server,test,fast_dds,cyclone_dds,ros2}/	top-level CMake plus Ros2Native/CMakeLists.txt, none of those cmake/* subdirectories exist in UE5

The single most important consequence: on ue4-dev the publisher headers embed the templated PublisherImpl<T> directly, which pulls FastDDS headers into every translation unit. That is fine in the monolithic carla-server, but in UE5 the ROS2 facade lives in carla-server and must stay free of DDS symbols. So the cutover must keep the existing opaque PImpl, the templated impl and the MiddlewareFactory includes belong only in .cpp files compiled into the shared library, plus one small DDS-free bridge function exported from the library so the facade can select the middleware without including any vendor header.

Two further points: in UE5 there is exactly one compilation of the ROS 2 sources (the shared library) built with both vendor macros, so the dual-archive symbol-clash CMake restructure from #9644 is not needed. And PR 1 raises the shared library to C++20 (it previously compiled at the Clang C++17 default because it never set the standard), so the ported headers may use the full C++20 toolset.

Per-PR pertinence ledger

ue4-dev PR	Disposition	Reason
#9608 abstraction layer	Port	core interfaces, factory, impl templates (reshaped behind PImpl)
#9612 POD types + conversions	Port (trimmed)	keep the POD structs, drop FastDDSConversions/FastDDSTypeMap (superseded by CDR and removed before the series ended)
#9619 migrate publishers	Port (reshaped)	the cutover, with the PImpl reshape above
#9620 CycloneDDS enum + stubs	Fold	land the real CycloneDDS implementation directly
#9643 unified CDR	Port (split)	serialization goes in the first PR, the FastDDS generic type in the second
#9644 CycloneDDS impl + selection	Port (split) + drop fixes	port the CycloneDDS implementation and the runtime selection, drop the dual-archive CMake restructure
#9645 remove generated types	Port	deletion of the existing generated type files in UE5
#9665 forward flags to editor	Adapt	reimplement against the UE5 launch path (no make scripts)
#9666 install headers for Windows	Drop	edits a LibCarla/cmake/server/ path that does not exist in UE5
#9669 guard test deps	Adapt	re-express the intent against the UE5 test build
#9672 update example	Adapt	reconcile the CycloneDDS config and rviz launcher with the existing UE5 example
#9681 type hashes + shared participant	Fold	already baked into the final files being ported, arrives for free
#9692 rename to --rmw=	Fold	adopt the final flag name from the start
#9711 rename to Middleware	Fold	adopt the final names and paths from the start
#9735 Zenoh middleware	Port (adapted)	third middleware over the same abstraction; the zenoh-c build moves from Setup.sh + LibCarla/cmake/zenoh/ (neither exists in UE5) into the Ros2Native/ ExternalProject, linked into the shared library
#9778 configurable domain id	Port	MiddlewareConfig + --ros-domain-id with ROS_DOMAIN_ID fallback, honored by all three middlewares; merged on ue4-dev, ported from the merged squash commit

Proposed plan (9 PRs)

Every PR compiles, every PR keeps FastDDS publishing functional, every PR stays within the 40-file limit (the deletion-only PR excepted). The series table below is included in each PR description, and commit titles carry no series index so the order can change without rewriting commits.

PR	Scope
1	POD message types, unified CDR serialization, CDR round-trip tests, ROS-2-gated test wiring
2	Middleware abstraction interfaces and factory, FastDDS-on-CDR middleware, generic publisher/subscriber impl, shared participant
3	Cutover, route publishers and subscribers through the abstraction, add the DDS-free middleware-selection bridge, FastDDS remains the default
4	Remove the now-dead generated FastDDS type files
5	CycloneDDS middleware and the CycloneDDS shared-library ExternalProject, plugin runtime dependencies (Linux)
6	Runtime --rmw= selection, plugin integration, examples, docs, one consolidated CHANGELOG entry for the decoupling
7	Zenoh middleware (--rmw=zenoh): shared session, wire format, publisher/subscriber middlewares, zenoh-c in the Ros2Native/ build, example and docs (from #9735)
8	Configurable ROS 2 domain id (--ros-domain-id, ROS_DOMAIN_ID fallback, default 0) honored by all middlewares, example and docs (from #9778, merged on ue4-dev)
9	Upgrade Fast-DDS to 2.14.6 (LTS): bump the pin and migrate the shared CDR serializer and the FastDDS generic type to the Fast-CDR 2.x API (classic CDR kept as XCDRv1, renamed length accessor, 2.13+ representation-aware overloads); wire format byte-identical (from #9789). Sequenced after PR 4.

PRs 7 and 8 are standalone user-visible features appended after the core series; each carries its own CHANGELOG line, mirroring its ue4-dev source PR. PR 8 depends on PR 7 because the upstream domain id change already covers the Zenoh wire format. PR 9 upgrades Fast-DDS to the 2.14.6 LTS line (which ships Fast-CDR 2.x); it is sequenced after PR 4 because the breaking Fast-CDR 2.x API does not compile against the generated *PubSubTypes files that PR 4 removes.

A feasibility review against the merged sources surfaced two adaptations beyond the build-system move, both contained within their PRs:

• PR 7, Zenoh config path. Upstream ZenohSharedSession.cpp locates zenoh_session_config.json5 through a compile-time __FILE__ source path, which cannot resolve in a packaged UE5 build where the source tree is absent at runtime. The port stages the json5 with the package (plugin runtime dependencies), resolves it relative to the binary, and falls back to the zenoh default config when the file is missing. On the build side the question of static vs shared linking answers itself: the zenoh-c 1.8.0 prebuilt bundle ships only a static PIC libzenohc.a, which links directly into libcarla-ros2-native.so, so no plugin-side library staging is needed.
• PR 8, domain id across the shared-library boundary. On ue4-dev the --ros-domain-id value flows from CarlaSettings straight into the monolithic LibCarla. On ue5-dev the flag is parsed in the plugin but consumed inside libcarla-ros2-native.so, so the DDS-free middleware-selection bridge from PR 3 gains a second export carrying the domain id. The ROS_DOMAIN_ID environment fallback works unchanged, since it is read inside the shared library.

Test strategy

ue5-dev currently has no ROS 2 middleware unit tests. The ue4-dev suite is 69 GoogleTest cases (CDR serialization, topic info, factory, publisher and subscriber impl, generic type, large payload) plus ROS 2 smoke tests. The unit tests split cleanly across the first two PRs and run vendor-agnostically against a mock middleware. CycloneDDS correctness cannot be unit-tested (it needs a live participant), so it is covered by smoke tests running the package with --rmw=cyclonedds. PR 7 ports the Zenoh factory and availability unit tests from #9735 and is smoke-tested with --rmw=zenoh against an rmw_zenoh peer; PR 8 ports the domain id unit tests from #9778 (parsing, env fallback precedence, range validation) and is validated end to end by checking that topics appear only on the selected domain.

Notes

• CycloneDDS and Zenoh are Linux only here. On Windows, --rmw=cyclonedds and --rmw=zenoh degrade to a logged error and FastDDS is used.
• Accepted --rmw= values: fastdds (default), cyclonedds, and after PR 7, zenoh. Only valid together with --ros2.
• The domain id resolution order in PR 8 is --ros-domain-id, then the ROS_DOMAIN_ID environment variable, then 0; out-of-range values (valid range 0 to 232) are ignored and the next source is used. This addresses the 0.10.0 half of Changing ROS_DOMAIN_ID in 0.10.0 #8853.

[youtalk]

@JArmandoAnaya Would it be possible to create draft PRs for this issue in advance? I have high expectations for the outcome of this activity.

[JArmandoAnaya]

Hello @youtalk, I have the intention to start today. I already have a migration plan to start the porting. I know the core team is busy, and they are merging changes slowly, but yes, I can start the PR drafts. They will be around 5-6 PRs.

[youtalk]

@JArmandoAnaya Thank you so much! I think that maintainers in Europe are currently on their summer break.