fix(LibCarla/ros2): add REP-2011 type hashes and shared FastDDS participant for Jazzy compatibility - #9681
Conversation
…cipant for Jazzy compatibility Addresses two ROS 2 Jazzy interoperability issues: - carla-simulator#9674: ROS 2 Iron+ RMWs log "Failed to parse type hash for topic 'rt/...'" because no USER_DATA QoS is set on CARLA's DDS endpoints. Every publisher and subscriber (FastDDS and CycloneDDS) now advertises a REP-2011 RIHS01 type hash in the REP-2016 KV payload "typehash=RIHS01_<hex>;" (PID_USER_DATA = 0x002c per OMG DDSI-RTPS v2.5 9.6.2.2.2), hardcoded per msg type because CARLA has no IDL code-generation step at build time. - carla-simulator#9675: FastDDS crashes with "Topic with name : rt/tf already exists" then "Signal 11" when multiple sensors on the same actor (or multiple actors) create publishers on the shared topic. Introduces a refcounted shared DomainParticipant (FastDDSSharedParticipant) that owns per-type registration refcounts, and routes create_topic duplicates through find_topic so the second and later endpoints reuse the existing topic instead of provoking FastDDS's duplicate-name error. Additional changes: - carla/multigpu/listener.cpp: treat boost::asio::error::operation_aborted as the normal shutdown path; stop logging "Secondary server: Operation canceled." when Listener::Stop cancels async_accept. - ROS2::Shutdown tears down publishers and tf publishers before subscribers so the shared participant refcount reaches zero in the documented order. - CycloneDDS publisher and subscriber middlewares also set USER_DATA from build_user_data_for<T>() for parity with FastDDS. - Util/ros2/compute_type_hash.sh: Docker-only helper to compute a RIHS01 hash for a new message type, no local ROS 2 required. - Docs/ros2/adding_message_types.md: 7-step guide covering .msg to POD, CDR serialization, hash computation, CdrTopicInfo registration, and regression tests. - test_type_hash.cpp: format and uniqueness tests for all registered types, plus UserDataFormat payload tests. Verified: - make LibCarla ARGS="--ros2": clean, zero warnings in changed files - libcarla_test_server_{debug,release}: 120/120 pass (includes 4 new UserDataFormat tests and 2 TypeHash tests) - libcarla_test_client_release: 56/56 pass - smoke.test_ros2 on FastDDS and CycloneDDS: 5/5 pass - rviz2 Jazzy (rmw_fastrtps_cpp, rmw_cyclonedds_cpp) discovers topics with correct type hashes; no "Failed to parse type hash" warnings
|
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would update our CHANGELOG.md based on your changes. |
|
Hello @LuisPovedaCano, This PR is trying to fix the issues #9674 and #9675. |
There was a problem hiding this comment.
Tested on Jazzy with rmw_cyclonedds_cpp.
RViz2 subscribes to topics cleanly with zero Failed to parse type hash warnings
Great work!
Just two minor changes.
- In
FastDDSSubscriberMiddleware.h, the class doc comment ends mid-sentence in line 40See FastDDSPublisherMiddleware.h for the - The doc comment at the top of
CdrTopicInfo.h(lines 59–61) explicitly says:
/// Returns nullptr for CARLA-specific types whose
/// canonical hash depends on the carla_msgs package
/// IDL version — callers must skip setting user_data.
But CarlaCollisionEvent, CarlaEgoVehicleControl, and CarlaLineInvasion all have hardcoded hashes anyway, which contradicts that policy. It's not necessarily wrong, those definitions are stable and CARLA controls them, but either the comment should be updated to reflect that hardcoding carla_msgs hashes is acceptable, or the hashes should be removed and nullptr returned for those three.
As-is it's a bit confusing for anyone adding a new carla_msgs type in the future.
|
Done! Documentation texts are updated. |
…cipant for Jazzy compatibility (carla-simulator#9681) * fix(LibCarla/ros2): add REP-2011 type hashes and shared FastDDS participant for Jazzy compatibility Addresses two ROS 2 Jazzy interoperability issues: - carla-simulator#9674: ROS 2 Iron+ RMWs log "Failed to parse type hash for topic 'rt/...'" because no USER_DATA QoS is set on CARLA's DDS endpoints. Every publisher and subscriber (FastDDS and CycloneDDS) now advertises a REP-2011 RIHS01 type hash in the REP-2016 KV payload "typehash=RIHS01_<hex>;" (PID_USER_DATA = 0x002c per OMG DDSI-RTPS v2.5 9.6.2.2.2), hardcoded per msg type because CARLA has no IDL code-generation step at build time. - carla-simulator#9675: FastDDS crashes with "Topic with name : rt/tf already exists" then "Signal 11" when multiple sensors on the same actor (or multiple actors) create publishers on the shared topic. Introduces a refcounted shared DomainParticipant (FastDDSSharedParticipant) that owns per-type registration refcounts, and routes create_topic duplicates through find_topic so the second and later endpoints reuse the existing topic instead of provoking FastDDS's duplicate-name error. Additional changes: - carla/multigpu/listener.cpp: treat boost::asio::error::operation_aborted as the normal shutdown path; stop logging "Secondary server: Operation canceled." when Listener::Stop cancels async_accept. - ROS2::Shutdown tears down publishers and tf publishers before subscribers so the shared participant refcount reaches zero in the documented order. - CycloneDDS publisher and subscriber middlewares also set USER_DATA from build_user_data_for<T>() for parity with FastDDS. - Util/ros2/compute_type_hash.sh: Docker-only helper to compute a RIHS01 hash for a new message type, no local ROS 2 required. - Docs/ros2/adding_message_types.md: 7-step guide covering .msg to POD, CDR serialization, hash computation, CdrTopicInfo registration, and regression tests. - test_type_hash.cpp: format and uniqueness tests for all registered types, plus UserDataFormat payload tests. Verified: - make LibCarla ARGS="--ros2": clean, zero warnings in changed files - libcarla_test_server_{debug,release}: 120/120 pass (includes 4 new UserDataFormat tests and 2 TypeHash tests) - libcarla_test_client_release: 56/56 pass - smoke.test_ros2 on FastDDS and CycloneDDS: 5/5 pass - rviz2 Jazzy (rmw_fastrtps_cpp, rmw_cyclonedds_cpp) discovers topics with correct type hashes; no "Failed to parse type hash" warnings * docs(ros2): clarify type_hash contract and complete subscriber doc comment
Introduce the middleware-neutral type layer of the ROS 2 middleware decoupling series, ported from ue4-dev: - types/msg/*.h: 31 plain C++ structs, one per ROS 2 message type, no DDS dependency, standard library headers only, all members value-initialized (upstream ue4-dev #9612). - types/CdrSerialization.h: serialize_to_cdr(), deserialize_from_cdr() and cdr_serialized_size() for all msg::* types using Fast-CDR (classic CDR, little-endian, DDS encapsulation header). The buffers are wire-compatible with every ROS 2 distribution and can be handed directly to FastDDS write() paths or CycloneDDS dds_writecdr(), removing the need for per-vendor generated type files. A kMaxCdrSequenceElements cap rejects hostile sequence lengths during deserialization (upstream ue4-dev #9643). - types/CdrTopicInfo.h: per-type type_name(), REP-2011 RIHS01 type hash and max_serialized_size(); the hashes let ROS 2 Iron and newer RMWs parse the type hash CARLA advertises via USER_DATA (upstream ue4-dev #9681). - types/UserDataFormat.h: build_user_data() / build_user_data_for<T>() helpers producing the REP-2016 "typehash=RIHS01_<hex>;" key-value payload (upstream ue4-dev #9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so CdrSerialization.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) instead of the Fast-CDR 2.x forms the ue4-dev tip carries since its Fast-DDS 2.14.6 upgrade (ue4-dev #9789). Five lines differ; the wire format is identical either way and is pinned by the golden-bytes test added in the follow-up test commit. The FastDDSConversions.h / FastDDSTypeMap.h files from #9612 are deliberately not ported; they were superseded by unified CDR upstream. The new headers are not referenced by any build target yet; they start compiling when the middleware abstraction lands in the next PR of the series. (adapted from ue4-dev 542959a) (adapted from ue4-dev f53144c) (adapted from ue4-dev c64e8f4) (adapted from ue4-dev b865088) (cherry picked from commit 5d0d578)
Introduce the vendor-neutral middleware strategy layer of the ROS 2 middleware decoupling series and its FastDDS implementation, compiled into libcarla-ros2-native.so. Ported from ue4-dev: - middleware/Middleware.h: the Middleware enum plus the string and ROS 2 type-name helpers; the CycloneDDS value and its availability branches are present but stay compiled out until the CycloneDDS middleware lands (upstream ue4-dev #9608). - middleware/IPublisherMiddleware.h, ISubscriberMiddleware.h: the type-erased publisher/subscriber strategy interfaces. Subscribers write received samples straight into caller-owned storage to avoid a copy (upstream ue4-dev #9608). - middleware/MiddlewareFactory.h: creates the active middleware for a traits type; each vendor arm is double-gated on its CARLA_ROS2_MIDDLEWARE_* macro and CARLA_ROS2_MIDDLEWARE_TESTING so the suite exercises the availability logic without linking DDS (upstream ue4-dev #9608). - middleware/ActiveMiddleware.{h,cpp}: a DDS-free bridge (SetActiveMiddleware) so ROS2.cpp, the only ROS 2 translation unit in carla-server, can select the middleware without any DDS header crossing the shared-library boundary. Nothing calls it until the cutover; the definition ships now so the shared lib has a translation unit that compiles MiddlewareFactory.h and the FastDDS headers with the real vendor macros. - middleware/fastdds/GenericCdrPubSubType.h: one FastDDS TopicDataType that serializes every carla::ros2::msg::* struct through the unified CdrSerialization.h path, replacing the generated per-type PubSubType classes; getSerializedSizeProvider reports the actual instance size so variable-length payloads (camera frames, point clouds) are not bounded by the static max size (upstream ue4-dev #9643). - middleware/fastdds/FastDDS{Publisher,Subscriber}Middleware.h: the FastDDS strategy implementations. Each endpoint advertises the REP-2016 "typehash=RIHS01_<hex>;" USER_DATA so Jazzy RMWs match on the REP-2011 type hash (upstream ue4-dev #9681). - middleware/fastdds/FastDDSSharedParticipant.{h,cpp}: a refcounted process-wide DomainParticipant shared across all FastDDS endpoints, avoiding the discovery storm that destroying N participants back to back caused on shutdown (upstream ue4-dev #9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so GenericCdrPubSubType.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) matching CdrSerialization.h from the previous PR of the series. The wire format is classic CDR little-endian and is unchanged. The existing publishers keep using the generated FastDDS types; the cutover to this abstraction and the PublisherImpl/SubscriberImpl rewrite land in the next PR of the series. No behavior change. (adapted from ue4-dev 82c28e2) (adapted from ue4-dev f53144c) (adapted from ue4-dev 02a83ef) (adapted from ue4-dev c64e8f4) (cherry picked from commit f6d9a5b)
Description
Fixes two ROS 2 Jazzy interoperability problems in the native DDS integration.
1. Missing REP-2011 type hash on every topic (#9674)
ROS 2 Iron and later RMWs (
rmw_fastrtps_cpp,rmw_cyclonedds_cpp) parse the DDSUSER_DATAQoS field during SEDP/PDP endpoint discovery and expect a REP-2016 KV payload of the formtypehash=RIHS01_<64-hex>;. When the payload is absent, the RMW logs for every discovered endpoint:CARLA did not set
USER_DATAon any of its 31 message types, so running a Jazzy subscriber (rviz2, anyros2 topic echo, any autonomy stack) against a CARLA server produced a warning per topic per endpoint, and prevented future strict type-hash-based endpoint matching.Fix: hardcode a REP-2011 RIHS01 hash per message type in
CdrTopicInfo<T>::type_hash()and attach it via a newUserDataFormat.h::build_user_data_for<T>()helper on everyDataWriterandDataReaderQoS in both FastDDS and CycloneDDS middlewares. Hashes are hardcoded because CARLA has no IDL code-generation step at build time and recomputing RIHS01 correctly requiresrosidl_generator_type_descriptioninside a ROS 2 build environment. The values are stable per distro for every standard message (std_msgs,geometry_msgs,sensor_msgs,nav_msgs,builtin_interfaces,tf2_msgs,rosgraph_msgs,ackermann_msgs) and match on Humble and Jazzy.A helper script
Util/ros2/compute_type_hash.shcomputes the RIHS01 hash for any message type using only Docker, so adding a new type needs no local ROS 2 install.2. FastDDS
rt/tfduplicate-topic crash (#9675)Since the shared
DomainParticipantwas introduced, multiple sensor publishers on the same actor (or multiple hero actors) all callcreate_topic("rt/tf", ...)on the same participant. FastDDS rejects duplicate topic creation and aborts the server:Fix: introduce
FastDDSSharedParticipant, a refcounted sharedDomainParticipantwith per-type registration refcount, and route duplicate topic names throughlookup_topicdescription+find_topicso the second and later endpoints reuse the existing topic (balanced bydelete_topicin the destructor, which decrements FastDDS's internal topic refcount). CycloneDDS is unaffected (it tolerates duplicatedds_create_topicwhen types match) but the shared-participant lifetime now drives both middlewares for parity.The shutdown order in
ROS2::Shutdown()also changes: publishers and TF publishers are destroyed before subscribers soDataWriterunregister/dispose messages are sent while the shared participant is still alive.3. Additional hardening
carla/multigpu/listener.cpp: treatboost::asio::error::operation_abortedas the normal shutdown path instead of loggingSecondary server: Operation canceled.Every graceful CARLA shutdown previously emitted that spurious line.Docs/ros2/adding_message_types.md: 7-step guide covering.msgto POD, CDR serialization, RIHS01 computation,CdrTopicInforegistration, and regression tests. Includes a workedsensor_msgs/msg/Imuside-by-side example of the.msgand its POD struct.Files by area
LibCarla/source/carla/ros2/types/CdrTopicInfo.h,LibCarla/source/carla/ros2/types/UserDataFormat.hLibCarla/source/carla/ros2/dds/fastdds/FastDDSSharedParticipant.{h,cpp},LibCarla/source/carla/ros2/dds/fastdds/FastDDSPublisherMiddleware.h,LibCarla/source/carla/ros2/dds/fastdds/FastDDSSubscriberMiddleware.hLibCarla/source/carla/ros2/dds/cyclonedds/CycloneDDSPublisherMiddleware.h,LibCarla/source/carla/ros2/dds/cyclonedds/CycloneDDSSubscriberMiddleware.hLibCarla/source/carla/ros2/ROS2.cppLibCarla/source/carla/multigpu/listener.cppLibCarla/cmake/ros2/CMakeLists.txt,LibCarla/cmake/fast_dds/CMakeLists.txtLibCarla/source/test/server/test_type_hash.cpp(format, uniqueness, USER_DATA payload)Util/ros2/compute_type_hash.sh,Docs/ros2/adding_message_types.mdFixes #9674
Fixes #9675
Where has this been tested?
Results:
make LibCarla ARGS="--ros2": clean, zero warnings in changed files.libcarla_test_server_release: 120 / 120 pass (includes 4UserDataFormat.*tests and 2TypeHash.*tests).libcarla_test_server_debug: 120 / 120 pass.libcarla_test_client_release: 56 / 56 pass.smoke.test_ros2on port 3654 with--dds-middleware=fastddsand--dds-middleware=cyclonedds: 5 / 5 pass each. FastDDS log is clean ofrt/tf already exists,Failed to create Topic, andSignal 11. CycloneDDS log is clean ofSecondary server: Operation canceled.osrf/ros:jazzy-desktop) withRMW_IMPLEMENTATION=rmw_fastrtps_cppandRMW_IMPLEMENTATION=rmw_cyclonedds_cpp, against CARLA running either middleware: topics discover, type hashes parse, noFailed to parse type hashwarnings.Possible Drawbacks
Adding a new ROS 2 message type now requires a one-time RIHS01 hash computation through
Util/ros2/compute_type_hash.sh. The 7-step workflow is documented inDocs/ros2/adding_message_types.md. No runtime dependency on ROS 2 orrosidlis introduced.Hashes are hardcoded per message definition. If a standard ROS 2 message definition ever changes between distros (none of the types currently in CARLA have changed between Humble and Jazzy), the hash for that type must be updated in
CdrTopicInfo.h.test_type_hash.cppvalidates format and uniqueness but cannot catch a stale value versus an upstream redefinition.The refcounted shared participant is FastDDS-only; CycloneDDS uses its own shared participant via
dds_create_participant. The two are independent and neither affects the other.This change is