Make AMQP 1.0 faster #14238
Merged
Make AMQP 1.0 faster #14238
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit results in great performance improvements for AMQP 1.0. Stream filtering via AMQP SQL Filters or AMQP Property Filters where the broker reads messages from the stream into memory and the filter returns false, i.e. messages are not sent to the client: Before this commit: ~400,000 msgs/s After this commit: ~500,000 msgs/s There is also a ~10% increase in end-to-end throughput for normal AMQP workloads, e.g. when sending to and receiving from a classic queue. Prior to this commit, a lot of garbage was created leading to many minor garbage collections. This commit reduces the garbage being generated. The new module amqp10_composite performs very well: * Single tuple update setting multiple fields at once, sometimes done even in-place without copying the tuple. * The list of fields is passed into X registers, no need for any new allocations. On the serialisation side, this commit also generates less garbage by: 1. Avoiding tuple_to_list/1 2. Omitting trailing elements of the list that are null This will also lead to fewer bytes per message sent on the wire and less resource usage for the clients as they need to parse fewer fields.
This commit fixes the failing test cases
```
make -C deps/rabbit ct-msg_size_metrics t=tests:message_size
make -C deps/amqp10_client ct-system t=rabbitmq:roundtrip_large_messages
```
The second test case failed with:
```
flush {amqp10_event,
{connection,<0.193.0>,
{closed,
{framing_error,
<<"frame size (131073 bytes) > maximum frame size (131072 bytes)">>}}}}
```
This commit fixes the following test case: ``` make -C deps/rabbit ct-amqp_dotnet t=cluster_size_1:fragmentation ``` Previously, the server sent a frame that was 1 byte too large.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR results in noteworthy performance improvements for AMQP 1.0 (depending on the workload).
Stream filtering via AMQP SQL Filters or AMQP Property Filters where the
broker reads messages from the stream into memory and the filter returns
false, i.e. messages are not sent to the client:
There is also a ~10% increase in end-to-end throughput for normal
AMQP workloads, e.g. when sending to and receiving from a classic queue.
Prior to this PR, a lot of garbage was created leading to many minor
garbage collections.
This PR reduces the garbage being generated.
The new module amqp10_composite performs very well:
without copying the tuple.
allocations.
On the serialisation side, this PR also generates less garbage by:
This will also lead to fewer bytes per message sent on the wire and less
resource usage for the clients as they need to parse fewer fields.