Skip to content

wire: Optimize message reads#3598

Merged
davecgh merged 1 commit intodecred:masterfrom
jrick:optimize_reads
Feb 25, 2026
Merged

wire: Optimize message reads#3598
davecgh merged 1 commit intodecred:masterfrom
jrick:optimize_reads

Conversation

@jrick
Copy link
Copy Markdown
Member

@jrick jrick commented Jan 8, 2026

This commit contains several performance and memory optimizations to improve message deserialization, both when generically called through the BtcDecode method of the Message interface and when reading wire protocol using ReadMessageN.

Special cases have been added for the bytes.Buffer and bytes.Reader reader types, avoiding the allocation and synchronization costs associated with using temporary buffers from the binary freelist and avoiding the heap allocations required due to the reader interface leaking the slice parameter.

ReadMessageN has reduced the number of allocations it must incur by avoiding the readElements helper function.

Finally, a new internal buffer type is introduced that is used exclusively by ReadMessageN. Since the lifetime and memory of this buffer is under the control of the wire package, and the buffer is never reset or truncated, this allows transaction script deserialization to slice the internal bytes of this buffer rather than pulling temporary buffers from the scriptPool freelist and copying these into a new contiguous allocation.

@jrick
Copy link
Copy Markdown
Member Author

jrick commented Jan 8, 2026

Using the newly added benchmark, before:

$ go test -run=^$ -bench=BenchmarkReadMessageN -benchmem 
goos: openbsd
goarch: amd64
pkg: github.com/decred/dcrd/wire
cpu: AMD Ryzen 7 5800X3D 8-Core Processor           
BenchmarkReadMessageN-8   	  334888	      3370 ns/op	     920 B/op	      15 allocs/op
PASS
ok  	github.com/decred/dcrd/wire	1.180s

After:

$ go test -run=^$ -bench=BenchmarkReadMessageN -benchmem 
goos: openbsd
goarch: amd64
pkg: github.com/decred/dcrd/wire
cpu: AMD Ryzen 7 5800X3D 8-Core Processor           
BenchmarkReadMessageN-8   	  537861	      1988 ns/op	    1432 B/op	       9 allocs/op
PASS
ok  	github.com/decred/dcrd/wire	1.106s

Copy link
Copy Markdown
Member

@davecgh davecgh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Nice changes overall. Saving several allocs on every single message read adds up to quite a few allocations over time, and will help to remove more GC pressure during the initial sync.

This commit contains several performance and memory optimizations to improve
message deserialization, both when generically called through the BtcDecode
method of the Message interface and when reading wire protocol using
ReadMessageN.

Special cases have been added for the bytes.Buffer and bytes.Reader reader
types, avoiding the allocation and synchronization costs associated with using
temporary buffers from the binary freelist and avoiding the heap allocations
required due to the reader interface leaking the slice parameter.

ReadMessageN has reduced the number of allocations it must incur by avoiding
the readElements helper function.

Finally, a new internal buffer type is introduced that is used exclusively by
ReadMessageN.  Since the lifetime and memory of this buffer is under the
control of the wire package, and the buffer is never reset or truncated, this
allows transaction script deserialization to slice the internal bytes of this
buffer rather than pulling temporary buffers from the scriptPool freelist and
copying these into a new contiguous allocation.
@davecgh davecgh merged commit 04fef0b into decred:master Feb 25, 2026
33 checks passed
@jrick jrick deleted the optimize_reads branch February 26, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants