A sub-buffer built on another sub-buffer composes its host pointer and its sync range through the
parent, but takes its device address from the underlying allocation, so only the innermost offset is
applied. The same object then names two different regions depending on which accessor you ask.
xrt::device dev{0};
auto root = xrt::bo{dev, 4096, xrt::bo::flags::host_only, 0};
auto outer = xrt::bo{root, 2048, 1024}; // 1024 into root
auto inner = xrt::bo{outer, 512, 256}; // 1280 into root
npu4, XRT 2.21.75 (4eb1f43):
root at 0x7f22b4db2000 / 0x7f22b4db2000
outer host +1024 device +1024 expected +1024 ok
inner host +1280 device +256 expected +1280 MISMATCH
In src/runtime_src/core/common/api/xrt_bo.cpp, buffer_sub composes two of its three paths through
the parent and one through the allocation:
m_hbuf = m_parent->get_hbuf() + m_offset — composes.
sync() does off = offset + m_offset then m_parent->sync(dir, sz, off) — composes.
get_address() returns bo_impl::get_address() + m_offset, and bo_impl::get_address() reads
paddr off handle, which bo_impl(const bo_impl* parent, size_t) copies from the parent. That
handle is the root allocation's whatever the depth, so the result is root address plus the
innermost offset only.
get_offset() has the same shape: it returns m_offset, and its consumers pair it with
bo_int::get_buffer_handle(), which is likewise the root handle.
Both are reachable from a kernel launch: an xrt::bo argument is encoded as bo.address()
(xrt_kernel.cpp, hs_arg_setter::set_arg_value), and bind_arg_at_index() passes
(get_buffer_handle(bo), get_offset(bo)). So a nested sub-buffer passed to a kernel has the device
writing one region while the host reads another.
alloc_sub() accepts any parent, so nesting is reachable from the public API with no diagnostic.
Two ways out: accumulate the offset from the root in buffer_sub, or reject a sub-buffer whose
parent is itself a sub-buffer. I am happy to send whichever you prefer.
Full reproducer
// A sub-buffer of a sub-buffer: does its device address land where its host
// pointer does?
//
// root 4096 bytes
// outer = xrt::bo(root, 2048, 1024) -> 1024 into root
// inner = xrt::bo(outer, 512, 256) -> 1280 into root
//
// Build: g++ -std=c++17 nested_subbo_address.cpp -lxrt_coreutil -o nested_subbo_address
#include <cstddef>
#include <cstdio>
#include <xrt/xrt_bo.h>
#include <xrt/xrt_device.h>
int main()
{
xrt::device dev{0};
auto root = xrt::bo{dev, 4096, xrt::bo::flags::host_only, 0};
auto outer = xrt::bo{root, 2048, 1024};
auto inner = xrt::bo{outer, 512, 256};
auto *hbase = static_cast<char *>(root.map());
auto dbase = root.address();
auto row = [&](const char *name, xrt::bo &b, long want) {
printf(" %-6s host +%-6ld device +%-6lu expected +%ld %s\n", name,
static_cast<long>(static_cast<char *>(b.map()) - hbase),
static_cast<unsigned long>(b.address() - dbase), want,
(b.address() - dbase) == static_cast<unsigned long>(want) ? "ok" : "MISMATCH");
};
printf("root at %p / 0x%lx\n", static_cast<void *>(hbase),
static_cast<unsigned long>(dbase));
row("outer", outer, 1024);
row("inner", inner, 1280);
return 0;
}
A sub-buffer built on another sub-buffer composes its host pointer and its sync range through the
parent, but takes its device address from the underlying allocation, so only the innermost offset is
applied. The same object then names two different regions depending on which accessor you ask.
xrt::device dev{0}; auto root = xrt::bo{dev, 4096, xrt::bo::flags::host_only, 0}; auto outer = xrt::bo{root, 2048, 1024}; // 1024 into root auto inner = xrt::bo{outer, 512, 256}; // 1280 into rootnpu4, XRT 2.21.75 (4eb1f43):
In
src/runtime_src/core/common/api/xrt_bo.cpp,buffer_subcomposes two of its three paths throughthe parent and one through the allocation:
m_hbuf = m_parent->get_hbuf() + m_offset— composes.sync()doesoff = offset + m_offsetthenm_parent->sync(dir, sz, off)— composes.get_address()returnsbo_impl::get_address() + m_offset, andbo_impl::get_address()readspaddroffhandle, whichbo_impl(const bo_impl* parent, size_t)copies from the parent. Thathandle is the root allocation's whatever the depth, so the result is root address plus the
innermost offset only.
get_offset()has the same shape: it returnsm_offset, and its consumers pair it withbo_int::get_buffer_handle(), which is likewise the root handle.Both are reachable from a kernel launch: an
xrt::boargument is encoded asbo.address()(
xrt_kernel.cpp,hs_arg_setter::set_arg_value), andbind_arg_at_index()passes(get_buffer_handle(bo), get_offset(bo)). So a nested sub-buffer passed to a kernel has the devicewriting one region while the host reads another.
alloc_sub()accepts any parent, so nesting is reachable from the public API with no diagnostic.Two ways out: accumulate the offset from the root in
buffer_sub, or reject a sub-buffer whoseparent is itself a sub-buffer. I am happy to send whichever you prefer.
Full reproducer