Skip to content

xrt::bo: a sub-buffer of a sub-buffer reports a device address that drops the outer offset #9952

Description

@atassis

A sub-buffer built on another sub-buffer composes its host pointer and its sync range through the
parent, but takes its device address from the underlying allocation, so only the innermost offset is
applied. The same object then names two different regions depending on which accessor you ask.

xrt::device dev{0};
auto root  = xrt::bo{dev, 4096, xrt::bo::flags::host_only, 0};
auto outer = xrt::bo{root, 2048, 1024};   // 1024 into root
auto inner = xrt::bo{outer,  512,  256};  // 1280 into root

npu4, XRT 2.21.75 (4eb1f43):

root at 0x7f22b4db2000 / 0x7f22b4db2000
  outer  host +1024   device +1024   expected +1024  ok
  inner  host +1280   device +256    expected +1280  MISMATCH

In src/runtime_src/core/common/api/xrt_bo.cpp, buffer_sub composes two of its three paths through
the parent and one through the allocation:

  • m_hbuf = m_parent->get_hbuf() + m_offset — composes.
  • sync() does off = offset + m_offset then m_parent->sync(dir, sz, off) — composes.
  • get_address() returns bo_impl::get_address() + m_offset, and bo_impl::get_address() reads
    paddr off handle, which bo_impl(const bo_impl* parent, size_t) copies from the parent. That
    handle is the root allocation's whatever the depth, so the result is root address plus the
    innermost offset only.

get_offset() has the same shape: it returns m_offset, and its consumers pair it with
bo_int::get_buffer_handle(), which is likewise the root handle.

Both are reachable from a kernel launch: an xrt::bo argument is encoded as bo.address()
(xrt_kernel.cpp, hs_arg_setter::set_arg_value), and bind_arg_at_index() passes
(get_buffer_handle(bo), get_offset(bo)). So a nested sub-buffer passed to a kernel has the device
writing one region while the host reads another.

alloc_sub() accepts any parent, so nesting is reachable from the public API with no diagnostic.

Two ways out: accumulate the offset from the root in buffer_sub, or reject a sub-buffer whose
parent is itself a sub-buffer. I am happy to send whichever you prefer.

Full reproducer
// A sub-buffer of a sub-buffer: does its device address land where its host
// pointer does?
//
//   root  4096 bytes
//   outer = xrt::bo(root,  2048, 1024)   ->  1024 into root
//   inner = xrt::bo(outer,  512,  256)   ->  1280 into root
//
// Build: g++ -std=c++17 nested_subbo_address.cpp -lxrt_coreutil -o nested_subbo_address

#include <cstddef>
#include <cstdio>

#include <xrt/xrt_bo.h>
#include <xrt/xrt_device.h>

int main()
{
  xrt::device dev{0};

  auto root  = xrt::bo{dev, 4096, xrt::bo::flags::host_only, 0};
  auto outer = xrt::bo{root, 2048, 1024};
  auto inner = xrt::bo{outer, 512, 256};

  auto *hbase = static_cast<char *>(root.map());
  auto dbase = root.address();

  auto row = [&](const char *name, xrt::bo &b, long want) {
    printf("  %-6s host +%-6ld device +%-6lu expected +%ld  %s\n", name,
           static_cast<long>(static_cast<char *>(b.map()) - hbase),
           static_cast<unsigned long>(b.address() - dbase), want,
           (b.address() - dbase) == static_cast<unsigned long>(want) ? "ok" : "MISMATCH");
  };

  printf("root at %p / 0x%lx\n", static_cast<void *>(hbase),
         static_cast<unsigned long>(dbase));
  row("outer", outer, 1024);
  row("inner", inner, 1280);

  return 0;
}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions