Summary
When a CUDA kernel symbol is missing from a loaded module, the resulting error
does not say which kernel. CudaDevice::get_or_load_func takes the name as an
argument and discards it on the failure path:
// candle-core/src/cuda_backend/device.rs
pub fn get_or_load_func(&self, fn_name: &str, mdl: &kernels::Module) -> Result<CudaFunc> {
let ms = self.modules.read().unwrap();
if let Some(mdl) = ms.mdls[mdl.index()].as_ref() {
let func = mdl.load_function(fn_name).w()?; // <- fn_name dropped
...
}
drop(ms);
let mut ms = self.modules.write().unwrap();
let cuda_module = self.context.load_module(mdl.ptx().into()).w()?;
ms.mdls[mdl.index()] = Some(cuda_module.clone());
let func = cuda_module.load_function(fn_name).w()?; // <- and here
...
}
.w() wraps cudarc's DriverError unchanged, so the caller gets:
DriverError(CUDA_ERROR_NOT_FOUND, "named symbol not found")
The name that would make this actionable was in scope at the point of failure
and was thrown away.
Why this is worse than it looks
Two things conspire to make it hard to recover the information afterwards.
A backtrace does not identify the kernel. The error propagates upward as a
value through ?; the eventual panic happens at whatever unwrap/expect
consumes it, often several layers up in user code. RUST_BACKTRACE=1 names
that consumer, not the get_or_load_func call site. The natural first
debugging step yields nothing.
A missing symbol is a legitimate, reachable state, not a corrupt build.
candle-kernels guards kernels by target architecture, and the guards are not
uniform across a dtype. For F8E4M3 specifically, the casts sit under one
threshold and the elementwise and indexing ops under a higher one:
| kernels |
guard |
cast_f8_e4m3_f32, cast_f8_e4m3_bf16, … (cast.cu) |
__CUDA_ARCH__ >= 800 |
affine_f8_e4m3 (affine.cu) |
__CUDA_ARCH__ >= 890 |
is_*_f8_e4m3, gather_*_f8_e4m3, scatter_add_f8 (indexing.cu) |
__CUDA_ARCH__ >= 890 |
So a build for, say, sm_86 has a genuinely partial F8E4M3 kernel set: some
operations on such a tensor work and others fail at lookup time. That is a
reasonable design — but it means "symbol not found" is a routine outcome of
running a supported dtype on a supported device, and the one thing the user
needs in order to understand it is the name that is currently discarded.
Concretely: we hit CUDA_ERROR_NOT_FOUND while building a model on an
sm_86 device and, after reading the guards above and tracing the load path by
hand, still cannot say which kernel is missing. That is the entire content of
this report — not that the lookup failed, but that a failed lookup is
undiagnosable from its own error.
Why not fix this in cudarc
DriverError is a newtype over the raw driver result:
pub struct DriverError(pub sys::CUresult);
It has a public field, and its std::error::Error impl is behind
#[cfg(feature = "std")]. Carrying a symbol name would require an owned
String — allocation in a no_std-friendly FFI binding, a change to a
pattern-matchable public shape, and a breaking change across every API that
returns it. cudarc already surfaces the driver's own description; the caller's
identifier is the caller's to attach.
Shape of the fix
Attach the name at both load_function sites, and the module with it, since
"which module was searched" is the natural follow-up question:
cuda: kernel `affine_f8_e4m3` not found in module `affine`
(this kernel may not be compiled for the target architecture)
The parenthetical matters as much as the name: for a user on a pre-sm_89
device, "not found" reads as a broken installation, when it is in fact the
architecture guard doing its job.
Acceptance
- A failed lookup names the kernel and the module it was searched in.
- The underlying
DriverError remains in the source chain, so
CUDA_ERROR_NOT_FOUND is still distinguishable from other driver failures.
- Both call sites are covered — the cached-module path and the
load-then-lookup path fail identically today and must keep doing so.
Summary
When a CUDA kernel symbol is missing from a loaded module, the resulting error
does not say which kernel.
CudaDevice::get_or_load_functakes the name as anargument and discards it on the failure path:
.w()wraps cudarc'sDriverErrorunchanged, so the caller gets:The name that would make this actionable was in scope at the point of failure
and was thrown away.
Why this is worse than it looks
Two things conspire to make it hard to recover the information afterwards.
A backtrace does not identify the kernel. The error propagates upward as a
value through
?; the eventual panic happens at whateverunwrap/expectconsumes it, often several layers up in user code.
RUST_BACKTRACE=1namesthat consumer, not the
get_or_load_funccall site. The natural firstdebugging step yields nothing.
A missing symbol is a legitimate, reachable state, not a corrupt build.
candle-kernelsguards kernels by target architecture, and the guards are notuniform across a dtype. For
F8E4M3specifically, the casts sit under onethreshold and the elementwise and indexing ops under a higher one:
cast_f8_e4m3_f32,cast_f8_e4m3_bf16, … (cast.cu)__CUDA_ARCH__ >= 800affine_f8_e4m3(affine.cu)__CUDA_ARCH__ >= 890is_*_f8_e4m3,gather_*_f8_e4m3,scatter_add_f8(indexing.cu)__CUDA_ARCH__ >= 890So a build for, say,
sm_86has a genuinely partialF8E4M3kernel set: someoperations on such a tensor work and others fail at lookup time. That is a
reasonable design — but it means "symbol not found" is a routine outcome of
running a supported dtype on a supported device, and the one thing the user
needs in order to understand it is the name that is currently discarded.
Concretely: we hit
CUDA_ERROR_NOT_FOUNDwhile building a model on ansm_86device and, after reading the guards above and tracing the load path byhand, still cannot say which kernel is missing. That is the entire content of
this report — not that the lookup failed, but that a failed lookup is
undiagnosable from its own error.
Why not fix this in cudarc
DriverErroris a newtype over the raw driver result:It has a public field, and its
std::error::Errorimpl is behind#[cfg(feature = "std")]. Carrying a symbol name would require an ownedString— allocation in ano_std-friendly FFI binding, a change to apattern-matchable public shape, and a breaking change across every API that
returns it. cudarc already surfaces the driver's own description; the caller's
identifier is the caller's to attach.
Shape of the fix
Attach the name at both
load_functionsites, and the module with it, since"which module was searched" is the natural follow-up question:
The parenthetical matters as much as the name: for a user on a pre-
sm_89device, "not found" reads as a broken installation, when it is in fact the
architecture guard doing its job.
Acceptance
DriverErrorremains in the source chain, soCUDA_ERROR_NOT_FOUNDis still distinguishable from other driver failures.load-then-lookup path fail identically today and must keep doing so.