references/numba.md · 24000
On-Device Debug Mode
@cuda.jit(debug=True, opt=False)
def kernel_debug(data):
# Enables CUDA exception checking — much slower but catches errors
...
Interoperability
Numba supports the CUDA Array Interface (version 3) — any object exposing __cuda_array_interface__ can be passed directly to Numba kernels with zero copy.
With CuPy
import cupy as cp
from numba import cuda
@cuda.jit
def add_kernel(x, y, out):
i = cuda.grid(1)
if i < x.shape[0]:
out[i] = x[i] + y[i]
# CuPy arrays work directly — zero copy
a = cp.arange(1000, dtype=cp.float32)
b = cp.ones(1000, dtype=cp.float32)
out = cp.zeros(1000, dtype=cp.float32)
add_kernel[4, 256](a, b, out)
With PyTorch
import torch
from numba import cuda
t = torch.tensor([1, 2, 3], dtype=torch.float32, device="cuda")
d_array = cuda.as_cuda_array(t) # Zero-copy Numba view of PyTorch tensor
Checking GPU Arrays
cuda.is_cuda_array(obj) # True if obj has __cuda_array_interface__
cuda.as_cuda_array(obj) # Wrap as Numba device array (zero copy)
Compatibility is object-specific: check cuda.is_cuda_array(obj). Convert JAX or TensorFlow through a supported DLPack consumer such as CuPy instead of assuming every tensor exposes this interface.
Common Pitfalls
-
Forgetting bounds checks. If
blocks * threads > array_size, out-of-bounds threads corrupt memory silently. Always:if i < array.size. -
Trying to return values from kernels. Kernels cannot return — write to output arrays instead. Return values are silently discarded.
-
Implicit synchronous transfers. Passing host (NumPy) arrays directly to kernels triggers synchronous copy-back. Use explicit
cuda.to_device()/copy_to_host(). -
Shared memory size must be a compile-time constant for static allocation. Use dynamic shared memory (size=0) for runtime-determined sizes.
-
Dynamic shared memory aliasing. All
cuda.shared.array(0, ...)in the same kernel share the same memory. Slice manually for multiple arrays. -
syncthreads()in divergent branches. All threads in a block must reach the samesyncthreads()call. Divergent paths → undefined behavior. -
Atomic operation type restrictions.
atomic.addsupports int32, float32, float64. Bitwise atomics only work on integer types. -
Forgetting
cuda.synchronize(). Kernel launches are async. Reading host-side results before sync gives stale/incomplete data. -
Unsupported Python features in kernels. No dynamic allocation, no Python objects, no string operations, no exceptions (unless debug mode). Stick to numeric types and math.
-
Using float64 on consumer GPUs. Consumer NVIDIA GPUs (GeForce) have heavily throttled float64 throughput (often 1/32 of float32). Use float32 unless you need the precision.