AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

optimize-for-gpu

GPU-accelerates scientific Python on NVIDIA hardware and verifies that the result is correct and faster. Use for CUDA/GPU optimization; CPU-bound NumPy, SciPy, pandas, scikit-learn, NetworkX, scikit-image, vector-search, image-processing, graph, simulation, or file-I/O workloads; CuPy, cuDF, cuML, cuGraph, cuVS, cuCIM, KvikIO, Warp, Newton, Numba-CUDA, or RAFT questions; and profiling, memory-transfer, kernel, or multi-GPU bottlenecks. Also use when large data-parallel Python code is slow and GPU acceleration is a plausible option, even if the user does not name CUDA.

Original source
K-Dense-AI
License
MIT
Pinned revision
5f2bc0341354fd5590aef76a533ada1e1e5b4e62

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read k-dense-ai--optimize-for-gpu at revision 5f2bc0341354fd5590aef76a533ada1e1e5b4e62, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 16

references/numba.md · 24000

On-Device Debug Mode

@cuda.jit(debug=True, opt=False)
def kernel_debug(data):
    # Enables CUDA exception checking — much slower but catches errors
    ...

Interoperability

Numba supports the CUDA Array Interface (version 3) — any object exposing __cuda_array_interface__ can be passed directly to Numba kernels with zero copy.

With CuPy

import cupy as cp
from numba import cuda

@cuda.jit
def add_kernel(x, y, out):
    i = cuda.grid(1)
    if i < x.shape[0]:
        out[i] = x[i] + y[i]

# CuPy arrays work directly — zero copy
a = cp.arange(1000, dtype=cp.float32)
b = cp.ones(1000, dtype=cp.float32)
out = cp.zeros(1000, dtype=cp.float32)
add_kernel[4, 256](a, b, out)

With PyTorch

import torch
from numba import cuda

t = torch.tensor([1, 2, 3], dtype=torch.float32, device="cuda")
d_array = cuda.as_cuda_array(t)  # Zero-copy Numba view of PyTorch tensor

Checking GPU Arrays

cuda.is_cuda_array(obj)       # True if obj has __cuda_array_interface__
cuda.as_cuda_array(obj)       # Wrap as Numba device array (zero copy)

Compatibility is object-specific: check cuda.is_cuda_array(obj). Convert JAX or TensorFlow through a supported DLPack consumer such as CuPy instead of assuming every tensor exposes this interface.


Common Pitfalls

  1. Forgetting bounds checks. If blocks * threads > array_size, out-of-bounds threads corrupt memory silently. Always: if i < array.size.

  2. Trying to return values from kernels. Kernels cannot return — write to output arrays instead. Return values are silently discarded.

  3. Implicit synchronous transfers. Passing host (NumPy) arrays directly to kernels triggers synchronous copy-back. Use explicit cuda.to_device() / copy_to_host().

  4. Shared memory size must be a compile-time constant for static allocation. Use dynamic shared memory (size=0) for runtime-determined sizes.

  5. Dynamic shared memory aliasing. All cuda.shared.array(0, ...) in the same kernel share the same memory. Slice manually for multiple arrays.

  6. syncthreads() in divergent branches. All threads in a block must reach the same syncthreads() call. Divergent paths → undefined behavior.

  7. Atomic operation type restrictions. atomic.add supports int32, float32, float64. Bitwise atomics only work on integer types.

  8. Forgetting cuda.synchronize(). Kernel launches are async. Reading host-side results before sync gives stale/incomplete data.

  9. Unsupported Python features in kernels. No dynamic allocation, no Python objects, no string operations, no exceptions (unless debug mode). Stick to numeric types and math.

  10. Using float64 on consumer GPUs. Consumer NVIDIA GPUs (GeForce) have heavily throttled float64 throughput (often 1/32 of float32). Use float32 unless you need the precision.