references/resource-configuration.md
Task Resource Configuration
This reference targets latch==2.77.1. Resource shapes are operational
configuration and can change independently of examples. Inspect the installed
SDK before making cost or capacity guarantees. Task bodies below are illustrative
resource-configuration skeletons, not complete scientific implementations.
Selection Strategy
- Start with the smallest named decorator that can run the task.
- Measure CPU, peak RSS, temporary storage, wall time, and GPU utilization.
- Move to a larger named decorator only when evidence justifies it.
- Use
custom_taskfor CPU-only shapes not represented by a named decorator. - Use a named GPU decorator;
custom_taskdoes not acceptgpuorgpu_typearguments. - Re-measure with representative scientific data before release.
Named CPU Decorators
from latch.resources.tasks import large_task, medium_task, small_task
@small_task
def parse_manifest():
...
@medium_task
def align_reads():
...
@large_task
def assemble_genome():
...
SDK 2.77.1 configures these scheduler requests:
| Decorator | CPU | RAM | Ephemeral storage |
|---|---:|---:|---:|
| small_task | 2 | 4 GiB | 100 GiB |
| medium_task | 30 | 100 GiB | 1500 GiB |
| large_task | 90 | 170 GiB | 4500 GiB |
The public guide may display nominal instance capacities rather than the schedulable requests encoded in the package. The installed package determines what registration serializes.
Named GPU Decorators
Generic GPU shapes:
from latch.resources.tasks import large_gpu_task, small_gpu_task
| Decorator | CPU | RAM | GPU |
|---|---:|---:|---|
| small_gpu_task | 7 | 30 GiB | 1× T4-class |
| large_gpu_task | 63 | 245 GiB | 1× A10G-class |
V100 shapes:
from latch.resources.tasks import (
v100_x1_task,
v100_x4_task,
v100_x8_task,
)
L40S shapes:
from latch.resources.tasks import (
g6e_xlarge_task,
g6e_2xlarge_task,
g6e_4xlarge_task,
g6e_8xlarge_task,
g6e_12xlarge_task,
g6e_16xlarge_task,
g6e_24xlarge_task,
g6e_48xlarge_task,
)
Current L40S scheduler requests and limits:
| Decorator | Request CPU | Request RAM | Limit CPU | Limit RAM | GPUs |
|---|---:|---:|---:|---:|---:|
| g6e_xlarge_task | 2 | 28 GiB | 4 | 32 GiB | 1 |
| g6e_2xlarge_task | 6 | 57 GiB | 8 | 64 GiB | 1 |
| g6e_4xlarge_task | 14 | 115 GiB | 16 | 128 GiB | 1 |
| g6e_8xlarge_task | 30 | 230 GiB | 32 | 256 GiB | 1 |
| g6e_12xlarge_task | 46 | 345 GiB | 48 | 384 GiB | 4 |
| g6e_16xlarge_task | 62 | 460 GiB | 64 | 512 GiB | 1 |
| g6e_24xlarge_task | 94 | 691 GiB | 96 | 768 GiB | 4 |
| g6e_48xlarge_task | 190 | 1382 GiB | 192 | 1536 GiB | 8 |
GPU availability, quotas, and platform mapping can change. Confirm in the current docs or with Latch support before promising a specific model to users.
Custom CPU, Memory, and Storage
Exact stable signature:
custom_task(
cpu,
memory,
*,
storage_gib=500,
timeout=0,
**task_options,
)
Example:
from datetime import timedelta
from latch import custom_task
@custom_task(
cpu=12,
memory=48,
storage_gib=750,
timeout=timedelta(hours=6),
retries=1,
)
def call_variants():
...
In SDK 2.77.1, custom_task accepts up to 126 CPU cores, 975 GiB RAM, and
4949 GiB ephemeral storage. Not every arbitrary combination fits a schedulable
node group; the decorator selects the smallest configured group that satisfies
all three requests.
custom_memory_optimized_task is deprecated. Use custom_task.
Dynamic Resources
Each cpu, memory, or storage_gib argument may be a function of task
inputs. The resource function's annotated parameters must exist in the task
with exactly matching annotations.
from latch import custom_task
from latch.types import LatchFile
def allocate_cpu(files: list[LatchFile]) -> int:
return min(32, max(2, len(files) * 2))
def allocate_storage(files: list[LatchFile]) -> int:
sizes = [file.size() for file in files]
if any(size is None for size in sizes):
raise ValueError("unable to determine every input size")
total_bytes = sum(int(size) for size in sizes)
estimated_gib = total_bytes / (1024**3)
storage = max(100, int(estimated_gib * 2) + 1)
if storage > 4949:
raise ValueError("estimated storage exceeds the custom_task limit")
return storage
@custom_task(
cpu=allocate_cpu,
memory=64,
storage_gib=allocate_storage,
)
def merge_files(files: list[LatchFile]) -> LatchFile:
...
Dynamic functions execute at runtime before the task launches.
Guardrails:
- Return integers.
- Bound every computed resource.
- Include overhead for decompression and intermediate files.
- Keep computation deterministic and fast.
- Restrict remote lookups to necessary input metadata (such as
file.size()). Do not download inputs or retrieve secrets in a resource function. - Test the exact annotation matching during staging registration.
Cache, Retries, and Timeout
Named decorators and non-dynamic custom_task accept Flyte task options:
@medium_task(
cache=True,
cache_version="aligner-2.1-reference-v4",
retries=2,
timeout=7200,
)
def align_reads():
...
- Cache deterministic outputs only.
- Include tool, reference, and behavior changes in
cache_version. - Use retries for transient infrastructure or network errors.
- Do not retry deterministic invalid-input failures.
- A timeout can be seconds or
datetime.timedelta.
Temporary and Persistent Storage
Ephemeral task storage is deleted with the task environment. Use it for:
- Decompressed inputs
- Tool work directories
- Sort spills
- Intermediate indexes
Return a LatchFile or LatchDir, or upload through LPath, for persistent
outputs. Never assume /tmp or /root survives task completion.
Estimate storage from peak simultaneous intermediates, not final output size:
requested storage
>= staged inputs
+ decompressed expansion
+ peak tool intermediates
+ final outputs
+ safety margin
Cost and Launch Safety
- Surface the chosen resource class before registration or launch.
- Obtain confirmation before starting large or GPU-backed executions.
- Prefer a representative small launch plan for validation.
- Parallelism multiplies cost; map width matters as much as per-task size.
- Cache hits can reduce cost but are not a substitute for reproducible inputs.
- Compare resource monitoring against scientific throughput, not utilization alone.
Troubleshooting
Out of memory
- Check peak RSS, not average memory.
- Identify whether an algorithm scales with records, bases, or samples.
- Increase memory only after ruling out unbounded accumulation.
Out of storage
- Inspect hidden tool caches and decompressed inputs.
- Clean intermediates within the task when safe.
- Increase
storage_gibbased on peak use.
CPU under-utilization
- Confirm the scientific tool received its thread/process flag.
- Avoid requesting more cores than the tool can use.
- Check I/O and memory bandwidth before scaling CPU.
GPU under-utilization
- Verify the tool was built with the expected CUDA support.
- Check batch size, data loading, and CPU preprocessing.
- Avoid multi-GPU shapes unless the application implements distributed work.
Official Sources
- Resource guide: https://wiki.latch.bio/workflows/sdk/python/defining-cloud-resources
- Resource monitoring: https://wiki.latch.bio/workflows/sdk/console/resource-monitoring
- Task source in the 2.77.1 release commit: https://github.com/latchbio/latch/blob/b3768e65c6d496868f6e530f11977d857ad85dc7/src/latch/resources/tasks.py
- Changelog in the 2.77.1 release commit: https://github.com/latchbio/latch/blob/b3768e65c6d496868f6e530f11977d857ad85dc7/CHANGELOG.md