AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

modal

Modal is a serverless cloud platform for running Python on demand, including on-demand GPUs. Use when deploying or serving AI/ML models, running GPU-accelerated workloads (training, fine-tuning, inference), serving web endpoints, scheduling batch jobs, or scaling Python code to cloud containers with the Modal SDK.

Original source
K-Dense-AI
License
MIT
Pinned revision
5f2bc0341354fd5590aef76a533ada1e1e5b4e62

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read k-dense-ai--modal at revision 5f2bc0341354fd5590aef76a533ada1e1e5b4e62, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 13

references/resources.md

Modal Resource Configuration

Reviewed against SDK 1.6.0 and the official resource guide and timeout guide. Examples are configuration fragments; no remote allocation or billing was measured.

CPU

Requesting CPU

@app.function(cpu=4.0)
def compute():
    ...
  • Values are physical cores, not vCPUs
  • Default: 0.125 cores
  • Modal auto-sets OPENBLAS_NUM_THREADS, OMP_NUM_THREADS, MKL_NUM_THREADS based on your CPU request

CPU Limits

  • Default soft limit: 16 physical cores above the CPU request
  • Set explicit limits to prevent noisy-neighbor effects:
@app.function(cpu=(1.0, 4.0))  # Request 1 core, soft limit 4 cores
def bounded_compute():
    ...

Memory

Requesting Memory

@app.function(memory=16384)  # 16 GiB in MiB
def large_data():
    ...
  • Value in MiB (mebibytes)
  • Default: 128 MiB

Memory Limits

Set hard memory limits to OOM-kill containers that exceed them:

@app.function(memory=(8192, 16384))  # 8 GiB request, 16 GiB hard limit
def bounded_memory():
    ...

This prevents paying for runaway memory leaks. A scalar memory=8192 is a request, not an explicit hard limit. Requests/limits are validated against platform capacity; do not assume a requested allocation is available.

Ephemeral Disk

For temporary storage within a container's lifetime:

@app.function(ephemeral_disk=102400)  # 100 GiB in MiB
def process_dataset():
    # Temporary files at /tmp or anywhere in the container filesystem
    ...
  • Value in MiB
  • Default: 512 GiB quota per container
  • Maximum: 3,145,728 MiB (3 TiB)
  • Data is lost when the container shuts down
  • Use Volumes for persistent storage

Larger disk requests increase the memory request at a 20:1 ratio for billing purposes.

Timeout

@app.function(timeout=3600)  # 1 hour in seconds
def long_running():
    ...
  • Default: 300 seconds (5 minutes)
  • Maximum: 86,400 seconds (24 hours)
  • Execution timeout is per attempt; configured retries get a fresh timeout
  • Queue time is excluded; startup_timeout controls startup independently
  • Expiry is approximate and eventually raises modal.exception.FunctionTimeoutError

Billing

You are charged based on whichever is higher: your resource request or actual usage.

| Resource | Billing Basis | |----------|--------------| | CPU | max(requested, used) | | Memory | max(requested, used) | | GPU | Time GPU is allocated | | Disk | Increases memory billing at 20:1 ratio |

Cost Optimization Tips

  • Request only what you need
  • Use appropriate GPU tiers (L40S over H100 for inference)
  • Set scaledown_window to minimize idle time
  • Use min_containers=0 when cold starts are acceptable
  • Batch inputs with .map() instead of individual .remote() calls

Complete Example

@app.function(
    cpu=8.0,              # 8 physical cores
    memory=32768,         # 32 GiB
    gpu="L40S",           # L40S GPU
    ephemeral_disk=204800, # 200 GiB temp disk
    timeout=7200,         # 2 hours
    max_containers=50,
    min_containers=1,
)
def full_pipeline(data_path: str):
    ...