foundry-agent/deploy/deploy.md · 24000
r generation jobs server-side.
- Returns in seconds.
- Writes pending operation IDs to local azd state.
- Writes a placeholder
eval.yamlat the agent root (override with--out-file <path>).
No skill-side polling, terminal handle, or later-turn re-check is needed. azd ai agent eval run (section 3) automatically resumes a pending generation, downloads artifacts, finalizes eval.yaml, then runs the eval.
If the user wants to wait synchronously instead (e.g., to inspect eval.yaml before running), drop --no-wait — generate will then submit the jobs, wait for completion, download review artifacts, and write the finalized eval.yaml before returning (typically several minutes).
3. Run the suite
azd ai agent eval run
Use azd ai agent eval show -O results.json to inspect run details, or azd ai agent eval list to see history.
4. Refresh datasets/evaluators (later)
When local files under datasets/<suite>/ or evaluators/<suite>/ change, run azd ai agent eval update --dataset-only or --evaluator-only to upload new versions. azd bumps the version fields in eval.yaml.
5. Prompt User
"Your agent is deployed and evaluation suite generation is submitted server-side (still running, takes several minutes). Would you like to run an evaluation now? azd ai agent eval run will wait for generation to finish, then execute the eval."
- Yes → run
azd ai agent eval run(this resumes the pending generation, then runs the eval — may take several minutes the first time), then follow the observe skill to interpret results. - No → stop. The user can return later via
azd ai agent eval run— it will pick up wherever the pending generation is. - Production trace analysis → follow the trace skill.
Non-Interactive / YOLO Mode
Even in
--no-prompt/--yolomode: if the user named a foundry project or asked to create one, go ahead; otherwise stop and ask before provisioning.
- Hosted: always pass
--no-prompt. - Prompt: all required values (project endpoint, agent name, model deployment) must come from the user message or
azd env get-values; missing values should fail loudly rather than prompt.