AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

autoresearch-agent

Use when a file should be improved against a measurable metric (speed, bundle size, pass rate, prompt or headline score), before hand-tuning it. Loops edit, run_experiment.py, keep or git reset on a fixed evaluator; needs an eval command and a git repo.

Original source
alirezarezvani
License
MIT
Pinned revision
e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

This skill also requires local files or dependencies. Remote reading alone may not be enough.

Ask your assistant

Use Agentic Memory skills to read alirezarezvani--autoresearch-agent at revision e4a1dc3695908dd5a6c811cc76ab7aa4cf95f2f0, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 3

references/experiment-domains.md

Experiment Domains Guide

Domain: Engineering

Code Speed Optimization

python scripts/setup_experiment.py \
  --domain engineering \
  --name api-speed \
  --target src/api/search.py \
  --eval "python -m pytest tests/bench_search.py --tb=no -q" \
  --metric p50_ms \
  --direction lower \
  --evaluator benchmark_speed

What the agent optimizes: Algorithm, data structures, caching, query patterns, I/O. Cost: Free — just runs benchmarks. Speed: ~5 min/experiment, ~12/hour, ~100 overnight.

Bundle Size Reduction

python scripts/setup_experiment.py \
  --domain engineering \
  --name bundle-size \
  --target webpack.config.js \
  --eval "npm run build && python .autoresearch/engineering/bundle-size/evaluate.py" \
  --metric size_bytes \
  --direction lower \
  --evaluator benchmark_size

Edit evaluate.py to set TARGET_FILE = "dist/main.js" and add BUILD_CMD = "npm run build".

Test Pass Rate

python scripts/setup_experiment.py \
  --domain engineering \
  --name fix-flaky-tests \
  --target src/utils/parser.py \
  --eval "python .autoresearch/engineering/fix-flaky-tests/evaluate.py" \
  --metric pass_rate \
  --direction higher \
  --evaluator test_pass_rate

Docker Build Speed

python scripts/setup_experiment.py \
  --domain engineering \
  --name docker-build \
  --target Dockerfile \
  --eval "python .autoresearch/engineering/docker-build/evaluate.py" \
  --metric build_seconds \
  --direction lower \
  --evaluator build_speed

Memory Optimization

python scripts/setup_experiment.py \
  --domain engineering \
  --name memory-usage \
  --target src/processor.py \
  --eval "python .autoresearch/engineering/memory-usage/evaluate.py" \
  --metric peak_mb \
  --direction lower \
  --evaluator memory_usage

ML Training (Karpathy-style)

Requires NVIDIA GPU. See autoresearch.

python scripts/setup_experiment.py \
  --domain engineering \
  --name ml-training \
  --target train.py \
  --eval "uv run train.py" \
  --metric val_bpb \
  --direction lower \
  --time-budget 5

Domain: Marketing

Medium Article Headlines

python scripts/setup_experiment.py \
  --domain marketing \
  --name medium-ctr \
  --target content/titles.md \
  --eval "python .autoresearch/marketing/medium-ctr/evaluate.py" \
  --metric ctr_score \
  --direction higher \
  --evaluator llm_judge_content

Edit evaluate.py: set TARGET_FILE = "content/titles.md" and CLI_TOOL = "claude".

What the agent optimizes: Title phrasing, curiosity gaps, specificity, emotional triggers. Cost: Uses your CLI subscription (Claude Max = unlimited). Speed: ~2 min/experiment, ~30/hour.

Social Media Copy

python scripts/setup_experiment.py \
  --domain marketing \
  --name twitter-engagement \
  --target social/tweets.md \
  --eval "python .autoresearch/marketing/twitter-engagement/evaluate.py" \
  --metric engagement_score \
  --direction higher \
  --evaluator llm_judge_copy

Edit evaluate.py: set PLATFORM = "twitter" (or linkedin, instagram).

Email Subject Lines

python scripts/setup_experiment.py \
  --domain marketing \
  --name email-open-rate \
  --target emails/subjects.md \
  --eval "python .autoresearch/marketing/email-open-rate/evaluate.py" \
  --metric engagement_score \
  --direction higher \
  --evaluator llm_judge_copy

Edit evaluate.py: set PLATFORM = "email".

Ad Copy

python scripts/setup_experiment.py \
  --domain marketing \
  --name ad-copy-q2 \
  --target ads/google-search.md \
  --eval "python .autoresearch/marketing/ad-copy-q2/evaluate.py" \
  --metric engagement_score \
  --direction higher \
  --evaluator llm_judge_copy

Edit evaluate.py: set PLATFORM = "ad".


Domain: Content

Article Structure & Readability

python scripts/setup_experiment.py \
  --domain content \
  --name article-structure \
  --target drafts/my-article.md \
  --eval "python .autoresearch/content/article-structure/evaluate.py" \
  --metric ctr_score \
  --direction higher \
  --evaluator llm_judge_content

SEO Descriptions

python scripts/setup_experiment.py \
  --domain content \
  --name seo-meta \
  --target seo/descriptions.md \
  --eval "python .autoresearch/content/seo-meta/evaluate.py" \
  --metric ctr_score \
  --direction higher \
  --evaluator llm_judge_content

Domain: Prompts

System Prompt Optimization

python scripts/setup_experiment.py \
  --domain prompts \
  --name support-bot \
  --target prompts/support-system.md \
  --eval "python .autoresearch/prompts/support-bot/evaluate.py" \
  --metric quality_score \
  --direction higher \
  --evaluator llm_judge_prompt

Requires tests/cases.json with test inputs and expected outputs:

[
  {
    "input": "I can't log in to my account",
    "expected": "Ask for email, check account status, offer password reset"
  },
  {
    "input": "How do I cancel my subscription?",
    "expected": "Empathetic response, explain cancellation steps, offer retention"
  }
]

Agent Skill Optimization

python scripts/setup_experiment.py \
  --domain prompts \
  --name skill-improvement \
  --target SKILL.md \
  --eval "python .autoresearch/prompts/skill-improvement/evaluate.py" \
  --metric quality_score \
  --direction higher \
  --evaluator llm_judge_prompt

Choosing Your Domain

| I want to... | Domain | Evaluator | Cost | |-------------|--------|-----------|------| | Speed up my code | engineering | benchmark_speed | Free | | Shrink my bundle | engineering | benchmark_size | Free | | Fix flaky tests | engineering | test_pass_rate | Free | | Speed up Docker builds | engineering | build_speed | Free | | Reduce memory usage | engineering | memory_usage | Free | | Train ML models | engineering | (custom) | Free + GPU | | Write better headlines | marketing | llm_judge_content | Subscription | | Improve social posts | marketing | llm_judge_copy | Subscription | | Optimize email subjects | marketing | llm_judge_copy | Subscription | | Improve ad copy | marketing | llm_judge_copy | Subscription | | Optimize article structure | content | llm_judge_content | Subscription | | Improve SEO descriptions | content | llm_judge_content | Subscription | | Optimize system prompts | prompts | llm_judge_prompt | Subscription | | Improve agent skills | prompts | llm_judge_prompt | Subscription |

First time? Start with an engineering experiment (free, fast, measurable). Once comfortable, try content/marketing with LLM judges.