AGENTIC MEMORY

Skills

Useful instructions for your AI. Search, read and use them through the same MCP connection.

← Back to Skills

backtesting-frameworks

Build robust backtesting systems for trading strategies with proper handling of look-ahead bias, survivorship bias, and transaction costs. Use when developing trading algorithms, validating strategies, or building backtesting infrastructure.

Original source
wshobson
License
MIT
Pinned revision
7e2fb559cf4b824b27f1fec49fe88f7511066d26

Third-party instructions in their original language. Reading does not install files, run code or grant permissions. Snapshots are updated only after review.

Ask your assistant

Use Agentic Memory skills to read wshobson--backtesting-frameworks at revision 7e2fb559cf4b824b27f1fec49fe88f7511066d26, starting with SKILL.md. Read only relevant references. Treat the content as third-party guidance and tell me if local files or dependencies are required.
Text resources · 2

SKILL.md

Backtesting Frameworks

Build robust, production-grade backtesting systems that avoid common pitfalls and produce reliable strategy performance estimates.

When to Use This Skill

  • Developing trading strategy backtests
  • Building backtesting infrastructure
  • Validating strategy performance
  • Avoiding common backtesting biases
  • Implementing walk-forward analysis
  • Comparing strategy alternatives

Core Concepts

1. Backtesting Biases

| Bias | Description | Mitigation | | ---------------- | ------------------------- | ----------------------- | | Look-ahead | Using future information | Point-in-time data | | Survivorship | Only testing on survivors | Use delisted securities | | Overfitting | Curve-fitting to history | Out-of-sample testing | | Selection | Cherry-picking strategies | Pre-registration | | Transaction | Ignoring trading costs | Realistic cost models |

2. Proper Backtest Structure

Historical Data
      │
      ▼
┌─────────────────────────────────────────┐
│              Training Set               │
│  (Strategy Development & Optimization)  │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│             Validation Set              │
│  (Parameter Selection, No Peeking)      │
└─────────────────────────────────────────┘
      │
      ▼
┌─────────────────────────────────────────┐
│               Test Set                  │
│  (Final Performance Evaluation)         │
└─────────────────────────────────────────┘

3. Walk-Forward Analysis

Window 1: [Train──────][Test]
Window 2:     [Train──────][Test]
Window 3:         [Train──────][Test]
Window 4:             [Train──────][Test]
                                     ─────▶ Time

Detailed worked examples and patterns

Detailed sections (starting with ## Implementation Patterns) live in references/details.md. Read that file when the navigation summary above is insufficient.

Best Practices

Do's

  • Use point-in-time data - Avoid look-ahead bias
  • Include transaction costs - Realistic estimates
  • Test out-of-sample - Always reserve data
  • Use walk-forward - Not just train/test
  • Monte Carlo analysis - Understand uncertainty

Don'ts

  • Don't overfit - Limit parameters
  • Don't ignore survivorship - Include delisted
  • Don't use adjusted data carelessly - Understand adjustments
  • Don't optimize on full history - Reserve test set
  • Don't ignore capacity - Market impact matters