Published thresholds · fixed
The Gauntlet · Trials v1
A strategy survives only if it passes every applicable Trial. The thresholds are published here and embedded in every report; if any threshold is ever loosened, the report says so. Nothing is scored until the submission is hashed. Pass is rare by design.
The eight Trials
1
Leakage
- Replay uses only bars up to and including the decision bar; indicators are strictly trailing.
- A signal at close t executes at the open of t+1. Nothing is filled at the price that generated it.
- Prices are split/dividend adjusted; the engine is covered by a planted-lookahead test that must not be rewarded.
- Code submissions:
decide(history, state)is called once per trading day inside an isolated sandbox, and each day's bar is delivered only after the previous day's decision has been returned. The submitted code never has access to future data.
2
Effect
- Out-of-sample information coefficient (Spearman rank, sign-oriented) between entries and forward returns, corrected for overlapping horizons and feature persistence.
- Bar: IC ≥ 0.03.
3
Costs
- Net-of-cost Sharpe at the submitter's declared round-trip cost (default 20 bp) must be ≥ 0.5.
- Sharpe at 2× the declared cost must remain positive.
4
Luck
- Matched-random null: the same number of entries at random dates, 500 draws.
- Circular-shift null: the entry series rotated in time, preserving its clustering, 250 draws.
- p vs luck (worse of the two) ≤ 0.05.
- Storey–Benjamini–Hochberg q across all of the submitter's attempts ≤ 0.10. Every attempt you make raises the bar for the next.
- Deflated Sharpe (Bailey & López de Prado) against the expected best of N worthless strategies, N = ideas_tried as declared, ≥ 0.90.
5
Tuning
- Every numeric parameter is nudged ±20%, one at a time. The median neighbour must retain ≥ 50% of the submitted Sharpe.
- A real edge degrades smoothly. A fitted peak collapses.
- Code submissions expose no parameters to nudge; their tuning risk is carried entirely by Trial 4's ideas_tried, so declare it honestly.
6
Stability
- Four chronological walk-forward folds; ≥ 60% of folds with entries must agree in sign.
- Drop-one-year: removing any single calendar year must leave the result positive in ≥ 80% of cases.
7
Evidence
- ≥ 250 days in market, ≥ 60 independent observations after overlap correction, ≥ 15 episodes, ≥ 55% of episodes positive.
- Mean episode return excluding the single best episode must be > 0. One lucky trade does not carry a pass.
8
Forward
- Cannot be passed at submission. A survivor of Trials 1–7 enters a 90-day probation on data that did not exist when it was submitted. The badge turns fully green only if the forward window agrees.
What a pass means
- The effect on the data window shown cannot be explained by lookahead, costs, luck, tuning, or a single regime, and it held up for 90 days after it was registered.
- It does not mean the strategy will make money. Edges decay and get crowded. The Gauntlet certifies the test, not the future.
Data (v0.1)
- Daily bars from the consolidated US tape (all exchanges), split- and dividend-adjusted, January 2016 to present: 22 ETFs spanning US equity indices (SPY, QQQ, IWM, DIA), the nine sector funds, Treasuries (TLT, IEF), high yield (HYG), gold, silver, oil (GLD, SLV, USO), international (EFA, EEM) and real estate (VNQ). The window includes the 2018 Q4 selloff, the 2020 crash, the 2022 bear market and the 2025 tariff shock.
- Where a provider adjustment misses a corporate action, history starts after it rather than inventing a factor (XLF from 2016-09-19, after the real-estate spin-off).
- Every verdict is stamped with the dataset version it was scored on. When the dataset changes, every attempt is re-scored on the new data; nothing is grandfathered.
- Submissions are hashed (SHA-256 of the canonical spec) before any data is read. The hash is the attempt id.