1. What is being graded
A verdict describes one simulation of one strategy specification on historical data. The specification comes from the caller. Lagias fits it on a development window, measures how it behaves on a later window the fitting never saw, and reports what the two windows say about each other.
The evaluator is a deterministic pipeline. No language model assigns a badge, chooses a threshold, or edits a number. The proposing side and the grading side are separate programs, and the grading side has no incentive to like the answer.
2. The published thresholds
Three measurements decide the badge. They are constants in the engine, identical for every caller and every strategy, and they are returned inline with every verdict so a response can be held against this page.
| Measurement | Passes at | Hard fails at |
|---|---|---|
| Deflated Sharpe probability | 0.75 or above | below 0.60 |
| Probability of backtest overfitting | 0.25 or below | 0.50 or above |
| Out-of-sample Sharpe decay | 85 percent or below | above 85 percent |
3. How the badge follows
The rule is mechanical, and there is no discretionary override anywhere in it.
| Badge | Rule |
|---|---|
| Robust | All three measurements pass. |
| Likely overfit | Any one measurement hard fails. |
| Worth testing | Everything between those two. |
4. What each measurement is
- Deflated Sharpe probability. The probability that the observed Sharpe ratio is greater than zero once the number of variants tried, the length of the record, and the shape of the return distribution are accounted for. A high raw Sharpe found after two hundred attempts is worth less than the same number found on the first, and this is the statistic that says by how much.
- Probability of backtest overfitting. The share of data splits in which the variant that looked best in the training half failed to stay above median in the held-out half. It answers a question a single backtest cannot: would this selection procedure have picked a winner, or did it pick the luckiest curve?
- Out-of-sample Sharpe decay. How much of the development-window Sharpe survives into the window the fitting never saw, as a percentage lost. A strategy that keeps most of its edge decays little; one that only worked where it was fitted decays almost entirely.
The first two follow the deflated Sharpe ratio and the probability of backtest overfitting as defined by Bailey and López de Prado. We did not invent the tests, and that is deliberate: a house metric can be tuned until it flatters the house.
5. How decay is described
Alongside the badge, every verdict states in one word what happened to the edge outside the fitting window.
| Word | When it applies |
|---|---|
| Holds | Decay of 35 percent or less. |
| Fades partly | Decay above 35 and up to 85 percent. |
| Fades | Decay above 85 percent. |
| Inconclusive | Fewer than 60 out-of-sample observations, or no decay figure could be computed. |
6. Hypothetical performance
Everything a verdict reports is hypothetical. It is generated from criteria and assumptions the caller supplies, on an interactive tool the caller operates, and it describes a simulation rather than an account.
Hypothetical results have inherent limitations. They do not reflect actual trading, they do not reflect the impact that material economic and market factors would have had on real decisions, and they carry the benefit of hindsight over the period simulated. Actual results may differ materially from those shown. Backtested performance is not indicative of future performance.
Simulations include an allowance for trading costs and slippage, and the assumptions behind any figure are available on request.
7. Where this method is weak
- The overfitting probability counts the trials it is told about. A caller who runs the same idea across many sessions and reports only the best run has defeated the statistic, and no server-side check can see that.
- A held-out window is one draw of history. Passing it says the result was not fitted to the development window; it does not say the regime that produced the edge will continue.
- Cost and slippage assumptions are estimates. A strategy whose edge is thinner than its cost assumption is fragile in a way the badge alone will not show.
- A robust badge is a statement about evidence, not a suggestion to do anything. It says the result survived the tests on this page, and nothing else.
Publishing thresholds only helps if the limits are published with them.
8. Checking a verdict against this page
Every verdict response carries the thresholds it was graded against, the underlying statistics, and a caveat that travels with the numbers. Recompute the badge from the table in section 3 and it must match. If it ever does not, that is a defect, and we would like to hear about it at [email protected].
Substantiation for any figure we publish, including the parameters behind it, is available on request at the same address.
Last updated: August 2026