An independent verdict on a trading strategy — before you fund it.
You get a returns CSV and a Sharpe ratio. You don't get to see whether it survives Combinatorial Purged Cross-Validation, a Deflated Sharpe Ratio corrected for every configuration that was tried, PBO, and realistic costs — until someone runs the same calibrated judge on every strategy that asks for capital.
The problem
Why most backtests lie
Not through fraud — through the default way a backtest is written. Every one of these is easy to do by accident, and each one alone is enough to turn noise into a Sharpe ratio worth pitching.
Look-ahead bias
The strategy peeks at information from bar close that wasn't actually available when the position was opened.
In-sample scoring
Training and evaluating on the same window. The equity curve you're shown was fit to, not tested against.
Costs added later, or not at all
Taker fees, slippage and funding are the difference between a gross edge and a net one — and they're often missing.
Multiple testing, uncorrected
Try a few hundred configurations and show the best. The winner's Sharpe is inflated by the search itself, not by edge.
Survivorship
Delisted pairs, dead exchanges and abandoned parameter sets quietly disappear from the sample that gets reported.
What you get
A mechanical PASS/KILL verdict, not an opinion
Every submission runs through the same calibrated judge: Combinatorial Purged CV builds honest out-of-sample paths, the Deflated Sharpe Ratio corrects for the number of trials, PBO reports whether the search overfit, and a drawdown and trade-count gate keep the verdict grounded.
- Checks table — trade count, OOS Sharpe, max drawdown, DSR, each against a pre-registered threshold.
- Path Sharpes — median and worst-path Sharpe across every CPCV split, not one lucky curve.
- DSR with trial count — the Sharpe deflated by how many configurations were actually tried.
- PBO — probability of backtest overfitting, reported for context (it never overrides the verdict).
- Cost sensitivity — the verdict re-run at 0x / 0.5x / 1x / 2x your stated cost, so "profitable at zero fees" is visible for what it is.
- A reproducible command — the exact invocation that produced the report, so anyone can re-run it on the same data.
$ qh-audit --returns audit_sample_returns.csv --trials 20 \
--out reports/audit.md --json reports/audit.json
qh-audit: VERDICT KILL (audit_sample_returns)
data : 730 periods x 4 configs, 2.00 years, 365 periods/year
trials : 20 (effective)
OOS Sharpe : +1.066 median path, +0.461 worst path
max DD : 0.114 trades: 730
PSR 0.9335 DSR 0.5885 (N=20) PBO 0.6865
checks : trades_ok=yes oos_sharpe_ok=yes drawdown_ok=yes dsr_ok=NO
ASSUMPTION: trades not provided; turnover assumed at 1
trade(s) per bar for every config — the trade-count gate
and any cost deduction rest on this assumption, not on data
report : reports/audit.md json : reports/audit.json
statistical report, not investment advicePositive OOS Sharpe, three of four gates passed — and still a KILL: the fourth gate, the Deflated Sharpe, fails once it accounts for the 20 configurations that were actually tried. This is the judge working as designed.
Who it's for
Anyone on the capital side of "trust me, it works"
Allocators & funds
A pre-registered, mechanical screen to run before capital moves — same thresholds for every manager.
Prop desks
Screen internal submissions against the same bar external capital would apply, before risk is allocated.
Bot marketplaces & copy-trading platforms
An independent gate on listed strategies, run the same way for every vendor on the platform.
AI-agent builders
A strategy an agent generated needs the same skepticism a human strategy would get — this is that check.
This product's only job is to say no more accurately than a Sharpe ratio in a pitch deck can. That's uncomfortable to sell and valuable to buy — the entire value to the capital side is that the verdict was never allowed to be talked up after the fact.
How it works
Three steps, or run it yourself
Send your returns CSV
Per-period returns for every configuration you tried, plus the trial count and your realistic cost assumption. A trades CSV is optional but sharpens the turnover and cost estimate.
The judge runs
Combinatorial Purged CV, Deflated Sharpe, PBO and the drawdown / trade-count gates run against pre-registered thresholds — the same ones for every submission.
Report back in 48h
A Markdown and JSON report with the verdict, every check, path Sharpes, cost sensitivity and the exact command to reproduce it.
Self-serve — pip install
Run the same judge yourself, on your own machine, on your own data.
pip install quant-harness
qh-audit --returns returns.csv --trials 40 \
--costs-bps 5 --out audit.md --json audit.jsonAPI — per call
POST a returns matrix, get the same verdict back as JSON. No install, no queue.
POST /v1/verdict
{ "returns": [[...]], "n_trials": 40,
"costs_bps": 5.0, "freq": "1d" }
→ { "verdict": "KILL", "checks": {...}, ... }Pricing
Fixed price per audit, no revenue share
USDC / USDT accepted. Every tier runs the identical judge — pricing reflects turnaround and depth, never a friendlier threshold.
Audit — Basic
One strategy, one verdict.
from $400 per submission- Full PASS/KILL report (Markdown + JSON)
- CPCV, Deflated Sharpe, PBO, drawdown & trade gates
- 48h turnaround
Audit — Full
Cost sensitivity and a written cause-of-KILL explanation.
from $1,200 per submission- Everything in Basic
- Cost-sensitivity table at 0x / 0.5x / 1x / 2x
- Short written note on which gate failed and why
- One re-run included if you fix an input error
API — per call
Programmatic verdicts for your own pipeline.
$0.05 per POST /v1/verdict- Same judge, JSON in and out
- No queue, no manual review step
- Good fit for marketplace / listing pipelines
Pay in stablecoins
Calibration
The judge was measured, not just trusted
A "reject everything" tool would look identical to a calibrated one until you check it against known ground truth. It was checked: 200 Monte-Carlo runs per cell, synthetic data with a known planted edge, mapping the full PASS/KILL response surface.
Full methodology and every calibration table: reports/CALIBRATION.md in the repository. This does not track live returns as a product — an internal research paper-tracker exists purely for methodology testing and is not part of what an audit reports on.
FAQ
Questions worth answering upfront
What if my strategy fails?
You get a KILL verdict and the report explaining exactly which gate failed — trade count, OOS Sharpe, drawdown or the Deflated Sharpe threshold — with the numbers behind it. A KILL is a real result, not a rejected submission; nothing stops you from re-testing a revised strategy.
Do you see my code?
No. The audit runs on a returns CSV (and optionally a trades CSV) — the outcome of your strategy, not its logic. Your signal, code and parameters never need to leave your environment.
Which markets does this work for?
Any market where you can produce a per-period returns series — crypto spot and perpetuals, equities, futures. The judge is asset-agnostic; it only needs the returns, the bar frequency, and how many configurations you tried.
What's the turnaround?
48 hours for a submitted audit. The API returns a verdict synchronously, in the time one CPCV run takes on your data.
Is this investment advice?
No. This is statistical software and a report on the data you supply — a screen against pre-registered thresholds, not a recommendation to trade, allocate to, or avoid any strategy, asset or manager. See the disclaimer below.
Refunds?
If the input can't be judged at all — malformed data, insufficient periods, an input error the CLI itself would reject — that submission is refunded or re-run for free. A KILL verdict on valid data is a delivered result, not a refund case: the judge did its job.