THE QUANT PRIMER 01｜WHERE ALPHA DISAPPEARS

The distance between a backtest and a tradable strategy.

The distance between a backtest and a tradable strategy.

Most introductions to quantitative trading begin with models. That is convenient, because models are visible. They can be graphed, compared and given names.

The difficult part sits elsewhere: deciding whether a statistical relationship can be turned into a position without losing its value along the way.

Suppose a researcher discovers that assets with a certain characteristic tend to outperform. The result may be genuine, but it still says very little about the strategy that should be traded. The relationship may exist only in the least liquid names. Its return may arrive slowly while the portfolio turns over quickly. It may overlap with risks already held elsewhere. Once spreads, market impact and financing are included, the apparent edge may disappear.

The signal was not false.

It was incomplete.

That distinction is the starting point for quant research.

The forecast describes a conditional relationship inside data.

A trade requires an additional set of decisions about size, timing, implementation and risk.

The model contributes to those decisions, but it does not replace them.

1. What market data can and cannot tell you

Prices are exact numbers generated by ambiguous causes.

A ten-basis-point move in Treasury yields might reflect an inflation surprise, dealer positioning, mortgage convexity hedging, a large liquidation or some combination of all four.

The tape records the result, not the motive. Researchers therefore work from indirect evidence: returns, volume, volatility, order imbalance, spreads, positioning and relationships across markets.

The aim is not to reconstruct every participant’s intention. That would be impossible and, for trading purposes, unnecessary.

The useful question is narrower: does the information available now alter the distribution of returns enough to justify taking risk?

This formulation avoids a common mistake. New researchers often treat prediction as a binary contest between being right and being wrong.

Markets are less accommodating.

The forecast can be directionally correct but economically irrelevant because the expected move is too small.

Another forecast may carry poor directional accuracy but identify rare outcomes with unusually large payoffs.

The value of information depends on how it changes a decision, not on how impressive it sounds when described.

2. Why hit rate is a weak measure

Consider two systems.

The first wins 60 percent of its trades. Its average winner earns 0.20 percent, while its average loser gives back 0.50 percent.

Before costs, its expected return per trade is negative:

0.60 × 0.20% − 0.40 × 0.50% = −0.08%

The second wins only 40 percent of the time, but the winners average 1.00 percent and the losers average 0.30 percent:

0.40 × 1.00% − 0.60 × 0.30% = +0.22%

The lower-accuracy system has the better economics.

Nothing unusual is happening here.

Trend-following strategies can lose frequently while relying on a small number of extended moves. Short-volatility strategies often display the opposite profile: many small gains interrupted by losses large enough to erase months of apparent consistency.

Accuracy ignores this asymmetry because it counts outcomes without weighting their economic importance.

The correct forecast of a negligible move receives the same label as a correct before a major break.

The account does not treat them equally, and neither should the research process.

A useful forecast therefore needs a scale. Direction matters, but so do expected magnitude, uncertainty and the shape of the payoff. Only after those are estimated can the portfolio decide how much risk, if any, the forecast deserves.

CHART 01 — ACCURACY IS NOT EXPECTANCY

3. Signals usually rank rather than predict

Many practical signals are more useful as rankings than as point forecasts.

The value measure does not need to predict the exact return of a stock over the next month. It may be enough that cheaper securities, defined consistently, have historically produced a better distribution than expensive ones. Momentum works similarly.

The signal identifies relative persistence, then assigns stronger exposure to the markets where that persistence appears most pronounced.

Once the signal is viewed as a ranking device, the portfolio problem becomes clearer. A score determines where an asset sits relative to others, but it does not specify the final weight.

That weight still depends on volatility, correlation, liquidity and the confidence placed in the estimate.

This separation is useful because statistical strength and tradability often diverge. Some anomalies are strongest among small, illiquid securities where implementation is expensive. Others decay too quickly to survive normal execution. A signal may also look diversified across dozens of instruments while expressing one underlying macro exposure.

The ranking contains information. Portfolio construction decides whether that information is usable.

4. The backtest includes the researcher

A backtest is often presented as though the strategy entered history once, met the data and produced a result. Research rarely happens that way.

The researcher changes the lookback window, modifies the universe, shifts the rebalance date, adds a filter, removes a difficult period and tries another definition.

Most versions fail quietly.

The surviving version becomes the strategy.

The final chart therefore reflects two things: whatever structure existed in the market and the amount of freedom used to search for it.

This is why an impressive Sharpe ratio cannot be interpreted without knowing how many alternatives were tested. A result selected from ten sensible specifications carries a different evidential weight from the same result selected after fifty thousand trials. The number printed on the page may be identical, but the second process gave noise far more opportunities to impersonate skill.

Bailey, Borwein, López de Prado and Zhu formalized this problem through the probability of backtest overfitting.

Their work addresses a basic fact of strategy research: repeated selection can produce strong in-sample performance even when the underlying configurations have little out-of-sample value.

Train, validation and test splits reduce the problem, though they do not eliminate it. Repeatedly redesigning a model after checking validation results can overfit that stage as well.

What matters is not only whether data was held out, but whether the full research path was recorded honestly.

The backtest process should be treated as the output of an experiment, not as a photograph of an untouched historical truth.

CHART 02 — THE SEARCH TAX

5. Costs are not a final adjustment

Researchers often calculate gross returns first and subtract a cost estimate later. That ordering is convenient for presentation, but misleading as a description of the strategy.

Transaction costs influence which trades should exist in the first place.

A portfolio with high turnover repeatedly pays spreads and commissions.

Larger orders create market impact. Less liquid securities require more patience or a wider execution allowance. Borrow fees can make apparently attractive short positions uneconomic. Financing alters the return on leveraged strategies, especially when rates change.

Once these frictions are modeled properly, the optimal design may look different. Rebalancing less frequently can preserve more of the signal. Weak trades may no longer clear the cost threshold. The investable universe may need to exclude names where the theoretical edge cannot be harvested at realistic size.

Novy-Marx and Velikov examined a large set of anomalies after transaction costs and found that implementation materially changes the profitability of many strategies.

Their results are a reminder that an anomaly documented in returns is not automatically an anomaly available to capital.

Execution also introduces a conflict between urgency and impact. Trading quickly reduces the chance that the market moves before the order is completed, but aggressive execution generally pays more.

Trading slowly reduces immediate impact while increasing exposure to price movement during the execution window. Almgren and Chriss modeled this trade-off explicitly.

The relevant question is not whether execution costs money. It always does. The question is whether the expected edge remains after choosing a feasible route into the position.

6. Position size is part of the thesis

A forecast has no natural dollar amount attached to it.

Suppose two assets receive equally strong signals, but one is twice as volatile. Equal notional positions would assign much more risk to the volatile asset. Volatility scaling attempts to correct this by reducing exposure as estimated risk rises.

That is only a first approximation. Correlation matters because several individually modest positions can combine into one large portfolio bet. Liquidity matters because the book may be easy to enter and difficult to exit. Confidence matters because not every signal estimate has the same reliability.

Sizing therefore expresses how much the system believes a forecast after accounting for what could go wrong. The weakly estimated opportunity in a liquid market may deserve less exposure than a slightly smaller edge supported by stable evidence. Positions that appear attractive in isolation may deserve no additional capital because the portfolio already holds the same underlying risk elsewhere.

This is why stop-losses are not a substitute for portfolio construction. A good stop system governs what happens after one position moves against the trader. It does not prevent several positions from failing together for the same reason.

The main risk decision occurs before the trade is entered, when the forecast is translated into exposure.

7. Indicators are implementations of mechanisms

A moving average is not a reason for trend following to work. It is one way to measure persistence.

This distinction matters because indicators are easy to copy, while mechanisms determine whether a strategy has any chance of surviving. Trend may persist because investors update beliefs gradually, institutions trade over long horizons, risk constraints force continued deleveraging or large economic shifts take time to be absorbed into prices.

Hurst, Ooi and Pedersen documented trend-following returns across a historical sample extending back to 1880. The significance of that result lies less in any specific parameter than in the recurrence of the underlying behavior across different markets and institutions.

A strategy tied only to a narrow rule is fragile. When performance weakens, the researcher has no framework beyond changing the parameter.

A strategy tied to an economic or behavioral mechanism can be examined more intelligently.

The relevant question becomes whether the mechanism still operates, whether competitors have compressed it, or whether implementation has become too expensive.

Monitoring should reflect that distinction. Changes in turnover, slippage, signal decay, concentration or regime performance may reveal more than the headline return. They help determine whether the strategy is experiencing normal variance or whether its original premise is deteriorating.

Parameter adjustment is easy.

Diagnosing why the parameter stopped working is the actual research task.

8. The model sits inside a longer chain

The path from raw data to realized return contains several transformations:

DATA → FEATURE → FORECAST → SIGNAL → POSITION → ORDER → FILL → RETURN

Each stage can damage the information produced by the one before it.

Incorrect timestamps create lookahead bias. Survivorship bias removes failed securities from the historical universe. Poor calibration turns modest forecast differences into unstable weights. Hidden correlations concentrate exposure.

Unrealistic fill assumptions overstate execution quality. Financing, borrow and delay reduce the return that finally reaches the account.

The model attracts attention because it is intellectually visible. Data cleaning, reconciliation and execution analysis are harder to display, though they often determine whether the result is real.

A disciplined research process therefore spends considerable time on details that appear mundane: when information became available, whether historical databases were revised, how corporate actions were handled, whether the simulated order could have traded at the assumed price and how actual fills differed from the model.

Meanwhile, the sophisticated forecast cannot repair a contaminated pipeline. It only makes the contamination harder to notice.

CHART 03 — THE ALPHA LEAKAGE MAP

9. What should be optimized

Maximizing historical return is an attractive objective because it is easy to measure. It is also likely to reward fragility.

A more useful objective is to seek net returns that remain plausible after accounting for estimation error, implementation costs, capacity and drawdown tolerance. This naturally favors models with fewer arbitrary choices, stable performance across related markets, lower dependence on a single period and enough liquidity to support real capital.

Such models may look weaker in sample. That is not necessarily a defect.

A highly flexible strategy can exploit details unique to the historical path, while a constrained strategy must ignore many of them.

The unused in-sample return is sometimes the cost of avoiding a claim that cannot be repeated.

Robustness should not become another vague virtue, however. It needs tests.

Small changes in lookback, rebalance date, cost assumptions or universe definition should not destroy the result. Performance should not depend entirely on one asset or one crisis. The economic explanation should remain coherent when the sample is divided into different periods.

The aim is not to produce a strategy that never loses. It is to produce one whose failure modes are understood before capital encounters them.

10. How to read a backtest

The equity curve is the least interesting place to begin.

First establish whether the data used in each decision was actually available at that time. Then examine the research process: how many versions were tested, which choices were made after seeing results and whether failed specifications were retained in the record.

Implementation comes next. Check the assumed spreads, position sizes, borrow availability, financing and execution schedule. Determine whether the quoted returns could have been achieved without moving the market.

Finally, study the distribution of performance. A strategy whose profits came from one brief episode is making a different claim from one that earned returns across several regimes.

The same is true when one market contributes nearly all of the Sharpe ratio.

These questions do not prove that a strategy will work. Nothing can. They determine whether the historical evidence deserves to be taken seriously.

Quant research is not the search for a rule that once made money. It is the attempt to identify information that survives statistical scrutiny, portfolio construction and implementation.

That standard eliminates many attractive backtests.

It should.

Conclusion

A signal is evidence that future outcomes may be distributed differently from the past average. Turning that evidence into a trade requires a separate judgment about exposure, liquidity, timing and portfolio risk.

This is why the model cannot be evaluated in isolation. Its usefulness depends on the system around it and on how much of its apparent edge remains after the assumptions meet real execution.

The backtest estimates what might have been available.

The account records what was actually captured.

The distance between those two numbers is where most quantitative strategies are decided.

Research Anchors

1. Bailey, David H.; Borwein, Jonathan M.; López de Prado, Marcos; Zhu, Qiji Jim. The Probability of Backtest Overfitting.

2. Bailey, David H. et al. The Effects of Backtest Overfitting on Out-of-Sample Performance.

3. Novy-Marx, Robert; Velikov, Mihail. A Taxonomy of Anomalies and Their Trading Costs.

4. Almgren, Robert; Chriss, Neil. Optimal Execution of Portfolio Transactions.

5. Hurst, Brian; Ooi, Yao Hua; Pedersen, Lasse Heje. A Century of Evidence on Trend-Following Investing.