Forward targets and look-ahead leakage

Look-ahead leakage is the failure mode that produces the best-looking results, which is why it has to be ruled out first. The rule is simple to state: for the row stamped $t$, the target is computed only from observations after $t$, and every feature only from observations at or before $t$. The ways to break it are less obvious.

import pandas as pd, numpy as np

df = pd.read_csv("all_stocks_5yr.csv")
df["date"] = pd.to_datetime(df["date"])
df = df.rename(columns={"Name": "ticker"}).sort_values(["ticker", "date"]).reset_index(drop=True)
per = df.groupby("ticker").size()
df = df[df["ticker"].isin(per[per >= 1000].index)].copy()
g = df.groupby("ticker")
df["ret"] = g["close"].pct_change()
df["fwd_ret"] = g["ret"].shift(-1)          # target: tomorrow. shift(-1) looks forward.
df["sig1"] = -df["ret"]                     # feature: today

The direction of the shift

shift(1) moves each value down one row, so the row for day $t$ sees day $t-1$: a lagged feature. shift(-1) moves values up, so the row for day $t$ sees day $t+1$: a forward target. Using shift(1) for a target means “predicting” yesterday’s return from today’s, which any signal built from today’s return does perfectly.

The one-line detector: correlate the signal with the same-day outcome. If a signal built from day $t$ correlates with day $t$‘s return at ±1, and with day $t+1$‘s at a few hundredths, the target is pointing the right way. If the numbers come out the other way round, the shift is backwards.

d = df.dropna(subset=["ret", "fwd_ret"])
def daily_ic(d, sig, tgt):
    return d.groupby("date")[[sig, tgt]].apply(lambda x: x[sig].corr(x[tgt], method="spearman")).dropna()
print("signal vs SAME-day return:  %+.4f" % daily_ic(d, "sig1", "ret").mean())
print("signal vs NEXT-day return:  %+.4f" % daily_ic(d, "sig1", "fwd_ret").mean())

signal vs SAME-day return:  -1.0000
signal vs NEXT-day return:  +0.0126

Any |IC| near 1 between a feature and a target means they share a day, and the result is not a result.

Six ways the future gets in

1. The wrong shift. Above.
2. A centred or forward-looking window. rolling(n, center=True) uses days after $t$. So does any “smoothed” series computed over the whole period. Trailing windows only.
3. Normalising with full-period statistics. Dividing by the standard deviation of the whole sample, or z-scoring against the whole period’s mean, uses data from the end of the sample to build a feature at the start. Normalise with trailing windows, or within each date.
4. Publication lag. A quarterly figure dated 31 March was published in May. Anyone using it on 1 April is using information nobody had. Lag fundamentals and macro releases by their publication date, not their reference date, and when the publication date is missing assume the worst plausible lag.
5. The universe. A file of the index’s members as of the end of the period was assembled with knowledge of who survived. Every “buy the losers” result is flattered by it. See common data-quality traps.
6. The split boundary. The last rows of the build period have targets that fall inside the check period. An embargo at least as long as the target horizon fixes it. See chronological splits with an embargo.

A deliberately leaked feature

A three-day centred moving average of returns on day $t$ includes day $t+1$‘s return, which is the target. It is the kind of feature that appears in a notebook under the name “smoothed momentum”.

df["ma3_centred"] = g["ret"].transform(lambda x: x.rolling(3, center=True).mean())      # uses t-1, t, t+1: LEAKS
df["ma3_trailing"] = g["ret"].transform(lambda x: x.rolling(3).mean())                  # uses t-2, t-1, t: clean
d = df.dropna(subset=["fwd_ret", "ma3_centred", "ma3_trailing"])
for c in ("ma3_centred", "ma3_trailing"):
    ic = daily_ic(d, c, "fwd_ret")
    print("%-13s mean IC %+.4f   t = %+.1f" % (c, ic.mean(), ic.mean() / (ic.std() / np.sqrt(len(ic)))))

ma3_centred   mean IC +0.5238   t = +144.3
ma3_trailing  mean IC -0.0102   t = -2.4

A t-statistic in the hundreds on a daily return signal is not a discovery. It is a leak, and the size of the number is the tell: nothing honest on this data scores above an IC of a few hundredths at one day ahead.

The habit

Print a few rows of one group with the feature and the target side by side and read the dates. Do it once for every new target. It takes ten seconds and it is the only check that catches a wrong shift before the number looks wonderful. See building features on a panel for the pattern.