Does a signal add anything beyond a known baseline?

Suppose relative volume today predicts the size of tomorrow’s move. Before that is interesting there is a boring explanation to rule out: volatility clusters, so a big move today predicts a big move tomorrow, and busy days are big-move days. Volume might be “predicting” tomorrow only by proxying for today’s move.

The protocol has to name the boring mechanism as the baseline and score it under exactly the same rules as the signal. Then there are three ways to ask whether the signal adds anything.

Setup: signal, target, and the boring baseline

import pandas as pd, numpy as np

df = pd.read_csv("all_stocks_5yr.csv")
df["date"] = pd.to_datetime(df["date"])
df = df.rename(columns={"Name": "ticker"}).sort_values(["ticker", "date"]).reset_index(drop=True)
per = df.groupby("ticker").size()
df = df[df["ticker"].isin(per[per >= 1000].index)].copy()
g = df.groupby("ticker")

df["ret"] = g["close"].pct_change()
df["vol20"] = g["ret"].transform(lambda x: x.rolling(20).std())
df["target"] = g["ret"].shift(-1).abs() / df["vol20"]                           # tomorrow's move, in units of own vol
df["relvol"] = df["volume"] / g["volume"].transform(lambda x: x.rolling(20).mean())  # the signal
df["base"] = df["ret"].abs() / df["vol20"]                                       # the boring baseline: today's move
df = df.replace([np.inf, -np.inf], np.nan).dropna(subset=["target", "relvol", "base"])

cut, embargo = pd.Timestamp("2016-06-30"), pd.Timedelta(days=7)
train = df[df["date"] <= cut].copy()
test = df[df["date"] > cut + embargo].copy()

def daily_ic(d, sig, tgt):
    return d.groupby("date")[[sig, tgt]].apply(lambda x: x[sig].corr(x[tgt], method="spearman")).dropna()

def report(ic, label):
    m, s, n = ic.mean(), ic.std(), len(ic)
    print("%-30s mean IC %+.4f   t = %+.2f   hit = %.2f   days = %d" % (label, m, m / (s / np.sqrt(n)), (ic > 0).mean(), n))

1. Score both under the same protocol

The minimum. If the signal’s IC is not higher than the baseline’s, stop.

report(daily_ic(test, "relvol", "target"), "signal: relative volume")
report(daily_ic(test, "base", "target"), "baseline: today's move")
print("mean daily rank correlation between signal and baseline: %.3f" % daily_ic(test, "relvol", "base").mean())

signal: relative volume        mean IC +0.1010   t = +29.36   hit = 0.93   days = 399
baseline: today's move         mean IC +0.0735   t = +19.21   hit = 0.84   days = 399
mean daily rank correlation between signal and baseline: 0.332

Higher, and the two are only moderately correlated with each other (0.33), so they are not the same thing. Still not proof that the signal adds anything given the baseline.

2. Combine them and see if the combination beats the baseline alone

Average the within-date ranks. If the combined IC exceeds the better single IC, each is carrying information the other lacks.

for d in (train, test):
    d["combined"] = d.groupby("date")["relvol"].rank(pct=True) + d.groupby("date")["base"].rank(pct=True)
report(daily_ic(test, "combined", "target"), "signal + baseline, test")

signal + baseline, test        mean IC +0.1062   t = +28.39   hit = 0.94   days = 399

A little better than the signal alone (0.1010) and clearly better than the baseline alone (0.0735). Cheap and convincing, but it does not give a t-statistic for the signal’s incremental contribution.

3. The incremental test: a rank regression per date

Within each date, regress the rank of the target on the ranks of both the baseline and the signal. Collect the two slopes, one pair per day. The mean slope on the signal, with its t across days, is the signal’s contribution holding the baseline fixed. This is the Fama–MacBeth construction, on ranks, in numpy.

def per_date_slopes(d, cols, tgt):
    def one(x):
        y = x[tgt].rank().values
        X = np.column_stack([np.ones(len(y))] + [x[c].rank().values for c in cols])
        beta = np.linalg.lstsq(X, y, rcond=None)[0]
        return pd.Series(beta[1:], index=cols)
    return d.groupby("date")[cols + [tgt]].apply(one)

for name, d in (("train", train), ("test", test)):
    slopes = per_date_slopes(d, ["base", "relvol"], "target")
    for c in slopes.columns:
        s = slopes[c]
        print("%-6s slope on rank(%s): mean %+.4f   t = %+.2f   days = %d" % (name, c, s.mean(), s.mean() / (s.std() / np.sqrt(len(s))), len(s)))

train  slope on rank(base): mean +0.0326   t = +10.82   days = 835
train  slope on rank(relvol): mean +0.0678   t = +27.04   days = 835
test   slope on rank(base): mean +0.0442   t = +11.30   days = 399
test   slope on rank(relvol): mean +0.0862   t = +24.69   days = 399

Holding today’s move fixed, relative volume still carries a slope with a t of 25 on the hold-out. The boring explanation accounts for part of the pattern and not most of it. That is the sentence to say: the baseline explains some of it; the signal survives controlling for the baseline with t of 25 out of sample.

Reporting it

The report line is not “volume predicts volatility”. It is: the signal’s IC is 0.10 against the baseline’s 0.07; combining them gets 0.106; and in a per-date regression on both, the signal keeps a t of 25 out of sample. Three numbers, in that order, and a reader can see for themselves how much of the pattern was the boring mechanism. The full study, with stability and sensitivity checks, is the relative volume case study.