Case Studies
Complete analyses on public data with every number produced by code that ran. Including the ones that found nothing.
Short-term reversal in daily equity returns
One complete pass through the research workflow on public data. Do yesterday's losers bounce? A small, real-looking effect that held out of sample, a better-looking candidate that did not, and an edge that does not survive costs.
Does relative volume predict the size of tomorrow's move?
A second complete pass on the same public file, with a different kind of question. Both quantities had to be defined precisely before anything could be scored, and the boring explanation had to be scored alongside the interesting one.
Does today's market return predict tomorrow's?
The single-series version of the reversal question, under a fixed protocol. The protocol's score finds nothing; a measure the protocol did not name finds something consistent. What to do with a result like that.
Twenty-day momentum, skipping the most recent day
Do the last month's winners keep winning? A study whose most significant in-sample number turned out to be an artefact of overlapping targets, and which reported nothing rather than something.
Predicting taxi tips: the column that contains the answer
A non-financial table with the two traps that matter most outside finance. A total column that includes the target, and a payment type for which the target was never recorded. What each does to a model, measured on a hold-out.
Repeated subjects: a model that learns who, not how ill
Voice recordings from 42 people, about 140 each, and a clinician's symptom score to predict. A random split gives an R² above 0.9; holding out whole subjects gives a negative one. What the model actually learned, and which question each split answers.
Hourly bike rentals: the random split that looked great
Two years of hourly rental counts with calendar and weather columns. A random split scores an R² of 0.86; training on the first year and testing on the second scores 0.63. Why the gap exists (the system grew), why adding a year column cannot fix it, and what does.
A classifier that looks too good: the call-duration column
45,211 phone contacts from a bank's deposit campaigns, 11.7% of them successful. With the call's duration a logistic model reaches an AUC of 0.91; that column is only known after the outcome. Without it, 0.76. Then the accuracy trap on an 88/12 target, and what a chronological check shows when the base rate moves from 5% to 52%.
Hotel cancellations: duplicates, post-booking columns and the threshold
119,390 bookings at two hotels, 37% of them cancelled. A forest scores an AUC of 0.88 on a shuffled split; a quarter of the rows are exact duplicates and four columns are written after the booking. With those handled and a chronological test, 0.74. Then what the probabilities are for, which is a decision, not a score.
Forecasting hourly electricity load: the naive forecasts are the competition
A year of ten-minute power readings for a city, resampled to hours. "Same hour last week" already explains 96% of the variance an hour or a day ahead. A ridge regression on lags and calendar dummies beats it on error every month; temperature and trailing means add nothing once the lags are in. How to name the horizon, build lags relative to it, and report a forecast against the right baseline.
Twelve monitoring sites, one sky: a panel where the entity does not matter
Four years of hourly air-quality readings at twelve sites, aggregated to station-days, with tomorrow's PM2.5 as the target. Holding out whole stations changes nothing; holding out dates cuts the forest's R² from 0.72 to 0.35. Which axis of a panel carries the variance decides which split is honest, and here it is the opposite of the patient data.
Flight delays: what is known two hours before departure
328,521 departures from three airports in one year, 21% of them more than fifteen minutes late. A model built from the schedule alone scores an AUC of 0.67; one that also uses the airport's observed delay state two hours before departure, 0.73. The weather at the departure hour, which nobody has in advance, adds nothing beyond the weather two hours earlier. How the moment of prediction decides the feature set, and how to encode four thousand aircraft without fooling yourself.
House prices: what a missing value means, and when it matters
2,930 home sales with 82 columns, a right-skewed price, and missing values of three different kinds. Dropping incomplete rows loses most of the data and costs accuracy; between imputation strategies the difference is nil. Absence tracks price in the raw table, but through "no garage" and through neighbourhood, not on its own. How to classify missingness before treating it, and how to tell whether absence is a signal.
Hourly traffic volume: a messy clock and a baseline that is hard to beat
Six years of hourly vehicle counts on an interstate, with duplicate timestamps, a ten-month hole, a temperature of absolute zero and a rain reading of ten metres. After the axis is cleaned, "same hour last week" explains 89% of the variance; a forest on the calendar reaches 95%, and the shuffled and chronological splits agree, because nothing drifts. When a random split does not lie, and what still can.