print(Purple)
A working wiki on quantitative research, pandas and honest validation.
Try: · · · · ·
kit.py and a notebook template: load, triage, clean, split and score a time-stamped table, with a printout at every step.Browse by topic
51 pagesHow to go from an open-ended question and an unfamiliar table to an honest number and a short presentation.
One-line answers to the operations that come up on every panel: shift, roll, rank and correlate within groups. Plus the idioms that survive library upgrades.
OLS from first principles, correlation measures, t-statistics from repeated looks, and the base-stack way to compute each.
Forward targets, look-ahead leakage, horizon decay, and why a single series gives you far fewer independent looks than it seems.
Choosing the split that matches the data, chronological splits with embargoes, hold-outs reported once, and what selection bias feels like from the inside.
Debugging a model that looks too good, what each kind of dataset lies about, leaked columns, missing values, drift, and whether an extreme value is real.
Building features on a panel, scoring predictive signals, checking they add anything, and attacking your own result.
Complete analyses on public data with every number produced by code that ran. Including the ones that found nothing.
Charts as PNGs for slides, and looking things up quickly without leaving the notebook.
Start here
I opened a dataset. What now?
A router for an unfamiliar table. What is one row, is there a clock, do entities repeat, what shape is the target, and which symptom are you seeing. Each answer sends you to the page and the worked example that deal with it.
Choosing a train/test split
The split has to follow the structure of the data, not the default of the library. A decision table by structure, the one diagnostic that says which axis to hold out, and the habit of running the shuffled split once as a control.
Debugging a model that looks too good
A checklist for the moment a number comes out better than the problem deserves. The size of the jump is the first clue; then the seven ways a score gets inflated, in the order they are worth checking, each with the worked example where it happened.
Prediction boundaries: what would actually be known?
The one question that removes most leakage before it happens. At the exact moment the prediction would be made, which columns exist? Sort every column into before, at, after, derived-from-the-future, or unsure, and do it before building a feature.
A research workflow for open-ended questions
The stages between receiving an unfamiliar table with a vague question and presenting an honest number. Interrogate, agree the question, fix the protocol, get a baseline, try one idea, attack it, stop.
Analysing a new dataset in a fixed block of time
How to pace a few hours with an unfamiliar table so that there is a defensible result at every stage. Orient, baseline, look, one real model, diagnose, improve only where the diagnostics say to, try to break it, write it up. With the stopping rules that keep the last hour from being a rescue.
How this wiki works
More about the site →Recently updated
I opened a dataset. What now?
A router for an unfamiliar table. What is one row, is there a clock, do entities repeat, what shape is the target, and which symptom are you seeing. Each answer sends you to the page and the worked example that deal with it.
Evaluating predictive signals
Three questions turn "does this predict that?" into one number with an error bar. What picture would I see, how do I get many independent looks at it, and how do I summarise them. The result is the daily cross-sectional rank IC.
Pre-analysis protocol
Five lines written down before any feature is built and not changed afterwards. Target, signal, split, score, baseline. With the template, the code that encodes it, and two filled-in examples.
Pandas by verb
One line for each operation that comes up on every panel. Shift within group, roll per group, transform versus apply, correlate per date, rank and bucket, merge on date, resample. Each written in the form that survives pandas upgrades.
A research workflow for open-ended questions
The stages between receiving an unfamiliar table with a vague question and presenting an honest number. Interrogate, agree the question, fix the protocol, get a baseline, try one idea, attack it, stop.
Charts as PNGs for slides
The charts that carry a signal-scoring presentation, each a few lines of plain matplotlib that shows in a notebook and saves a PNG at a size that fits a slide. Cumulative IC, deciles, rolling IC, build against hold-out, the input's distribution, the spread in bps, cost sweep, the target's tails and IC by year.