Research Workflows
How to go from an open-ended question and an unfamiliar table to an honest number and a short presentation.
I opened a dataset. What now?
A router for an unfamiliar table. What is one row, is there a clock, do entities repeat, what shape is the target, and which symptom are you seeing. Each answer sends you to the page and the worked example that deal with it.
Prediction boundaries: what would actually be known?
The one question that removes most leakage before it happens. At the exact moment the prediction would be made, which columns exist? Sort every column into before, at, after, derived-from-the-future, or unsure, and do it before building a feature.
A research workflow for open-ended questions
The stages between receiving an unfamiliar table with a vague question and presenting an honest number. Interrogate, agree the question, fix the protocol, get a baseline, try one idea, attack it, stop.
Analysing a new dataset in a fixed block of time
How to pace a few hours with an unfamiliar table so that there is a defensible result at every stage. Orient, baseline, look, one real model, diagnose, improve only where the diagnostics say to, try to break it, write it up. With the stopping rules that keep the last hour from being a rescue.
First-pass dataset interrogation
A table can be wrong in four places, plus one domain-specific way. One pandas line for each, run on a real file, and what to do with the three findings that come back wrong.
Questions to resolve before modelling
The ninety-second summary to put to whoever set the question, the decisions they will push you for, and the two questions to ask back before any feature is built.
Pre-analysis protocol
Five lines written down before any feature is built and not changed afterwards. Target, signal, split, score, baseline. With the template, the code that encodes it, and two filled-in examples.
Presenting a quantitative analysis
Seven slides, one sentence and one number or chart each, in an order that mirrors the work. Plus the questions a careful reader will ask and how to have the answers ready.
The last three slides
A fill-in template for the end of an analysis. Slide 5 is every signal scored in one table, slide 6 is one sentence, one figure and one uncertainty line on whether it holds, slide 7 is the same three questions every time. With the code for both figures and the table.