{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# Dataset day\n\n**09:00-09:40** rows, key, missing, wrong \u00b7 **10:00** call \u00b7 **by 11:00** first number \u00b7 **13:00** stop modelling, build slides \u00b7 **~14:00** present.\n\nNo theory before the first number."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Paste kit.py into this cell, or keep kit.py next to this notebook:\nfrom kit import *\n\nPATH = \"data.csv\"   # the file you were given\nT = \"ts\"            # time column\nE = \"asset\"         # entity column, or None for a single series"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "df = load(PATH, T, E)\ndf.head()"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "df.dtypes"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "info = triage(df, T, E)   # daily equities: triage(df, T, E, max_gap=pd.Timedelta(days=4))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "**A count looks odd?** Print the rows: `show_rows(df, <mask>, [T, E])`. Two rows per key means duplicates. Many rows on one timestamp means one event."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "df, log = clean(df, T, E)   # rules: dedupe keep last, drop bad prints, drop zero-volume bars"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## The 10:00 call\n1. Finding:\n2. Finding:\n3. Finding:\n\n**The question in my own words:**\n\n**My plan:** target / signal / score / split + embargo / baseline\n\n**Presentation time:**"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "when_known(list(df.columns))"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Plan: five blanks\n1. **Target:**\n2. **Signal** (known before or at the trade):\n3. **Score:** per-timestamp Spearman IC, mean, t, hit rate\n4. **Split:** build before ___, check from ___, embargo ___\n5. **Baseline:** last bar's return"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "df[\"ret\"] = df[\"close\"] / df[\"open\"] - 1          # target: return over the bar\ndf[\"prev_ret\"] = lag(df, \"ret\", T, E)             # baseline, blanked across gaps\nSIGNALS = [\"prev_ret\"]                              # add your signal column(s)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "build, check = split(df, T, \"2024-01-01\", \"1D\")   # set the cut date"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "tab = ic_table({\"build\": build, \"check\": check}, SIGNALS, \"ret\", T)   # slide 5"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# one hypothesis: does the signal add anything beyond the baseline?\nincremental_t(build, SIGNALS[-1], [\"prev_ret\"], \"ret\", T)\nincremental_t(check, SIGNALS[-1], [\"prev_ret\"], \"ret\", T)\nics = ic_series(df, SIGNALS[-1], \"ret\", T)\nby_period(ics)   # stable or decaying?"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# can you trade it? edge per trade against the cost\nq = quantile_returns(check, SIGNALS[-1], \"ret\", T)   # bps per bar\nif E: top_overlap(check, SIGNALS[-1], T, E)\nif \"spread_bps\" in df: print(\"median spread (bps):\", df[\"spread_bps\"].median())"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "save_rolling_ic(ics, 24 * 7, \"fig_rolling_ic.png\")   # window in bars\nsave_bars(q, \"fig_quantiles.png\", \"check period\", \"bps per bar\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Slides\n1. Question\n2. Data and what I removed (counts from `log`)\n3. Plan (five blanks)\n4. Signal against baseline, build and check\n5. Everything I tried: \"I tried N things, X held\" (`tab`)\n6. One result: a sentence, a figure, one uncertainty line\n7. Could it be fake? Can you trade it? What would I run next?\n\n**Answering questions:** claim, then number, then comparison."
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}