Live-DB revalidation — v0.6.0 + companion-CTE cap fix
Running featurizer's tests against the three live triage databases
(dirtyduck, chicago311, donorschoose) after the
v0.6.0 set-based pre-aggregation release — 2026-07-08.
Headline. v0.6.0 shipped with a latent SQL-generation bug:
an over-63-byte set-based companion-CTE name was hash-capped with a
~
separator and emitted unquoted. PostgreSQL parses ~ as an operator, so
the whole all-aggregator config failed with syntax error at or near "~" on any
data whose categorical column names are long (e.g. the food-inspections
transition_matrix_summary(inspection_type) stream). The DB-free tests and the
pre-v0.6.0 live stress test never hit it — only running the tests against the live DBs did.
Fixed in planner.py (cap separator folded to _ for the bare
CTE identifier), with a DB-free regression guard.1 · Test results
| tier | command | result |
|---|---|---|
| DB-free | just test-fast | 571 passed (added the cap regression test; 7 viz skips need the [viz] extra) |
| Integration (seeded) | just test-integration | 365 passed, 0 failed — was 8 failed (food-inspections) before the fix |
2 · Live full-cohort materialization (measured)
Each config materialized through featurizer's grouped/TEMP-shard path
(to_dataframe(connection=…)) at one as-of date, read-only, in a child process with a
360 s timeout. narrow = the triage config's own primitives; all-agg = the full
67-aggregator default set with identity; wide = all-agg × 14 transformers.
| database | variant | features | rows | seconds | status |
|---|---|---|---|---|---|
| dirtyduck | narrow | 147 | 22169 | 3.4 | materialized |
| dirtyduck | all-agg | 272 | 22169 | 356.8 | materialized |
| dirtyduck | wide | — | — | ln of negative | |
| chicago311 | narrow | 28 | 30654 | 1.0 | materialized |
| chicago311 | all-agg | 191 | 30654 | 10.1 | materialized |
| chicago311 | wide | — | — | ln of zero | |
| donorschoose | narrow | 175 | 3000 | 1.5 | materialized |
| donorschoose | all-agg | 1063 | 3000 | 281.1 | materialized |
| donorschoose | wide | — | — | timeout >360s | |
Read this honestly. The
~ fix means all-agg now
materializes on every live DB (it errored or would not finish before) — a real correctness
win. But the advanced tier is still heavy on the larger streams: dirtyduck all-agg is
356.8 s and donorschoose all-agg 281.1 s — far above ADR-0010's synthetic
<120 s acceptance target. That target was never actually executed against the real
dirtyduck DB (the config SyntaxError'd), so v0.6.0's real-data acceptance was synthetic-only.
chicago311 all-agg is a healthy 10.1 s. The wide variant is unrelated to v0.6.0 — it
trips unguarded ln/sqrt on negative/zero inputs (applying every
transformer to every signed feature), the known math-hardening follow-up.3 · Per-database detail
dirtyduck →
Food-inspection failure risk. facilities ← inspections (as-of). 272 all-agg features; the config that surfaced the ~ bug.
chicago311 →
311 service-request modeling. 191 all-agg features; the fast case (10.1 s).
donorschoose →
Classroom-project funding. 1063 all-agg features over 3k rows.
4 · Follow-ups (surfaced, not yet actioned)
- Advanced-tier performance on real data. dirtyduck / donorschoose all-agg run 5–6 min vs chicago311's 10 s. Worth profiling which families dominate (the non-migrated correlated special-config families, or the categorical/sequence tier over long per-group event streams) and whether ADR-0010's acceptance should be re-stated against real DBs, not synthetic.
- Unguarded
ln/sqrttransformers. They raise on negative/zero inputs. Either guard them (ln(nullif(greatest(x,0),0))→ NULL) or document that math transformers are opt-in only for non-negative features. Needs a semantic decision (possibly an ADR).