triage-pg vs DSSG triage
A dimension-by-dimension, honest comparison. triage-pg is an opinionated, intentionally breaking fork — this page says exactly what that buys and what it costs. If you want the original, full-featured, battle-tested toolkit, use dssg/triage.
Dimension by dimension
| Dimension | triage-pg | DSSG triage |
|---|---|---|
| Problem types | Four, behind one problem_type switch: classification,
regression-as-ranking, pure regression, and fully runnable survival
(scikit-survival estimators; C-index in PL/pgSQL matching
concordance_index_censored to 1e-9). ADR-0010/0026 |
Binary classification (the early-warning problem), deeply refined. |
| Feature engine | featurizer: PostgreSQL-native Deep Feature Synthesis over an entity-relationship graph; point-in-time-correct via as-of joins; per-as_of_date cost measured constant-to-sub-linear. ADR-0008, docs/featurizer-scale.md | Collate: SQL aggregation generation (intervals × functions × groups) — proven, more manual, no relationship graph. |
| Evaluation substrate | PL/pgSQL over the predictions table — metrics need only (entity, score, label), so nothing leaves the database; dashboards read views directly. Per-prediction-date rows + windowed rollups. ADR-0007 | pandas/sklearn in Python workers, persisted to the results schema. |
| Subset evaluations | Parity: named cohort slices, the subset re-ranked as its own population. migration 0015 | Yes (subset queries + subset_hash) — the feature triage-pg re-implemented. |
| Fairness | SQL group-bys over protected_groups: selection_rate, precision, TPR,
FPR, FDR, FNR, FOR, NPV — each with disparity vs a reference group and a per-row
τ verdict; config-driven ingestion; the Aequitas fairness tree as an interactive
wizard. No Aequitas dependency. migration 0014, docs/fairness.md |
The Aequitas library itself: the full metric catalog plus significance testing and the fairness-determination report (pandas<2 environments). |
| Model selection (audition) | In-database: distance-from-best, max regret, regret-next-time, and all 8
selection rules as audition_pick() — dashboard tab +
triage audition/leaderboard CLI. The Python module is
retired. migrations 0005/0013 |
Auditioner: the same concepts with matplotlib plots and an iterative
thresholding workflow in notebooks. |
| Postmodeling | Dissolved, not dropped: crosstabs + error trees computed once from the matrix
(triage postmodel), persisted to PG, read by the model card + CLI;
calibration deciles; top-k list overlap (Jaccard/Spearman); per-entity β·x
contributions for linear models. ADR-0011, migrations 0016/0017,
docs/postmodeling.md |
A standalone ~8.8k-LOC module: crosstabs, error analysis, plus SHAP-style and prototype tooling triage-pg deliberately defers. |
| Predictions | Append-only, timestamped, quarterly-partitioned — prediction history from day
one; "current" = latest scored_at. ADR-0006 |
Upserted per (model, entity, date); history requires care. |
| Production monitoring | First-class: scheduled triage score + drift (PSI/KS at scipy parity),
volume, calibration, realized-outcome tracking — SQL over the append-only table, no
daemon; a Monitoring dashboard view. ADR-0027, migration 0012 |
Not built in (predict-forward exists; monitoring is external). |
| Caching & provenance | Guix-style derivation DAG: every artifact hashed over its complete input closure (source pins + engine versions + parents); cache-hits verify the artifact file still exists; GC by reachability. ADRs 0013–0017 | replace=True-style flags + experiment hashes; provenance partly implicit. |
| Matrices | Parquet (Polars/pyarrow), local FS or S3 through one storage seam. | CSV(.gz) / HDF5 via a Store hierarchy. |
| Orchestration | In-process locally; one AWS Batch job per experiment in cloud. No rq / multiprocess grid (deliberate). ADR-0005/0020 | SingleThreaded / MultiCore / rq-based distributed experiments. |
| Multi-tenancy | One database per project + a registry control plane; per-project routing + a project
switcher; triage project create/drop. ADR-0002/0025 |
One results schema per database; multi-project is by convention. |
| UIs | A read dashboard (experiments ▸ groups ▸ models, live SSE run rail) + a write webapp (validated submissions) + OIDC auth — all thin, no business logic. ADR-0012/0024/0028 | None bundled (notebooks are the surface). |
| Tooling & stack | Python 3.12, uv, ruff, psycopg3, SQLAlchemy 2 (alembic only), pydantic v2, typer, loguru; tests spin their own PostgreSQL. | Older, broader dependency surface; the ecosystem many deployments already run. |
What DSSG triage has that triage-pg deliberately doesn't
| Battle testing | Years of production use across real agencies, an active maintainer team at CMU, and a community. triage-pg is young and single-maintainer. |
| Aequitas in full | The complete bias-metric catalog, statistical significance testing, and the fairness-determination report. triage-pg reimplements the operational core (8 metrics + disparities + τ + the tree) and stops there — deliberately. recorded waiver on ADR-0007: Aequitas is pandas-2-incompatible, so triage-pg's SQL is validated against hand-computed fixtures instead |
| Deep interpretability | SHAP-style explanations, prototype/nearest-neighbor identification. Deferred to a companion effort — not in v1. ADR-0011 |
| Distributed grids | rq/multicore parallel training for very large model grids. triage-pg keeps the grid×split loop serial per experiment and scales by running experiments as separate Batch jobs. ADR-0020 |
| Matrix format options | CSV/HDF5 matrices and the associated tooling; triage-pg is Parquet-only. |
| In-place upgrades | DSSG triage migrates old results schemas forward. triage-pg is greenfield-only: no migration path from existing triage databases. ADR-0001 |
Migration notes for DSSG users
There is no automated migration. triage-pg starts fresh
(ADR-0001): new repo, new schema, no upstream PRs. What follows is a translation guide,
not a converter.
| You knew… | In triage-pg… |
|---|---|
experiment config v8 (experiment.yaml) |
experiment.yaml: same spirit (cohort/label queries with
{as_of_date}/{label_timespan}, temporal_config, grid),
but feature_aggregations (Collate) becomes a featurizer
entity-relationship graph under feature_config, sources must be declared
(sources:, ADR-0014), and problem_type is explicit. |
triage experiment config.yaml |
triage run experiment.yaml --project-path …; inspect with
triage leaderboard/models/audition/model show or the dashboard. |
audition_config.yaml + Auditioner plots |
Gone — triage audition <experiment-hash> prints the ranking, all
8 rule picks, and the audition-vs-leaderboard divergence from the SQL views the
dashboard also renders. |
bias_audit_config / Aequitas |
bias_config: (query + parameter + ref_groups + τ + intervention) —
ingestion and metrics run inside the pipeline; the fairness tree drives the reading.
docs/fairness.md |
| postmodeling notebooks (crosstabs, error analysis) | triage postmodel crosstabs|error-tree|compare — persisted, then read
from the model card or SQL. docs/postmodeling.md |
predict_forward |
triage score <model_id> [date] (defaults to today, cron-ready) —
plus the monitoring views over the append-only history. |
Claims trace to the ADRs (docs/adr/), the conformance audit
(docs/adr-conformance.md), and the measured featurizer scaling notes
(docs/featurizer-scale.md). triage-pg preserves triage's MIT license and
copyright; the ideas at its heart — temporal CV, leakage discipline, model governance, the
fairness tree — are DSSG/DSaPP's. Self-contained page: inline CSS only.