Your stability data needs a contract

Stability studies produce measurements from a multitude of instrument families - SEC and other types of chromatography, light scattering, subvisible particle analysis, potency assays and many more. Each assay tends to carry its own export format and its very own way of being subtly wrong. A dashboard built on silently coerced data looks fine up to the audit.

Data contracts at the door

In AssayVault, an export format carries an explicit pandera schema: column names, dtypes, units, plausible value ranges, identifier patterns. A file either satisfies its contract or it does not get in. There is no “read_csv and hope”, because the failure modes of hope are quiet ones: a potency column parsed as string, a date read day-first, a sample identifier whose suffix encodes the storage condition and gets accidentally truncated.

The interesting design choice: what happens to rows that fail? They are not dropped but they also do not stop the load. They go to a quarantine table with the reason attached alongside a load audit which records what came in, what passed, and what did not.

The name is borrowed honestly: pharmaceutical warehouses keep physical quarantine areas where goods that fail incoming inspection wait, labeled, for a decision. Nothing goes in the dumpster, because the rejection is itself a record the business needs.

Quarantined rows serve as a signal

In the reference build, 68,154 synthetic raw rows arrived at the door, 68,152 were loaded and exactly 2 were quarantined. That number is reported on purpose: the quarantine rate is itself a data-quality metric. A sudden jump means an instrument’s export format changed or someone edited a template by hand which is important information.

The rule: validate explicitly, reject loudly, keep what you reject. A silently coerced value is a defect with a delay timer whereas a quarantined row is a documented decision.

In a regulated context, such an approach buys the thing that matters most: a traceable path from raw instrument export to every reported number including the numbers that did not make it. The correctness of what comes after the door is a separate discipline - AssayVault proves its central fact table by computing it twice - but no downstream proof can rescue data that entered wrong.