Prove your pipeline twice

Every data pipeline has tests. The awkward question is what those tests would actually catch. If the same head that wrote the transformation wrote the expectations, both can share a wrong assumption and pass together in green harmony - a very bad place to be when the number at the end is a shelf-life claim.

Two implementations that must agree

AssayVault computes its central fact table twice using two independent routes: once in Python, once in SQL. Neither is a port of the other, they are two readings of the same specification. A test proves row-level equality between them, including the out-of-specification flags which is the column where a disagreement matters most.

This is the classic defense against single-implementation blindness. A misread rounding rule or an off-by-one in a date window would have to be misread the same way twice and occur in two languages in order to survive. Most mistakes are not that coordinated.

Accounting reached the same conclusion centuries ago: double-entry bookkeeping records every movement twice, and at the end of the month, the ledgers must balance. The duplication is not waste. It is the audit.

Believe in your tests but mutate the constants

Agreement between implementations still cannot tell you whether your test suite has teeth. A mutation harness can be handy in doing so. It deliberately breaks the pipeline, for instance by swapping in a few wrong regulatory constants or mixing up rules, then asserts that the suite goes red. If an incorrect constant does not result in red tests, the suite has a blind spot exactly where the domain risk is highest.

Airport security works on the very same principle: inspectors routinely send test objects through the scanner, because a checkpoint that has never been probed with a known threat is a checkpoint of unknown quality. The mutation harness is that inspector with a bag of test objects, the test suite is the scanner.

Coverage tells you which lines ran. Mutation testing tells you which mistakes would be caught. For regulatory logic, both numbers should be considered.

The rule: a test suite is an instrument, and instruments get qualified: a standard approach from the lab that also applies to code. Prove the pipeline against an independent implementation, then prove the tests against deliberate defects.

So why doesn’t every piece of software employ a mutation harness? The answer lies within the cost which is real - you maintain the logic twice and the harness besides. In a quality-critical domain such as pharma where a wrong constant cannot be accepted, the second implementation is still the cheapest reviewer you will ever hire: it never gets tired, and it never assumes you were right.