Auditability as Architecture: What Finance Learned the Hard Way

Append-only records. Immutable history. The ability to prove what the state was at a particular moment without trusting anyone's memory.

These are familiar ideas around here. What is less widely appreciated is that traditional regulated finance needs exactly the same properties, for entirely different reasons, and has spent several decades mostly not building them.

The result is worth studying, because the failure mode is instructive.

The question that exposes everything

A bank examiner reviews a sample of closed cases, points at one, and asks: why was this alert closed?

At a firm with well-designed systems, this is a ten-minute answer. The case record contains the alert, the data exactly as it stood on that day, the version of the rule that fired, the reasoning the analyst was shown, and who approved it.

At most firms, this is a six-week reconstruction across four systems. The customer data has been updated forty times since. The threshold was changed at some point but nobody can say when or by whom. The analyst who closed it left last year.

The bank is not being evasive. The information genuinely no longer exists.

Why it no longer exists

One line of SQL, repeated a few million times:

UPDATE customers SET risk_rating = 'medium' WHERE id = ...;

Every in-place update destroys the only record of what a past decision was based on. The system is doing exactly what it was designed to do — maintain current state efficiently — and current state is the wrong thing to maintain when your obligation is to explain the past.

The correct model is bitemporal. Every record carries two time ranges: when the fact was true in the world, and separately, when this system believed it. Regulators ask about the second one, because the question is what you knew, not what turned out to be true.

customer_risk_rating
  customer_id, rating
  valid_from, valid_to        -- true in the world
  recorded_from, recorded_to  -- believed by us

Reconstruction becomes a query. Without it, reconstruction is impossible at any price.

Rules with no history

The second failure is more startling once you notice it.

Detection thresholds — the numbers that determine which transactions get flagged — usually live in a configuration table that an operations user can edit through an admin interface. No approval workflow. No version history. No test.

The same institution requires a code review, a test suite and a deployment record before an engineer can change a constant in a service.

The fix is to treat controls as versioned, immutable artefacts. Version 7 never changes; a threshold adjustment produces version 8 with its own approval record and effective-from date. Every alert stores which control version produced it, so the chain from regulation to rule to alert to decision is traversable in both directions.

There is a second benefit that pays for the work on its own: you can replay a proposed rule change against historical data before deploying it. A threshold adjustment that looks conservative can add forty percent to the alert queue, which the operations team absorbs by reviewing faster — a quality collapse nobody decided on. Replay turns that into a number in a report.

Note that replay only works with historical reference data, not current. Today's risk ratings were themselves influenced by alerts the old rule generated.

Where AI complicates it

Compliance analysts spend around forty minutes per alert, of which perhaps ninety seconds is actual judgement. The rest is gathering context across systems. Models are genuinely good at that gathering, and the forty minutes drops to about eight.

But this introduces a new auditability problem. If a model contributed to a recommendation, the record has to include the prompt, the retrieved context, the model version, the parameters and the raw output — otherwise the decision is unexplainable in a new way.

And you cannot pin a floating model alias in a control path. If the provider updates the model, the control changed without approval. Explicit version pinning, with a held-out evaluation set acting as a promotion gate on every version change, is the workable pattern.

The broader principle: constrain models to advisory roles where a human approves the output. That constraint is what keeps the arrangement explainable.

The portable lesson

Systems that must justify themselves to an adversarial reviewer years later need properties that ordinary systems do not: immutability, explicit versioning of both data and rules, and reconstruction rather than logging.

These properties cost roughly three engineering months if designed in at the start. They cannot be added afterwards at any price, because the required history was discarded every time a record was updated.

That asymmetry — cheap upfront, impossible later — is the whole argument, and it is one this community understood earlier than most.

Full technical treatment: Financial Compliance Software: Architecture for AI-Native Controls. We build these systems — custom software development.

Frequently Asked Questions

Why can't banks explain decisions made years ago?

Because their systems store current state and update records in place. Explaining a past decision requires the world as it appeared then — the risk rating then, the threshold then — and that history was destroyed by every in-place update.

What is bitemporal data modelling?

Every record carries two time ranges: when the fact was true in the world, and when the system believed it. Regulators ask about the second, since the question is what the firm knew at decision time, not what later proved true.

Why should detection rules be versioned like code?

Because examiners ask what a threshold was on a specific date and who approved it, and because a versioned rule can be replayed against historical data to measure its effect before deployment. Rules in an editable config table can answer neither question.

What breaks when you replay a rule against current data?

Circularity. Current risk ratings were themselves influenced by alerts the existing rule generated, so testing a new rule against them produces a meaningless result. Replay requires reference data as it stood at the time.

What must be recorded when a model contributes to a decision?

The prompt, the retrieved context, the model version identifier, the parameters and the raw output, all linked to the decision record. Treating inference as ephemeral creates the same unexplainability as not recording which rule version fired.

Why pin model versions explicitly?

Because a floating alias in a control path means the control can change without approval when the provider updates the model. Pinning plus a held-out evaluation set as a promotion gate keeps changes deliberate and reviewable.